# Question
Title: How do I setup an OpenShift route that will allow both 80 and 443 traffic to my app?
I'd like to setup a route that will pass both 80 and 443 traffic to the same service in OpenShift. When I try to create a second route it errors out. I want this to be in a yaml like would be used with `oc create -f route.yaml` so I can submit it to our IaC code repo.

# Answer
> 1 votes
You'll want to be sure your spec.tls section has the following two items to tell the router you want both secure and insecure traffic allowed in the edge.

```
apiVersion: route.openshift.io/v1
kind: Route
metadata:
  name: my-app-ui
  namespace: my-namespace
spec:
  host: my-host
  port:
    targetPort: http
  tls:
    insecureEdgeTerminationPolicy: Allow
    termination: edge
  to:
    kind: Service
    name: my-app-ui
    weight: 100
  wildcardPolicy: None

```

---
Tags: security, networking, openshift, encryption
---


# Question
Title: Should my zombie backend container be restarted when database is down?
I have a situation where I have a Postgres and a Node server which connects to it. I am using Kubernetes to deploy them.

If my Postgres container fails and stops working then its livenessProbe will restart him which will take some time, meanwhile, the server container is useless so the Server's readinessProbe should fail and Ingress stops routing traffic to it. If this happens what should be the behavior of the Server?

Some solutions I thought but was not sure about them:

1. Server's livenessProbe sees database down and restarts the container (doesn't make sense to me because it could generate a cascade effect, Postgres failure makes the Server's container restart)
2. Server keep trying to reconnect forever and becomes a useless running container while Postgres doesn't come back
3. Server tries to reconnect a limited number of times with exponential interval between them, after that it restarts

# Answer
> 1 votes
The standard option is #2, keep retrying the connection, possibly with slower intervals, but I'd have an upper limit to the longest interval. The liveness probe should only fail if that specific container is broken, not any of it's dependencies. If there are cases where your application is failing and won't self recover when the DB is restored, then the liveness probe should fail in that scenario, and I'd consider an init container to check for the availability of the DB before starting the full application.

There should be some logging and metrics enabled to make the DB failure visible so that you debug the correct thing. If there is a concern of too many retries impacting the network, then a service mesh with circuit breaker functionality may be a good option.

---
Tags: kubernetes, containers, microservices
---


# Question
Title: How to keep many servers (5000+) up-to-date?
Initially asked here: https://stackoverflow.com/questions/60674502/how-to-keep-many-servers-5000-up-to-date-with-git-and-its-rate-limits

The initial post:

---

We're making a php service that will run on many servers, think 5000+. We host our code on git (bitbucket). We wondered what the best way to keep the servers up-to-date would be.

We figured either post-commit hooks (but what would happen then if a few servers didn't receive the update notification?) or git fetching every minute with cron. We want to go with the cron way of doing things, since it isn't possible it would fail, even if a server would be offline (either turned off or disconnected from the network), it would still resolve itself eventually.

We're doing a fetch every minute, and then compare to see if it needs to pull, if so it pulls and runs the migration code.

We would like to run this every minute so that the servers will be synchronized with each other as soon as possible.

Now we wonder, what about rate limits? We're using bitbucket, and the rate limits are 60 000 requests per hour (so a 1000 per minute), which would limit us to 1000 servers max before we will get problems then?

But it also says, if we make a public repo, we can make unauthenticated calls, the limits of which go by IP rather than per user, so we won't run into any limits then no matter how many servers we will have. Downside is, we will have to encrypt the repo then, then on pull, decrypt it and copy over the decrypted files.

Is this the best way of handling this? It seems very unconventional. What is the standard or recommended way of handling this (if there is any)?

---

After having read the answers, we now think that having a group of servers that pull in any changes (from git). For readability, we're calling these the "gitpull-servers", and the 5000+ servers the "webservers".

The plan is to make the gitpull-servers fetch (and possibly pull) from git.

The webservers periodically query the gitpull-servers to see if there is an update, and if there is, the webservers will pull the update in either by downloading a tar/zip (requires more network) or by using rsync (requires more iops on the gitpull-servers).

We think this is a good solution, but none of us have any experience with deployments at this scale, so we aren't sure if we've overseen anything or if this could be done in any other (potentially better) way.

It would be really helpful to get feedback on this, as well as to know about the conventional or the recommended way of handling a problem like this at this scale.

# Answer
I don't think the proposed plan here makes much sense. Tools like Ansible are free and designed to do this with good, central management and logging.

* Put code in BitBucket.
* Make a Jenkins server (open source / free / low on resources / well documented and easy to use).
* Make BitBucket call Jenkins with a hook on change.
    + Or, if you feel like it, Jenkins can poll bit-bucket, but this is less efficient.
* Have Jenkins call ansible (also open source / free / low on resources / well documented and easy to use) in its pipeline.
* Ansible can target all 5000 servers with a dynamic inventory based on querying a database/file/API/whatever you have.
* This way, bit-bucket has almost no load, ansible does everything from one location, runs are recorded in Jenkins with info on each host.

You won't have any API / rate limits here and you'll easily be able to target more hosts, with no effort on your end, with the dynamic inventory.

This is industry standard tooling. You can also use Ansible to keep the servers up to date patch wise and all kinds of other stuff while you're at it, so it's a re-usable solution.
> 4 votes

---
Tags: git, deployment, continuous-deployment
---


# Question
Title: How to use n3dr with nexus 3 servers with self-signed certificates?
Is n3dr compatible with nexus 3 servers that use a self signed certificates? If so, how to access such repositories?

```
./n3dr-linux repositories -u xxxx -p xxxx-n https://xxxx -a
WARN[0000] ~/.n3dr.yaml does not exist or yaml is invalid
FATA[0000] Get https://xxxx/service/rest/v1/repositories: x509: certificate signed by unknown authority

```

\[UPDATE\]

Related issue on gihub: https://github.com/030/n3dr/issues/107

# Answer
> 1 votes
@titou10 created a pull request and solved it in version 3.3.4-rc1.

This is to inform that the issue has been solved.

https://github.com/030/n3dr

https://github.com/030/n3dr/tree/3.3.4-rc1

---
Tags: n3dr
---


# Question
Title: Conditionally stopping SQL replication during deployment
The project that i recently started working on uses a dacpac to deploy / update a database in MS SQL Server. The database is replicated, whenever the dacpac contains modifications to any table, we manually terminate the replication before continuing with the deployment. After the deployment, we run a script to start with a fresh replication. On the other hand, if only stored procedures have been modified, the replication can continue during the deployment.

The manual intervention is annoying me. Has anyone already solved this issue in an Azure Devops Release Pipeline?

The fallback is to use SqlPackage.exe /action;deployreport, but that means I have to write a file to disk to get the diff and subsequently inspect that outcome.

# Answer
I eventually found the solution :)

In an Azure Devops Release pipelines I've created a Command Line task that executes `SqlPackage.exe /Action:DeployReport /p:DoNotAlterReplicatedObjects=false ...` which creates an XML file that contains info on which tables etc would be changed by a deployment. It's important to note that `DoNotAlterReplicatedObjects` should be set to false, otherwise SqlPackage will fail.

In a second task in the Azure Release pipeline, I analyze the generated DeployReport (e.g. the XML file) using Powershell and XPath\**. The Powershell task will create an boolean output variable that will indicate whether replication needs to be stopped or not. In my case, I analyze the XML and if a table is created/altered/dropped, the replication will be stopped. The syntax for setting the output variable is

```
Write-Host "##vso[task.setvariable variable=haltReplication;isOutput=true]$haltReplication"`

```

Third step is yet another Powershell script that will stop the replication. This third script will only run if `$haltReplication` evaulates to true. In the Control Options, the custom condition is `and(succeeded(), eq(variables['haltReplication.haltReplication'],'1'))`

\**The XML file generated by SqlPackage.exe contains namespaces, which makes working with XPath a bit cumbersome. The following SO answer helped me a lot https://stackoverflow.com/questions/36417189/ignore-namespace-in-xpath-as-this-can-be-dynamic?rq=1
> 1 votes

---
Tags: azure-devops, sql
---


title: What are Deis Workflow Alternatives?

text:
# Question
Title: What are Deis Workflow Alternatives?
We're looking to build an internal Heroku-like PaaS that supports clustering and a DFS. We found Deis Workflow that seems to do exactly what we're looking for, but the project has since been discontinued.

Are there any alternatives that are similar to Deis Workflow that achieve the same thing?

# Answer
According to the README of the deis/workflow project on GitHub, it is no longer maintained since March 1, 2018.

However, the README indicates that there is a fork, called Hephy Workflow. At the time of writing, this is the first time I heard about Deis and Hephy and I have not worked with these solutions.

Alternatives to these products are OpenShift, Google App, AWS Elastic Beanstalk, Azure App Service, Cloud Foundry, Apollo according to StackShare.

You mentioned that you are looking to build something **internal**. This would probably mean that native-PAAS like AWS Elastic Beanstalk, Azure App Service and Google App are not an option as these are only available on their platform.

One of the options could be OpenShift. This product has an enterprise and open-source version that can be installed on your on-premise or cloud servers. You could also decide to create a bunch of VMs on Google Cloud and install OpenShift on these servers.
> 1 votes

# Answer
Jelastic is definitely a good multi-cloud alternative PaaS to Heroku, OpenShift, CloudFoundry and others. The platform providers auto-clustering for multiple software stacks (app server and DBs) and supports different file storage options. In general, Jelastic project has two main pillars:

1. The automation software platform which can be installed on top of any infrastructure
2. The wide network of public cloud providers that offer managed Jelastic PaaS with pay-per-use model

There is a variety of use cases and white-papers with end users stories.

Disclaimer: I'm a co-founder of Jelastic project.
> 0 votes

---
Tags: docker, kubernetes, containers, orchestration, platform-as-a-service
---


# Question
Title: How can CaaS, PaaS, and FaaS users know if the operating system of their server environment is of the newest release?
Please consider this chart a moment; it shows the different hosting models common today:

---

From the chart I learn that CaaS, PaaS and FaaS are three hosting-models by which a user can't control the OS it gets (as the OS of its server environment - webserver, email server and whatever-else-server leaning on the internet).

But I wonder how can a client for a CaaS/PaaS/FaaS service actually know if the operating system of their server environment is **really of the newest version in every possible manner**.

I wonder what ways I have as a customer to ensure a CaaS/PaaS/FaaS company won't literally lie to me when they report "Your Debian is latest" (say Debian 9 or 10 (that should come out soon, or 11 or 12), or "Your ArchLinux is latest" and so on and so forth.

In all three I can automate everything above the OS with Ansible but that still doesn't make me quiet in my mind; I'm still "anxious" regarding my OS; I desire all its upgrades to be fully automated as well.

**What tools should I have in my environment** (a CLI terminal wouldn't necessarily be enough here by itself) to actually really know if they report the truth and not plain lying to me regarding the OS version (minor, major, and release)?

Although a bit old (say 4-8 years) OS can be stable in relation to all up2date software above it, I would still prefer to be up2date in everything, if circumstances allow it (till now I had 5$-30$ IaaS VPSs on DigitalOcean).

# Answer
> 1 votes
Well, to begin with, one of the key concepts behind the CaaS/PaaS/FaaS services is the offloading of the OS maintenance burden :) So it shouldn't even matter what version (or even what OS! - it could even be a proprietary one) the provider runs under the hood as long as the CaaS/PaaS/FaaS service performs according to the respective SLAs.

Which is, in a way, why at least some of these services are considered serverless \- what does OS release means without "the server"?

As for being at the latest release - I seriously doubt a large service provider will use the latest OS release for any offering with a production-level SLA. For the simple motive that by the time the quality of that release reaches a decent enough / production quality level (according to the provider's QA qualification criteria) it's no longer the latest one :) Yes, qualifying a 3rd party provided OS release in a (large) enterprise environment is no small matter, it can take months/years to complete.

A more recent patch / service pack level is more likely (but I still have doubts about "latest"). It's the classic stability vs quality conflict. Look at any new OS and the list of serious bugs it gathers in the first few months after release.

Another consideration: running the latest OS release *might* not even be as critical in a CaaS/PaaS/FaaS context as one might think in some cases, depending on the service being provided and its implementation. For example, who cares about **OS system call vulnerabilities** if the PaaS runtime/execution sandbox doesn't allow the hosted applications to make system calls? See, for example, the Google App Engine's Python Sandbox. Or, listening ports vulnerabilities if the system isn't even directly exposed to the outside world?

Note: In some cases even the language support lag for the CaaS/PaaS/FaaS service could be significant and IMHO it's conceivable that a similar reasoning could potentially explain OS lagging.

# Answer
> 0 votes
You can use special release monitoring tools like https://release-monitoring.org/ to track software stacks updates and notify subscribers or trigger auto update actions.

---
Tags: linux, terminology, security, shell-script, platform-as-a-service
---


# Question
Title: Port forwarding for Windows Server like SSH port forwarding
I have a Windows server 2019 that have many VMs instances (running using HyperV). Each VMs has its own private IP and has some apps running inside and expose via ports.

It looks like:

* VM1: running app nodejs port 3000, private ip: 10.10.10.1
* VM2: running app php port 3001, private ip: 10.10.10.2

Now I want to setup a secure connection so my colleagues can access the apps inside each vm without expose them to internet, how can I achieve it?

If I have a Ubuntu Server and SSH I will use dynamic port forwarding like this article

https://www.digitalocean.com/community/tutorials/how-to-route-web-traffic-securely-without-a-vpn-using-a-socks-tunnel

Does Windows have something alternative?

I have did a lot of tutorials but it seem it does not work for me.

# Answer
You can use a ssh reverse proxy from any terminal (that has the ssh bin) to communicate with your applications.

For instance try this on the client side:

```
 ssh -NL 3000:localhost:3000 root@10.10.10.1

```

*This command won't print anything (except prompting for ssh key's password) and won't stop by itself. To kill the reverse proxy you can do ctrl + C*

Now you should be able to curl your application the following way on the client:

```
curl 127.0.0.1:3000

```
> 1 votes

---
Tags: vpn
---


# Question
Title: Necessity of mirroring git repos for small team considering switching to Azure
# Question:

> Are there any benefits to mirroring our git repos outside of Azure devops for a small (~7) team of developers who all work in the same office?

# Background Information

I'm a developer who also wears the "DevOps" hat when he has time to do so (all the time right now). We're looking at potentially moving from our self hosted and maintained instance of Gitlab to MS Azure DevOps. My manager has stated he's interested in keeping a "forest of git nodes" for a couple different reasons.

* One was access control (don't allow certain people to access node 'A')
* One was that we don't want all of our code in Azure for some reason
* One is so that we have extra backups of our code base
* One is because "that's what git was made for, several distributed remotes. That's how Linux does it"
* If Azure goes down we can still commit to the Gitlab repo and push that up later.

I think it's easy to tell what MY opinion of us doing this is, but I'd like others input on pros vs cons of going about structuring our git repo's in a way like stated above. From what I can tell he's thinking about the repo structure in one of the two ways shown below:

My reasoning for wanting to not bother with the mirrored repo's are the following:

* The entire teams works in the same office, so there's no distribution of associates over multiple geographic regions. I'm also not aware that it would be changing anytime soon.
* Adds complexity to the development environment.
* Increases the possibility of code being out of date on one (or both) remotes. (For some background, in times before I was hired source code was getting lost.)
* Access Control is built into both Azure DevOps AND Gitlab, so we're not getting anything new by doing that.
* MS Backup and data retention are MUCH MORE comprehensive than what we have going on currently. Moving to them would INCREASE our probability of recovering data in case of emergency.
* Relatedly, MS SLA is much higher than anything that I can guarantee (99.9%). Switching to Azure gets us that SLA for "free".
* Since we're using Git as VCS, we're already working in a "distributed" manner. We don't need a mirror / second remote to push code to when we have the code locally on our machines anyway. We would simply push when connectivity was restored.
* Admittedly selfish but, I don't want to have to worry about keeping all these tools (Nexus, Gitlab, OpenProject, all of those in containers on an unsupported Linux server that I don't have direct access to) up to date and patched. I'd rather be doing what I was hired to do, which is program.

With all of the above being said, I'm still open to having my mind changed. I'm new-ish to the development field and am very aware that I could be missing something crucial on why we would need a mirror or multiple remotes.

# Answer
> 1 votes
As you have realised Git is decentralised by design.

# Using LAN repos

If Azure DevOps goes down, or your office internet connection, you can still use LAN repos. These could be **simple file shares**. You can also use **emails** to share commits; nothing I've tried but Git has native support for it). I have however pushed and pulled commits directly from and to my colleagues computers using file shares (because our online Git provider back then was extremely bad and slow (and the repos were many GB).

If you don't have automatic sync (which you can setup using pipelines/CI) between Azure DevOps and GitLab then the most up-to-date repos will reside on the developers computers which makes it easy to push the repo to a new location when needed (simply add the new remote and click push). However, if some repo hasn't been used in a long time it'll probably not exist on any developers computer and you'll therefor not be able to access it.

If you want to be disaster-ready I'd suggest hosting an **Azure DevOps Server** instance on your LAN that's synced with the cloud. Azure DevOps Server is capable of processing the very same yaml pipelines as the online version. You'd be protected against much more than just Azure DevOps Services outages.

The maintenance costs of having Azure DevOps Server as a backup for a small team shouldn't be much different than having GitLab as backup.

*I'm not recommending these solutions, rather I'm just discussing them.*

*You might want to have a look at the Azure DevOps Proxy Server.*

---
Tags: git, azure-devops, gitlab, distributed-systems
---


# Question
Title: Terraform's local state (the 2 tfstate files) contain very sensitive info
Part of my IaC that describes my systems for Azure is CosmosDB, and Keyvaults. Terraform happily created everything and I was able to use an output from CosmosDB (db keys) as an input to the Keyvault (as a keyvault secret) so my db key was safely stored.

Then I poked through my terraform.tfstate and sure enough, ALL the cosmosdb keys (pri and sec, rw and ro) keys are in there plaintext, as is the plaintext of the vaults secrets.

So this leaves somewhat of a quandry here - These state files are apparently integral and they must be maintained (as far as I can tell) but I really dont like the idea of checking them into any repo. I guess theres no real good solution here other than locally-maintained repos that are redundant as possible?

# Answer
> 3 votes
Do not store your state in anything besides remote backend. Even if you store it in a remote backend the secrets will appear in a plain text again, this is an ongoing Terraform limitation.

In our case we are using AWS S3 backend ( although we have multi cloud environments ), we've got those key benefits out of the box:

* Encryption in transit and at rest.
* Access policy configuration.
* Durability and availability close to the maximum.
* Locking capability.
* Versioning.

Please do not go with `locally-maintained repos that are redundant as possible`.

# Answer
> 0 votes
The solution is to do it right:

* **Do not** check them into source control.
* **Do** use remote state.
* **Do not** allow user access to production remote state.
* **Do** enforce continuous deployment for production.

---
Tags: terraform, secret-management
---


# Question
Title: Should AWS arn values be treated as secrets?
I have a configuration file that contains AWS resource ARNs stored in a git repo. This includes some ARNs of AWS IAM role resources. Should these be treated as if they are a secret value, and not stored in a repository?

# Answer
> 15 votes
Well, from a security point of view, it's never bad to give people less information.

But, as long as you have sensible security policies in AWS, there is no reason that an arn has to be considered secret.

For example, if you know the arn of the role I use to access s3 from my EC2 instances, you can't do anything with that information unless I give you permission to assume that role from one of your roles. Between accounts, this is a very explicit and bidirectional action; so it will not happen by accident/without you knowing.

Losing the credentials to an IAM user could let people assume roles and would be terrible, but them knowing the arn alone does nothing for them.

---
Tags: amazon-web-services, aws-iam, secret-management
---


# Question
Title: EC2 EBS Free Tier - Is Storage Reusable?
Can I delete an EC2 instance and start off from scratch on my EBS free tier usage? For instance, I was running a Red Hat server with 10 GB of EBS being used. However, I deleted that EC2 within a few hours. I then spun up another machine with 8 GB of EBS. Do I now have 22 GB of storage left for the month or just 12 GB, after combining the 8 with the 10?

I assume, given that AWS doesn't reference hourly usage on their free tier category for EBS that the 30 GB is just a flat fee. Again, I simply don't understand if I can replete an instance, then start from scratch to zero EBS on a new instance.

# Answer
While I can't find it explicitly written anywhere, I do know that textbooks frequently reference that you can use the free tier to avoid costs, and that they frequently have you create and destroy instances between chapters. So, I find it **extremely** hard to believe that you would get billed for deleting and creating new EBS volumes.

So, I believe that if you create a 30GB one, destroy it, and create another, you should be fine. You should definitely make sure to remove the volumes and any related snapshots though just to be safe.

They definitely don't seem to explicitly confirm that though... e.g. in text like this:

> Notable offers for the first 12 months following your AWS account sign-up include some level of free usage for Amazon Elastic Compute Cloud (EC2), Amazon Elastic Block Store (EBS), Amazon S3, Amazon Relational Database Services (RDS), and many other useful services.

They use the free tier to get customers hooked. They want you to like it and not be frustrated so you're more likely to use the real services and so that you may advocate them in your businesses. So, by that logic, I similarly find it hard to believe they would "trick you" by billing you in this situation without making it more obvious. That would be bad for sales.
> 2 votes

---
Tags: amazon-ec2, architecture
---


# Question
Title: How can I build a Linux VM that I can use to test Rest API calls in a lab?
I work in Enterprise CloudOps Support and the products we support use Rest API for notifications. Frequently we get requests to help troubleshoot solutions like Pager Duty, PagerTree, ServiceNow, etc. Obviously I can't and won't signup for trials of these products, nor do I need to know them. However, I *do* need to know how Rest API works and how to set it up. I've been researching it and so far all I've found is this MS DevOps write-up: https://microsoft.github.io/PartsUnlimitedMRP/pandp/200.1x-PandP-LocustTest.html

However, as of yet I can't seem to make sense of some of the steps. I thought there must be a better way, so here I am. Ultimately what I'm after is a VM that I can build in a lab and use for testing Rest API calls that relay notifications, alerts, etc.

# Answer
You can test basically any rest api call from postman, a free app that runs perfectly well in windows or Linux (or Mac I’m sure). You don’t need anything special.

If you want to use Linux specific tools, you can sign up for AWS free tier and make a Linux box in minutes following any of their tutorials (it will be free for 1 year).

You can also use docker on your computer (including on windows). Docker will be suitable for running most non GUI based or web GUI based Linux apps. I see a locust image here for example - https://docs.locust.io/en/stable/running-locust-docker.html.
> 1 votes

---
Tags: api
---


# Question
Title: Istio: envoy proxy to https container in same pod
I've got a container that runs a https service on port 9999. The service is as follows:

```
apiVersion: v1
kind: Service
metadata:
  labels:
    role: pingfederate2
  name: pingfederate2
spec:
  ports:
    - name: https
      port: 443
      targetPort: 9999
  selector:
    role: pingfederate2

```

The envoy log generated when going through the gateway is

```
[2020-03-19T08:42:37.356Z] "- - -" 0 - "-" "-" 2046 7 11 - "-" "-" "-" "-" "127.0.0.1:9999" inbound|443|https|pingfederate2.default.svc.cluster.local 127.0.0.1:39954 10.25.132.9:9999 10.25.131.133:56890 outbound_.443_._.pingfederate2.default.svc.cluster.local -

```

To me, it looks like envoy is trying to communicate with the pod over http, not https. Is there a way to force Envoy to use https to locally communicate with the container? I could be barking up the wrong tree though.

# Answer
Changing the destination rule from ISTIO\_MUTUAL to SIMPLE fixed this issue. There's obviously a problem with my config with the mTLS.
> 1 votes

---
Tags: kubernetes, istio
---


title: Creating Keycloak Realm via Ansible

text:
# Question
Title: Creating Keycloak Realm via Ansible
Currently there is no module to create a Keycloak realm with Ansible. There is a PR but it is stucked since half a year or so. So I though, there is a Keycloak REST API (https://www.keycloak.org/docs-api/9.0/rest-api/index.html#\_realms\_admin\_resource) and I could call it with `uri` to create a Token and then send a JSON RealmRepresentation via REST. So I tried this:

```
- name: "Create Token for service Keycloak"
  uri:
    url: "https://keycloak-server/auth/realms/master/protocol/openid-connect/token"
    method: POST
    body_format: form-urlencoded
    body:
      username: "admin"
      password: "password"
      grant_type: "password"
      client_id: "admin-cli"
  register: keycloak_token

- name: "Create Realm for service Keycloak"
  uri:
    url: "https://keycloak-server/auth/"
    method: POST
    src: "realm.json"
    remote_src: "no"
    headers:
      Content-type: "application/json"
      Accept: "application/json"
      Authorization: "Bearer {{ keycloak_token.json.access_token }}"
  register: keycloak_realm_create

```

The token is available within `keycloak_token.json.access_token` and it is correct.

As `realm.json` I tested a minimal JSON (because all attributes in RealmRepresentation are marked as optional):

```
{
  "id": "myrealm",
  "realm": "myrealm",
  "displayName": "My Realm",
  "enabled": true,
  "sslRequired": "external",
  "registrationAllowed": false,
  "loginWithEmailAllowed": true,
  "duplicateEmailsAllowed": false,
  "resetPasswordAllowed": false,
  "editUsernameAllowed": false,
  "bruteForceProtected": true
}

```

or a complete JSON export from Keycloak via WebUI (export realm).

But in any case the following error happens:

```
fatal: [localhost]: FAILED! => {
  "changed": false,
  "connection": "close",
  "content": "{\"error\":\"RESTEASY003065: Cannot consume content type\"}",
  "content_length": "55",
  "content_type": "application/json",
  "elapsed": 0,
  "json": {"error": "RESTEASY003065: Cannot consume content type"},
  "msg": "Status code was 415 and not [200]: HTTP Error 415: Unsupported Media Type",
  "redirected": false,
  "status": 415,
  "url": "https://keycloak-server/auth/"
}

```

The documentation says it must be a `POST` to `/` and it consumes `application/json`. So, I'm not sure - is the URL correct? If I try `https://keycloak-server/auth/realms/`, I'm getting a 404. If I remove the `Content-type` \- I'm getting a 406 with "Not acceptable". If I remove Accepts too, I'm getting a HTML page.

So - what is wrong?

# Answer
> 4 votes
As @JSapkota mentioned the URL was wrong. It must be `https://keycloak-server/auth/admin/realms`.

```
- name: "Create Realm for service Keycloak"
  uri:
    url: "https://keycloak-server/auth/admin/realms"
    method: POST
    src: "realm.json"
    remote_src: "no"
    status_code:
     - 201
    headers:
      Content-type: "application/json"
      Accept: "application/json"
      Authorization: "Bearer {{ keycloak_token.json.access_token }}"
  register: keycloak_realm_create

```

Take care about the return status code. It is "201 created" and the `uri` module must know, that this is fine.

This works one time. Because in the next run the REST API responses with 409 the `uri` module fails. To be idempotent, you need to check first, if the realm exists. This can be done via `GET /auth/admin/realms/{{ keycloak_realm_name }}` and check the return code, if it is 404 or 200. If it is 404, then you make a "POST" otherwise a "PUT". So, this a complete example:

```
- name: "Set facts"
  set_fact:
    keycloak_admin_user: "admin"
    keycloak_admin_pass: "password"
    keycloak_base_url: "https://keycloak.server"
    keycloak_realm_name: "myrealm"
    keycloak_realm_data_file: "realm.json"

- name: "Create Token for service Keycloak"
  uri:
    url: "{{ keycloak_base_url }}/auth/realms/master/protocol/openid-connect/token"
    method: POST
    body_format: form-urlencoded
    body:
      username: "{{ keycloak_admin_user }}"
      password: "{{ keycloak_admin_pass }}"
      grant_type: "password"
      client_id: "admin-cli"
  register: keycloak_token

- name: "Find out, if Realm {{ keycloak_realm_name }} for service Keycloak exists"
  uri:
    url: "{{ keycloak_base_url }}/auth/admin/realms/{{ keycloak_realm_name }}"
    method: GET
    status_code:
     - 200
     - 404
    headers:
      Accept: "application/json"
      Authorization: "Bearer {{ keycloak_token.json.access_token }}"
  register: keycloak_realm_exists

- name: "Create Realm {{ keycloak_realm_name }} for service Keycloak"
  uri:
    url: "{{ keycloak_base_url }}/auth/admin/realms"
    method: POST
    src: "{{ keycloak_realm_data_file }}"
    remote_src: "no"
    status_code:
     - 201
    headers:
      Content-type: "application/json"
      Accept: "application/json"
      Authorization: "Bearer {{ keycloak_token.json.access_token }}"
  register: keycloak_realm_create
  when: "keycloak_realm_exists.status == 404"

- name: "Update Realm {{ keycloak_realm_name }} for service Keycloak"
  uri:
    url: "{{ keycloak_base_url }}/auth/admin/realms/{{ keycloak_realm_name }}"
    method: PUT
    src: "{{ keycloak_realm_data_file }}"
    remote_src: "no"
    status_code:
     - 204
    headers:
      Content-type: "application/json"
      Accept: "application/json"
      Authorization: "Bearer {{ keycloak_token.json.access_token }}"
  register: keycloak_realm_create
  when: "keycloak_realm_exists.status == 200"

```

Also I had the problem, that after every request the Keycloak server responses with a 403 when I don't refresh the token before the next call to Keycloak. But I think, this depends on Realm token settings of the admin realm.

---
Tags: ansible, keycloak
---


# Question
Title: How to get server url from kub dashboard?
I have access to kubernetes dashboard on aws with token. And now I want to connect with kubectl, how to get server url for kubectl config?

# Answer
> 1 votes
When running an EKS cluster, the best way to configure your `~/.kube/config` is by using the SDK's update-kubeconfig:

```
aws eks update-kubeconfig --name your-cluster-name

```

Make sure to configure your credentials as `AWS_*` environment variables. Or even better use something like aws-vault.

---
Tags: kubernetes, kubectl
---


title: Common config in Kubernetes ConfigMap

text:
# Question
Title: Common config in Kubernetes ConfigMap
Kubernetes already provides a way to manage configuration with ConfigMap.

However, I have a question/problem here.

If I have multiple applications with different needs deployed in Kubernetes, all these deployments might share and access some common config variables. Is it possible for ConfigMap to use a common config variable?

# Answer
It is not possible for a ConfigMap to reuse or import other ConfigMaps.

You can, however, have many deployments reuse the same ConfigMap. In this case, you should extract out the common configuration to a single ConfigMap and reuse it across deployments while still allowing each individual deployment to have their own specific config.
> 1 votes

---
Tags: kubernetes, configuration-management
---


# Question
Title: Error when creating new recource using backend AWS s3
When I try create a new resource using `terraform apply` (Terraform init was success) received an error:

```
data.terraform_remote_state.environment_data: Refreshing state...

Error: Unable to find remote state

  on backend.tf line 15, in data "terraform_remote_state" "project_data":
  15: data "terraform_remote_state" "project_data" {

No stored state was found for the given workspace in the given backend.

Error: Unable to find remote state

  on backend.tf line 25, in data "terraform_remote_state" "environment_data":
  25: data "terraform_remote_state" "environment_data" {

No stored state was found for the given workspace in the given backend.

```

And here is my point, it's a new resource the Terraform should be create the new tfstate in using my backend configuration in this case AWS S3.

I have this same recipe in use to other resources running...

Anyone can clarify me?

# Answer
> 1 votes
Guys I discovered what happened. It was necessary to apply these two resources that I informed (project\_data and environment\_data), but they were in the above directories and I was not aware.

Something like this:

```
Application
|
|-Environment
  |-environment_data.tf
|
|- project_data.tf

```

---
Tags: terraform, amazon-s3
---


# Question
Title: Unable to Trigger Downstream Jenkins Job Based on the Promoted Builds Plugin
I have a request from a user who is currently using the Promoted Builds plugin for Jenkins. This user would like Job\_B to be triggered when a build from Job\_A has been promoted.

In Manage Jenkins \> Configure System, there are three "Promotion Levels" defined.

When I navigate to Job\_B \> Config, check the "Build when another project is promoted", and enter the name of Job\_A in the Job Name box. However, the "Promotion" drop-down does not populate with any of the three promotion levels.

Job\_A is a Freestyle job, while Job\_A is a Pipeline. I thought this was the source of the issue, but when creating a test FreeStyle job, I still wasn't able to see anything populate the Promotion drop-down. Any idea what I may be missing?

# Answer
Job chaining is a CI/CD anti-pattern and one of the big drivers behind developing pipelines. If the next step in the equation is to do more work, just make it a stage in the pipeline. If you need separation for whatever reason, make the freestyle job a pipeline job and add a post-\>success step.

https://jenkins.io/doc/book/pipeline/syntax/#post
> 1 votes

---
Tags: jenkins
---


# Question
Title: Azure Pipelines: Exclude folders using Azure App Service Deploy
I have an **Azure DevOps Pipeline** which includes an **Azure App Service Deploy** task (`AzureRmWebAppDeployment`) for deploying an **ASP.NET Core 3.1** project:

```
- task: AzureRmWebAppDeployment@4
  inputs:
    ConnectionType: 'AzureRM'
    azureSubscription: 'Azure Subscription(01234567-89ab-cdef-0123-456789abcdef)'
    appType: 'webApp'
    WebAppName: 'MyStagingSite'
    packageForLinux: '$(Build.ArtifactStagingDirectory)/**/*.zip'
    enableCustomDeployment: true
    DeploymentType: 'webDeploy'
    enableXmlTransform: false
    enableXmlVariableSubstitution: true
    RemoveAdditionalFilesFlag: false

```

The **Azure App Service** destination, however, contains files in several pre-established folders which are managed independent of the continuous delivery process. We would like to use the *Remove additional files at destination* flag (`RemoveAdditionalFilesFlag`) while leaving those folders intact.

> *Disclaimer:* I don't consider this a best practice and, in the future, we will be moving these files off to a separate storage location. Until then, I'd like to find a solution that will work resolve this.

In **Visual Studio 2019**, we achieve this by excluding those files from our publishing process using the `MsDeploySkipRules` in our `csproj` file:

```
<Project Sdk="Microsoft.NET.Sdk.Web">
  …
  <ItemGroup>
    <MsDeploySkipRules Include="CustomSkipFolder">
      <ObjectName>dirPath</ObjectName>
      <AbsolutePath>wwwroot\\Uploads</AbsolutePath>
    </MsDeploySkipRules>
</Project>

```

This approach works well for Visual Studio. Those rules do not appear to be honored by the `AzureRmWebAppDeployment` task, however, even when using the "Web Deploy" deployment method (`DeploymentType`).

Is there a way to honor the `MsDeploySkipRules` when using the `AzureRmWebAppDeployment` task? If not, if there a way to provide a list of folders which should be skipped or ignored as part of the deployment process? Or, alternatively, if there another task that will permit one of these options?

# Answer
> 3 votes
As @yang-shen-msft notes on Stack Overflow, there doesn't appear to be a way to honor the `MSDeploySkipRules` defined in the `csproj` file. Instead, files and folders can be skipped by defining the *Additional Arguments* (`AdditionalArguments`) parameter of the *Azure App Service Deploy* (`AzureRmWebAppDeployment`) task, as discussed in another Stack Overflow answer.

Since there doesn't appear to be any official documentation for the `-skip` rules, and the `MSDeploy.exe` documentation that Azure Pipelines references is out-of-date, the following provides additional details.

First, it's useful to recognize that when you deploy a project via Visual Studio, it's simply taking the `MSDeploySkipRules` configured in the `csproj` file and adding them to it's internal call of `msdeploy.exe` as `-skip` rules. So, given the following rule defined in the `csproj`:

```
<Project Sdk="Microsoft.NET.Sdk.Web">
  …
  <ItemGroup>
    <MsDeploySkipRules Include="CustomSkipFolder">
      <ObjectName>dirPath</ObjectName>
      <AbsolutePath>wwwroot\\Uploads</AbsolutePath>
    </MsDeploySkipRules>
  </ItemGroup>
</Project>

```

The `-skip` rule is interpreted as:

```
msdeploy.exe -skip:objectName=dirPath,absolutePath=wwwroot\\Uploads

```

Translating this to the *Azure App Service Deploy* (`AzureRmWebAppDeployment`) task, the resulting `yml` might look like:

```
- task: AzureRmWebAppDeployment@4
  inputs:
    ConnectionType: 'AzureRM'
    azureSubscription: 'Azure Subscription'
    appType: 'webApp'
    WebAppName: 'MyStagingSite'
    packageForLinux: '$(Build.ArtifactStagingDirectory)/**/*.zip'
    enableCustomDeployment: true
    DeploymentType: 'webDeploy'
    enableXmlTransform: false
    enableXmlVariableSubstitution: true
    RemoveAdditionalFilesFlag: true
    AdditionalArguments: '-skip:objectName=dirPath,absolutePath=wwwroot\\Uploads'

```

> *Note:* Multiple `-skip` rules can be defined on the same `msdeploy.exe` call.

Unfortunately, as mentioned above, there doesn't *appear* to be any official, first-party documentation for the `-skip` rules on `msdeploy.exe`. The 2014 documentation acknowledges them, and provides two examples, but doesn't expand on the options. That said, way back in 2012, @richard-szalay wrote a useful article, "Demystifying MSDeploy skip rules", which provides useful direction here for anyone requiring additional control over their `-skip` rules.

---
Tags: azure-devops, azure, azure-pipelines
---


# Question
Title: How to `terraform destroy` an `aws_instance` without also destroying its `aws_ebs_volume` (separate resource)
```
$ terraform -version

```

> ```
> Terraform v0.12.21
> + provider.aws v2.34.0
>
> ```

Given a terraform configuration for one set of single `aws_instance`, `aws_ebs_volume` and `aws_volume_attachment` resources, each named, say,`foo`;

How would one `terraform destroy -target=aws_instance.foo` *without* terraform also destroying the `aws_ebs_volume.foo` as the resulting plan insists that it will?

* The aws console reports *`Delete on termination`* for the volume's block device to be **`false`**;
* I've tried to `terraform destroy -target=aws_volume_attachement.foo` first;
... no-change.
* I've tried removing the `aws_volume_attachment` from the configuration, `terraform apply` it and *then* `terraform destroy -target=aws_instance.foo`;
... didn't help either

Somehow terraform really wants to destroy the `aws_ebs_volume` resource together with the `aws_instance` resource.

I, otoh, would very much like to preserve that `aws_ebs_volume` for later attachment to a re-created `aws_instance`...

Can it be done?

# Answer
This may be what you are doing already (in which case this sounds like a bug), yet it seems just based on the limited information in the question that you may be forgetting a step: Run `plan -destroy -target=... -target=...` first. The `terraform destroy` command will try to operate on changes listed in the `.plan` file.

The following example assumes a Terraform module named `foo`, with 3 EBS volumes and attachments named as: `dev-sdb`, `dev-sdf`, `dev-sdg`. All resources are assumed to have `count = local.enabled ? 1 : 0`. So this is the more advanced case you may encounter where each resource would have to be targeted with the array syntax `[0]`.

1. First create a targeted plan file with `terraform plan -destroy -target=...`. Do this before trying to run `terraform destroy`

```
terraform plan -destroy \
               -target=module.foo.aws_volume_attachment.dev-sdb-attachments[0] \
               -target=module.foo.aws_volume_attachment.dev-sdf-attachments[0] \
               -target=module.foo.aws_volume_attachment.dev-sdg-attachments[0] \
               -target=module.foo.aws_instance.foo[0]  \
               -target=module.foo.aws_iam_instance_profile.foo[0] \
               -out=intermediate.plan

```
2. Run `terraform destroy`, optionally also using `-target=...` for safety precautions if you wish:

```
terraform destroy -target=module.foo.aws_volume_attachment.dev-sdb-attachments[0] \
                  -target=module.foo.aws_volume_attachment.dev-sdf-attachments[0] \
                  -target=module.foo.aws_volume_attachment.dev-sdg-attachments[0] \
                  -target=module.foo.aws_instance.foo[0]  \
                  -target=module.foo.aws_iam_instance_profile.foo[0] \
                  intermediate.plan

```

If you need this to be unattended for automation, add `-auto-approve` flag to the `terraform destroy` command. Otherwise, it assumes an interactive terminal session and will prompt before destroying any resources.
> 2 votes

# Answer
A workaround to your issue is to create a snapshot of the volume(s) before deleting them. The snapshot will live on after the volume is deleted and can be recreated if necessary.

> After you no longer need an Amazon EBS volume, you can delete it. After deletion, its data is gone and the volume can't be attached to any instance. However, before deletion, you can store a snapshot of the volume, which you can use to re-create the volume later.

**Reference**

EBS Deleting Volume
> 0 votes

---
Tags: amazon-web-services, terraform
---


# Question
Title: Is it possible to downgrade Jenkins container plugin thought CLI?
I have a job that creates a container from Jenkins image. I would like to add to the script one more option: Currently, it's installing the image with DSL plugin version 1.77. is there a command I can run from CLI to downgrade the plugin version to 1.57?

# Answer
> 1 votes
Check out Config as Code for installing plugins.

If you're using the jenkins image, just use an older version of the image that has the proper plugin version

---
Tags: jenkins-plugins
---


# Question
Title: How to deploy a Kubernetes service using NodePort on AWS?
I have created a cluster on AWS EC2 using `kops` consisting of a master node and two worker nodes, all with public IPv4 assigned.

Now, I want to create a deployment with a service using NodePort to expose the application to the public.

After having created the service, I retrieve the following information, showing that it correctly identified my three pods:

```
nlykkei:~/projects/k8s-examples$ kubectl describe svc hello-svc
Name:                     hello-svc
Namespace:                default
Labels:                   app=hello
Annotations:              kubectl.kubernetes.io/last-applied-configuration:
                            {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"hello"},"name":"hello-svc","namespace":"default"},"spec"...
Selector:                 app=hello-world
Type:                     NodePort
IP:                       100.69.62.27
Port:                     <unset>  8080/TCP
TargetPort:               8080/TCP
NodePort:                 <unset>  30001/TCP
Endpoints:                100.96.1.5:8080,100.96.2.3:8080,100.96.2.4:8080
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

```

However, when I try to visit any of my public IPv4's on port 30001, I get no response from the server. I have already created a Security Group allowing all ingress traffic to port 30001 for all of the instances.

Everything works with Docker Desktop for Mac, and here I notice the following service field not present in the output above:

```
LoadBalancer Ingress:     localhost

```

I've already studied https://kubernetes.io/docs/concepts/services-networking/service/, and think that NodePort should serve my needs?

Any help is appreciated!

# Answer
More often than not you should not be using NodePort.

You're in AWS so you can consume the AWS Cloud Provider. Try using `ingress` resources or `LoadBalancer` service types. NLBs can do TCP/UDP port passthrough if that's what you need.

**Edit:** After further review, it looks like your just exposing a web server (I noted port `8080`). `NodePort` is 100% not what you want. Look at Ingress
> 1 votes

---
Tags: docker, amazon-web-services, kubernetes
---


title: Challenges of using AWS Lambda chain

text:
# Question
Title: Challenges of using AWS Lambda chain
**Context:** I have a pipeline of 6 lambda functions (chained together), triggered by an SNS notification which is generated whenever a file lands on S3. This pipeline essentially takes the file(few GBs), filters it (Spark cluster is created to run the job, then deleted at the end), and inserts it into a DB. Lambdas are orchestrating the flow.

**Issues:** If one Lambda fails, the chain breaks hence no effective failure handling. Secondly, we experience timeouts if a polling/computation takes longer than 5 minutes, so no effective retry. It takes a long time to test/debug an issue if a lambda fails. Also there is no visibility, say for example how many jobs failed and how many passed? we dont know. Getting a bunch of SNS notifications on email is not very effective/helpful. If the chain breaks, we cannot perform cleanup operations like deleting SPark cluster or housekeeping steps.

**My Questions:** Is AWS Step Functions a good choice for solving the above issues? When would you not use a Step Function service? If you cannot invoke Step Function through SNS, then what would be the best way to call it whenever a file lands on S3? Feel free to share any other approach to easily and effectively tackle this usecase.

# Answer
I am using lambda in response to objects appearing on S3 buckets. For me in my particular use case, there are around 600 triggers a day. I have written our lambda functions to use their stdout as their log file, writing all key events to standard out in a semi-structured textual sentence mixed with key/value pairs for data values, and then feed the cloudwatch logs into splunk.

Here is what I have found...

1. Lambdas will (occasionally) be triggered multiple times for the same object.
2. Lambdas will (sometimes) take noticeably longer to run than normal.
3. To see / diagnose issues caused by concurrent invocations that interfere with each other, the AWS Web Console doesn't do it. You need a tool like splunk.

However, does the value of this work provide enough revenue to cover the costs of using ECS(Elastic Container Service) for your process management? If you are able to run an ECS container, you could trigger the lambda on S3 object creation to create a message on an SQS message queue. Your ECS container is listening to your queue and when a message arrives, processes it just like within your lambda. Its output to trigger the next stage is to pop a message onto the SQS queue handling the next task.

This will solve a few problems for you, such as escaping the 5 minute maximum for Lambdas, and creating an easy way to retry failures.

However, it will create a few new issues too.

* Creating and deploying docker images is more work than editing lambda on the AWS console.
* Infrastructure costs are more fixed in nature, rather than proportional to workload.
> 1 votes

# Answer
Yes, Step Functions seems like a good fit for your use case.

Step Functions standard workflows guarantees exactly-once execution and provides a step-by-step history of your workflows, hence making it easy to debug and giving you the visibility that you are looking for. You can additionally use retry/catch functionality and SNS service integration.

You can use EventBridge to trigger Step Functions from S3.

Resources: https://docs.aws.amazon.com/eventbridge/latest/userguide/eventbridge-tutorial-ecs.html (change target to Step Functions instead of ECS) https://docs.aws.amazon.com/step-functions/latest/dg/concepts-error-handling.html https://docs.aws.amazon.com/step-functions/latest/dg/connect-sns.html

I work for AWS but all opinions are my own.
> 1 votes

# Answer
I'm still getting into AWS Lambda so I'm not sure if it will help you but I started using a lambda performance monitoring tool for my functions and it seemed to be working well. I get a message if something breaks in 60 seconds (+ daily reports but that's less important) and it helped me figure out what broke.

Now, I don't know if you can indeed use Step Functions to fix your issues but I do know you can set up events that trigger a Lambda function whenever a file gets to S3. Here's a link to s3 docs that might help.
> 0 votes

---
Tags: amazon-web-services, aws-lambda, serverless
---


# Question
Title: Terraform error Reference to “count” in non-counted context for resources not necessarily data
here's my part of my code that I am having issues with. Unfortunately I could not understand the relationship between the last answer, despite being almost what I need, and my situation.

```
data "aws_internet_gateway" "zyz_igw" {
  depends_on = [aws_internet_gateway.this]

    filter {
    name   = "attachment.vpc-id"
    values = aws_vpc.this[count.index]
    #values = [aws_vpc.this[count.index]]   #based on https://www.terraform.io/docs/providers/oci/guides/filters.html
  }

  filter {
    name   = "tag:Name"
    values = ["ZYZ_Mirror_ZYZ-tp-zyz-IGW"]
  }
}

```

The error I am getting is the following:

```
Error: Reference to "count" in non-counted context

  on modules/xyz/main.tf line 292, in data "aws_internet_gateway" "zyz_igw":
 292:     values = aws_vpc.this[count.index]

The "count" object can be used only in "resource" and "data" blocks, and only
when the "count" argument is set.

```

I get confused since I believe my issue may be with the resource and not sure how to change the data. I also saw something related to 'for\_each' to use that instead of 'count' on 0.12. But the in the very beginning of my code I have the normal 'aws\_vpc' resource:

```
resource "aws_vpc" "this" {
  count = var.zyz_create_vpc ? 1 : 0

```

# Answer
As far as your error goes.... `count` is a reserved name. It's only accessible when the block you're writing has a `count` set!

You just need to specify the `count` of `aws_internet_gateway` data resources you're trying to get. Your code implies there is more than one.

```
data "aws_internet_gateway" "zyz_igw" {
   count = 1000000000
   ...

```
> 2 votes

---
Tags: amazon-web-services, terraform, data
---


# Question
Title: Terraform - how to modify a existing S3 bucket that was created from module?
I already have my S3 bucket called `foo-name` created and existing with terraform. I want to modify the existing S3 bucket and a policy. But when I do this, it seems to want to add a new s3 bucket instead. I imagine `s3_website` in `resource "aws_s3_bucket" "s3_bucket"` is incorrect. But I don't know what I would use in place of that because of the way the S3 bucket was created.

How do I get it to modify my existing S3 bucket that was created in `module s3_website {}`?

```
provider "aws" {
  alias = "east1"
  region = "us-east-1"
}

module "acm_cert" {
  source = "foo-git"

  cluster = var.cluster

  domain      = var.website_domain
  dns_zone    = var.zone_name

  tags       = var.tags
  extra_tags = var.extra_tags

   providers = {
    aws = aws.east1
  }
}

module s3_website {
  source = "foo-git"

  cluster        = var.cluster
  env            = var.tags.Environment
  app            = var.tags.Application
  website_domain = var.website_domain
  s3_prefix      = var.s3_prefix

  certificate_arn = module.acm_cert.certificate_arn
  error_document = "index.html"
  index_document = "index.html"

  custom_error_response = [
    {
        "error_code"         = 404
        "response_code"      = 200
        "response_page_path" = "/index.html"
    },
    {
        "error_code"         = 403
        "response_code"      = 200
        "response_page_path" = "/index.html"
    },
  ]

  tags = var.tags
}

data "aws_route53_zone" "base" {
  name = var.zone_name
}

resource "aws_route53_record" "this" {
  zone_id = data.aws_route53_zone.base.zone_id
  name    = var.website_domain
  type    = "A"

  alias {
    name                   = module.s3_website.domain_name
    zone_id                = module.s3_website.zone_id
    evaluate_target_health = false
  }
}

resource "aws_s3_bucket" "s3_website" {
  bucket = "foo-name"
  policy = <<POLICY
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "PublicReadGetObject",
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:GETObject",
      "Resource": "arn:aws:s3:::foo-name/*"
    }
  ]
}
POLICY
}

```

# Answer
> 2 votes
If you already have your S3 bucket recognized by Terraform and it was created by the code you pasted, you schould not be having any diffculties if the operation could be done in place.

There are a few important MUST KNOW things about how TF handle resources. MUST KNOW because in the end of the day it's infrastructure :) Some of the parameters are immutable, although it might not seem that intuitive changing the resource identifier would trigger delete/create scenario.

Could you paste your TF plan output ?

# Answer
> 1 votes
You'll have to modify the underlying module to make changes here.

Be sure you're aware of the impact this will have on other modules using your `s3_website` module. You might want to start using module source versioning.

---
Tags: amazon-web-services, terraform
---


# Question
Title: Upload data into docker container from docker volume
I wish to create a volume and mount it to a docker container in order to collect some artifacts, and I want to reuse these artifacts into another docker container later on. Can i just mount the same volume to the new container and upload the artifacts? Is there an upload command for this?

# Answer
You can mount a volume e.g. at `/app/artifacts` then set up your application to place the artifacts on that folder.

```
docker run -d \
-v artifacts-vol:/app/artifacts \
artifact-creator:latest

```

then after that, you can mount the same volume to a separate container and specify on the new container to read the artifacts from the volume location

```
docker run -d \
-v artifacts-vol:/app/artifacts \
artifact-reader:latest

```
> 1 votes

---
Tags: docker, volume
---


title: Set job start timeout in Gitlab CI

text:
# Question
Title: Set job start timeout in Gitlab CI
My pipeline consists of very long jobs that build docker images by compiling a set of packages; all of them require ~ 3 hours on my runner.

I set up my CI so that only one job can run at a time, in order to reduce the disk I/O overhead. But in this way many jobs fail, supposedly because they hit a job start timeout which seems to be bout 1h, with this message:

> There has been a timeout failure or the job got stuck. Check your timeout limits or try again

I have not been able to find how to increase the job start timeout, since all the documentation I found is relative to the job duration timeout.

So I’d need help with this, thanks. I'm using `GitLab CE 12.8.1` and `Gitlab Runner 11.5.1`.

# Answer
> 5 votes
There are three types of timeouts in `Gitlab CI`:

* `Project` timeout
* `Runner` timeout
* `Jobs` timeout

## Project `timeout`

According to GitLab docs:

> Timeout defines the maximum amount of time in minutes that a job is able run. This is configurable under your project’s **Settings \> CI/CD \> General pipelines settings**. The default value is 60 minutes. Decrease the time limit if you want to impose a hard limit on your jobs’ running time or increase it otherwise. In any case, if the job surpasses the threshold, it is marked as failed

## Runner `timeout`

According to GitLab docs:

> For each `Runner` you can specify a *maximum job timeout*.

`Runner` timeout settings are defined in runner's edit page

## `Jobs` timeout

GitLab CI/CD Pipeline Configuration Reference | GitLab

`Job`'s `timeout` allows you to configure a timeout for a specific job. For example:

```
build:
  script: build.sh
  timeout: 3 hours 30 minutes

test:
  script: rspec
  timeout: 3h 30m

```

# Precedence of different types of timeout

The `job`-level `timeout` can exceed the `project`-level timeout but can not exceed the Runner-specific timeout.

If `runner` timeout smaller than `project` defined timeout, will take the precedence.

## Examples of precedence of `timeout` directive

See Configuring GitLab Runners | GitLab

```
name: **Example 1 - Runner timeout bigger than project timeout**
  project_timeout: 2h
  runner_timeout: 24h
  job_timeout: 4h
  resulting_timeout: 4h

name: **Example 2 - Runner timeout not configured**
  project_timeout: 2h
  job_timeout: 4h
  resulting_timeout: 4h

name: **Example 3 - Runner AND job timeout are not configured**
  project_timeout: 24h
  resulting_timeout: 24h

name: **Example 4 - Runner timeout smaller than project timeout**
  project_timeout: 2h
  runner_timeout: 30m
  resulting_timeout: 30m

name: **Example 5 - Runner timeout smaller than Project timeout, Job timeout is bigger than Runner timeout**
  project_timeout: 2h
  runner_timeout: 30m
  job_timeout: 1h
  resulting_timeout: 30m

```

---
Tags: gitlab, gitlab-ci-runner
---


title: Mount Azure FileShare using terraform

text:
# Question
Title: Mount Azure FileShare using terraform
What is a good way to mount an Azure File Storage automatically during provisioning on VMs?

There is a Microsoft doc describing how a file share can be mounted in general. This requires that Azure CLI is installed and a user is logged in.

As an alternative, I have created a working setup using terraform working on local host:

First, the existing storage account needs to be defined:

```
data "azurerm_storage_account" "storage-account" {
  name                = "storageaccount"
  resource_group_name = "RG-storage-account"
}

```

When a VM is provisioned, the share is mounted by a remote execution provisionier:

```
resource "azurerm_virtual_machine" "vm" {
  # ...
  # stuff

  provisioner "remote-exec" {
    inline = [
      "sudo mkdir -p /mnt/${data.azurerm_storage_account.storage-accoun.name}/share",
      "sudo mount -t cifs //${data.azurerm_storage_account.storage-account.name}.file.core.windows.net/share /mnt/${data.azurerm_storage_account.storage-account.name}/share -o vers=3.0,dir_mode=0755,file_mode=0755,serverino,username=${data.azurerm_storage_account.storage-account.name},password=${data.azurerm_storage_account.storage-account.primary_access_key}",
    ]
  }
}

```

Similar it is also possible to inject the mount to `/etc/fstab`, also by using cloud init.

However, after applying, the key (accessed as `primary_access_key`) remains in the terraform state. Is this considered safe? Is there a way to delete the key from the state? Also, when using cloud-init, the key will remain in the custom data of the VM.

Is there another safe way to mount data on the VM without storing the keys?

# Answer
> 2 votes
Recently I've read a good book about Terraform that I think answers your question (Terraform Up & Running 2nd edition. ISBN-13: 978-1492046905. ISBN-10: 1492046906).

Basically it says that all data in Terraform state file is stored in plain text, including sensitive data. The author says that at the moment of the writing (May 2019) there is no real solution for this, but workarounds like encryption and IAM to control user access based on policies (the book talks about AWS, so you have to find the Azure equivalent and prevent unauthorized access to the state file).

You can find more details on Chapter 3, pages 78 and 79. There is also an open issue at Github about this: https://github.com/hashicorp/terraform/issues/516.

---
Tags: terraform, azure
---


# Question
Title: What is the AWS user permission that allows attaching and detaching IAM Roles to instances?
My coworker is trying to attach IAM roles to EC2 instances and doesn’t have permissions. I’m trying to work out which permissions to give him.

My question is: **What is the AWS user permission that allows attaching and detaching IAM Roles to instances?**

# Answer
> 5 votes
From Granting a User Permissions to Pass a Role to an AWS Service:

> To pass a role (and its permissions) to an AWS service, a user must have permissions to *pass the role* to the service. This helps administrators ensure that only approved users can configure a service with a role that grants permissions. To allow a user to pass a role to an AWS service, you must grant the `PassRole` permission to the user's IAM user, role, or group.
>
> When a user passes a role ARN as a parameter to any API that uses the role to assign permissions to the service, the service checks whether that user has the `iam:PassRole` permission. To limit the user to passing only approved roles, you can filter the `iam:PassRole` permission with the `Resources` element of the IAM policy statement.

Is this what you're looking for?

An example from the above-metioned page:

> Example 1
>
> Imagine that you want to grant a user the ability to pass any of an approved set of roles to the Amazon EC2 service upon launching an instance. You need three elements:
>
> * An IAM *permissions policy* attached to the role that determines what the role can do. Scope permissions to only the actions that the role needs to perform, and to only the resources that the role needs for those actions. You can use AWS managed or customer-created IAM permissions policy.
>
> ```
> {
>     "Version": "2012-10-17",
>     "Statement": {
>         "Effect": "Allow",
>         "Action": [ "A list of the permissions the role is allowed to use" ],
>         "Resource": [ "A list of the resources the role is allowed to access" ]
>     }
> }
>
> ```
> * A *trust policy* for the role that allows the service to assume the role. For example, you could attach the following trust policy to the role with the `UpdateAssumeRolePolicy` action. This trust policy allows Amazon EC2 to use the role and the permissions attached to the role.
>
> ```
> {
>     "Version": "2012-10-17",
>     "Statement": {
>         "Sid": "TrustPolicyStatementThatAllowsEC2ServiceToAssumeTheAttachedRole",
>         "Effect": "Allow",
>         "Principal": { "Service": "ec2.amazonaws.com" },
>        "Action": "sts:AssumeRole"
>     }
> }
>
> ```
> * An IAM *permissions policy* attached to the IAM user that allows the user to pass only those policies that are approved. `iam:PassRole` usually is accompanied by `iam:GetRole` so that the user can get the details of the role to be passed. In this example, the user can pass only roles that exist in the specified account with names that begin with `EC2-roles-for-XYZ-`:
>
> ```
> {
>     "Version": "2012-10-17",
>     "Statement": [{
>         "Effect": "Allow",
>         "Action": [
>             "iam:GetRole",
>             "iam:PassRole"
>         ],
>         "Resource": "arn:aws:iam::<account-id>:role/EC2-roles-for-XYZ-*"
>     }]
> }
>
> ```
>
> Now the user can start an Amazon EC2 instance with an assigned role. Applications running on the instance can access temporary credentials for the role through the instance profile metadata. The permission policies attached to the role determine what the instance can do.

The procedure(s) to attach a policy to a user/role (inlining it might work as well) are described in Attaching and Detaching IAM Policies:

> **To attach a managed policy (console)**
>
> 1. Sign in to the AWS Management Console and open the IAM console at https://console.aws.amazon.com/iam/.
> 2. In the navigation pane, choose **Policies**.
> 3. In the list of policies, select the check box next to the name of the policy to attach. You can use the **Filter** menu and the search box to filter the list of policies.
> 4. Choose **Policy actions**, and then choose **Attach**.
> 5. Select one or more identities to attach the policy to. You can use the **Filter** menu and the search box to filter the list of principal entities. After selecting the identities, choose **Attach policy**.

...

> **To embed an inline policy for a user or role (console)**
>
> 1. Sign in to the AWS Management Console and open the IAM console at https://console.aws.amazon.com/iam/.
> 2. In the navigation pane, choose **Users** or **Roles**.
> 3. In the list, choose the name of the user or role to embed a policy in.
> 4. Choose the **Permissions** tab.
> 5. Scroll to the bottom of the page and choose **Add inline policy**.
>
> **Note**
>
> You cannot embed an inline policy in a service-linked role in IAM. Because the linked service defines whether you can modify the permissions of the role, you might be able to add additional policies from the service console, API, or AWS CLI. To view the service-linked role documentation for a service, see AWS Services That Work with IAM and choose **Yes** in the **Service-Linked Role** column for your service.
> 6. Choose from the following methods to view the steps required to create your policy:
>
> * Import an Existing Managed Policy – You can import a managed policy within your account and then edit the policy to customize it to your specific requirements. A managed policy can be an AWS managed policy or a customer managed policy that you created previously.
>     * Create a Policy with the Visual Editor – You can construct a new policy from scratch in the visual editor. If you use the visual editor, you do not have to understand JSON syntax.
>     * Create a Policy on the JSON Tab – In the JSON tab, you can use **JSON** syntax to create a policy. You can type a new JSON policy document or paste an example policy.
> 7. After you create an inline policy, it is automatically embedded in your user or role.

# Answer
> 1 votes
To create (and assign) IAM roles and permissions that user needs to have Administrator level rights within the AWS Account. You can search `admin` in `Attach existing policies directly` window in the Add permissions to \<\< user name \>\>\` of the IAM Module accessible via the Console.

From there look at the different levels of Administrator accounts offered by default and you can further look at the JSON provided examples to better understand the specific access rights you could begin to assign to lock down the user to only specific functions.

Edit 2018-01-26 From the AWS Console Page: \- Click Services, type in IAM \- From the IAM Console Click Users \- Select the user then via the Permissions Tab, Click Add Permissions \- Locate "AdministratorAccess"

# Answer
> 0 votes
I hope you've figured it out yourself. In case you (or some other people) have that issue:

The user that needs to attach / detach the instance profile to an instance needs:

* EC2 actions to perform that. That includes:

```
{
    "Sid": "InstanceProfileAccess",
    "Effect": "Allow",
    "Action": [
        "iam:GetInstanceProfile",
        "iam:ListInstanceProfiles",
        "iam:ListInstanceProfilesForRole",
        "iam:AddRoleToInstanceProfile",
        "iam:RemoveRoleFromInstanceProfile"
    ],
    "Resource": "*"
}

```
* The `passRole` action to pass the role to the specific resource.

```
{
    "Sid": "PassDefaultInstanceProfileToEC2",
    "Effect":"Allow",
    "Action":"iam:PassRole",
    "Resource":"arn:aws:iam::<ID>:role/<ROLE_NAME>"
}

```

That user can gain access via policies or roles or group. I hope this helps.

You can read more here:

---
Tags: amazon-ec2, aws-iam
---


# Question
Title: Application version and Database Schema version mapping
I am trying to integrate database schema versioning in an existing project where schema is not versioned at all.

Is it a good practice to create a tight mapping between application version and database schema version so much so that the application deployment will fail if the schema version is not the one expected by the app version? This is because the application has frequent schema changes.

What are the pitfall of using such a logic in application init code?

# Answer
Honestly, you have several questions to unwrap and there is no perfect guidance. Database versioning is difficult at best. However, there is guidance that I have been successful with. In all cases, starting where you are as "version 0" is your best bet.

For your first statement, use one of the following methodologies (or a combination that works best):

* EntityFramework Code First Migrations \- Treat the current state of your schema as well as any reference tables as version 0.
* DbUp and the DbUp Azure Pipelines tasks \- This is a very simple, but effective way to get a "poor-man's" database migration methodology, if you follow some simple rules/conventions. There are also ways to integrate "running" DbUp migrations as part of the start-up of your application, too, but I've had success executing DbUp migrations via Azure Pipelines releases.
* Red Gate Database Lifecycle Management (DLM) Automation Suite \- This requires a bit more planning and comes with a cost, but Red Gate's tools integrate well and give you more of a complete DLM picture from integration of SSMS with Git version control, drift detection, Azure Pipelines tasks, and more.

As far as the question of *should you* couple versioning of the database with the application, you'll need to answer this question. Is the database shared between more than one application or is it dedicated to that application. If "dedicated" is your answer, then versioning of the database with the application is the right thing to do and should be done so within the same source code repository. If not, it will be difficult to decide which application "owns" the versioning of the database or if it needs to be versioned entirely independent of your apps. If it is versioned independent of your apps, then you better have strong unit testing in all of the apps.

Lastly, I will assume your "pitfalls of using such logic" refers to using your applications initialization/startup code to run an upgrade of the database to the intended version. If I've assumed correctly, there is no right/wrong way. What matters is that you understand where you need the automated process to fail to ensure your application does not incur a downtime because of a failed database migration. If you upgrade a DB during initialization/startup of an application, then your app better have a fallback mechanism where it can function with the previous version of the DB until which time it's fixed. This is where I'd recommend making failure of a DB upgrade part of the deployment process itself and not part of the application. That way, if the DB upgrade fails (and you're doing it correctly), reverting to the previous version of the app upon failure ensures that downtime is minimized.

And, when I refer to doing DB upgrades "the right way," I refer to using methods that ensure the state of the DB schema is only changed atomically and transactionally such that failures in the middle of a deployment can be completely rolled back.
> 1 votes

# Answer
I think having such check in place is a good thing, especially if running the app with the wrong schema would cause app failures or, even worse, leave room for potentially costly corruptions in the database.

The only pitfall I can think of would be dependent on the actual deployment process - if it stops the already deployed app version before deploying the new app version the additional check will increase the number and duration of app outages. Not necessarily a bad thing when taking into account the cost of running the app with the wrong schema version. But IMHO it *should* be possible to adjust the deployment process to avoid such outages by only stopping the existing app version after successful deployment of the new version, typically by switching traffic from one version to the other.

Another approach to this problem would be to always have the app capable of running in backwards-compatible manner, automatically handling the schema changes. If such capability exists then you shouldn't have such check in place.
> 0 votes

---
Tags: continuous-integration, continuous-deployment, databases
---


# Question
Title: Can I run a Ubuntu 16 Docker image on a Ubuntu 18 host? Will I need to install hardware drivers already available to the host?
We have some Docker images built from Ubuntu 16.04 base image, and we've been running them on hosts running Ubuntu 16.04; these hosts are laptop computers from different makes. However, we have had some problems finding drivers and we wind up installing them manually instead of having a smooth and seamless Ubuntu installation.

We think that it could be an option to use Ubuntu 18.04 as the OS in the hosts, as this newer versions has better support for newer PCs.

The question is: suppose that the host running Ubuntu 18.04 has "built-in" drivers for, say, the WiFi card, but those are not automatically available for Ubuntu 16.04. Will we need to anyways install those drivers manually in the Ubuntu 16.04 Docker image or will it just run OK since they are correctly installed in the host?

I cannot get it from Docker's own system overview.

# Answer
The docker container will be abstracted from the way that your host accesses the internet/network, and it should work fine / not care about if it's wifi or not/etc.

You can happily run ubuntu docker containers on docker on top of windows, other linux variants (e.g. centos), mac, or anything else without issues. So, this should pose no difficulty in practice.
> 1 votes

# Answer
Docker containers do not use hardware drivers. A containers network connection is abstracted by Docker bridge.

I highly recommend you seek some free tutorial videos or check out Linux Academy to better understand Docker. I promise it will give you great insights.
> 3 votes

# Answer
docker is a framework to run containers isolated independent of operational system under of you image in use. In theory you can run normally without problems. will depend on what you intend to run in the container.
> 0 votes

---
Tags: docker, containers, ubuntu
---


# Question
Title: How to deploy Dev Azure SQL code to Prod Azure SQL using dacpac without knowing the target server credentials
Here is my scenario -

I have access to source Azure SQL Database where i did few changes to schema (eg adding a table). Now I want to deploy this SQL code to prod Azure SQL Database using Azure DevOps dacpac task.

Build pipeline - I have used MS Build task to generate .dacpac file

Release pipeline - I know Azure SQL server name and Database name but do not know Server admin credentials. How to deploy dacpac to the Prod target server? What are all the tasks required in Release pipeline to achieve this and it should be repeatable process.

Any help would be really appreciated

# Answer
> 1 votes
You either have to have the login information in order to connect to the database, or, you have to configure your database server to allow access to Azure Services. This means that the SQL Database and server and the Azure DevOps pipelines have to be part of the same account (not necessarily the same login, but Azure AD). I'd lean towards requiring the connection information rather than open up everything broadly between all your Azure services.

---
Tags: azure-devops, azure-pipelines, databases
---


# Question
Title: Correct way to handle 'optional' array of resources in Terraform
In one of our our Terraform modules we have a variable to optionally create a load balancer. As part of this we create some subnets thusly:

```
resource "aws_subnet" "subnet_loadbalancer" {
  count = var.create_loadbalancer ? length(var.az_names) : 0
  vpc_id            = aws_vpc.vpc_main.id
  cidr_block        = var.lb_subnet_cidrs[count.index]
  availability_zone = var.az_names[count.index]
}

```

Now, if we are creating a loadbalancer I want to output the first subnet for later use in other modules:

```
output primary_LB_Subnet {
  value = aws_subnet.subnet_loadbalancer[0]
}

```

This is fine if we create the loadbalancer, but not if `var.create_loadbalancer` is false. In that case I get the following error:

```
Error: Invalid index

  on ..\..\modules\vpc\loadbalancer.tf line 19, in output "primary_LB_Subnet":
  19:   value = aws_subnet.subnet_loadbalancer[0]
    |----------------
    | aws_subnet.subnet_loadbalancer is empty tuple

The given key does not identify an element in this collection value.

```

I can see why this would be a problem, as we haven't create the subnets, so my immediate thought was to do something like:

```
output loadbalancer_sg_id {
  value = var.create_loadbalancer ? aws_subnet.subnet_loadbalancer[0] : null
}

```

But this doesn't help the issue and I still get the same issue. What is the correct way to address this within Terraform?

# Answer
Could you wrap the assignment to value using the try type conversion function?

```
output primary_LB_Subnet {
  value = try(aws_subnet.subnet_loadbalancer[0], null)
}

```
> 2 votes

---
Tags: terraform
---


# Question
Title: Running Two infinite commands on single pod/image
I have a Dockerfile with a CMD, it calls a .sh file that runs a Django app ( makemigrations, migrate and uwsgi ) And that runs fine in a pod.

There's a django admin script that runs infinitely that I'd like to run in the same pod.

* Per the docker docs I can't have a second command in the Dockerfile
* Adding the script on the sh file wouldn't work because the both processes ( the app or command script ) are running and don't end so the second one never runs.
    + If I go in the pod and run exec -it bash to manually run the script i can't leave the pod without killing the process (Ctrl-C) or can I?

Any suggestions?

# Answer
> 1 votes
Though it's not advised to have multiple processes inside a container, but you have 2 options:

Use a process manager like supervisord or wrapper script to use as CMD (meta code)

```
start_first_process_in_bg
if START_FAILED
  exit
start_second_process_in_bg
if START_FAILED
  exit

while sleep 5; do
  check_first_process
  check_second_process

  if ANY_PROCESS_FAILED; then
    exit

```

---
Tags: dockerfile, kubectl
---


# Question
Title: Why does running echo with exec form in a Dockerfile create /etc/resolv.conf?
I was trying to build a Docker image from a `Dockerfile` and I had a line in there which reads:

```
RUN ["echo", "'anything'", ">", "./a_file"]

```

This didn't work, and I eventually had to use:

```
RUN ["sh", "-c", "echo 'anything' > ./a_file"]

```

But before I fixed the instruction, I used `dive` to see what that line changed inside the image. To my surprise, the `a_file` was not created; instead an empty `/etc/resolv.conf` file was created instead.

The `dive` output below shows the modified files for that layer.

```
│ Layers ├─────────────────────────────────────────────────────────────────────────── ┃ ● Current Layer Contents ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Cmp   Size  Command                                                                   Permission     UID:GID       Size  Filetree
    4.4 MB  FROM 45c1c01c359b59b                                                      drwxr-xr-x         0:0        0 B  └── etc
     57 MB  addgroup -g 1000 node     && adduser -u 1000 -G node -s /bin/sh -D node   -rwxr-xr-x         0:0        0 B      └── resolv.conf
    5.1 MB  apk add --no-cache --virtual .build-deps-yarn curl gnupg tar   && for key
       0 B  WORKDIR /home/node/
    716 MB  COPY /home/node/ ./ # buildkit
       0 B  RUN echo 'anything' > ./a_file # buildkit

│ Layer Details ├────────────────────────────────────────────────────────────────────

Tags:   (unavailable)
Id:     6af1a38f29e248b7fb1787b36f24edf492cd3b71ebbd0045233ae402c9c0db8c
Digest: sha256:9743570920cc3cf0c2009ec80a2a24828b869fe53f245f30ccb3d8020d6f0195
Command:
RUN echo 'anything' > ./a_file # buildkit

│ Image Details ├────────────────────────────────────────────────────────────────────

Total Image size: 782 MB
Potential wasted space: 391 kB
Image efficiency score: 99 %

Count   Total Space  Path
    2        167 kB  /root/.gnupg/pubring.kbx
    2        153 kB  /root/.gnupg/pubring.kbx~
    3         34 kB  /lib/apk/db/installed
    3         29 kB  /lib/apk/db/scripts.tar
▏^C Quit ▏Tab Switch view ▏^F Filter ▏Space Collapse dir ▏^Space Collapse all dir ▏^A Added ▏^R Removed ▏^M Modified ▏^U Unmodified ▏^B Attributes ▏

```

I'd expect no files would be created, but why, specifically, was the `/etc/resolv.conf` created?

Note that the base image is `node:8.15.0-alpine`.

# Answer
> 1 votes
The `/etc/resolv.conf` is injected as a bind mount in containers by the docker engine with settings local to that docker engine. An empty file in the parent filesystem is an artifact of that bind mount. You can safely ignore that.

If `a_file` is not being created, check that the file is not in a volume. Temporary (anonymous) volumes are created with the temporary containers in a RUN command, and changes to a volume are not included in changes to the container filesystem.

---
Tags: docker, dockerfile, dns, alpine
---


# Question
Title: Is there a way through which I can control type of load balancer like round robin , least-connections etc using helm charts?
I am researching different types of `loadbalancing` methods available through kubernetes .I want to implement & see before deciding on one .I'm stuck at how to actually define `typeof-loadbalancer`.If it isn't possible through `helm` can anyone suggest other options please .

# Answer
Well there are 2 possibble options.

One is to create a **LoadBalancer** type for each service you want to be exposed. In case you have just one service that's great. That's rarely the case of course :)

If you build your k8s cluster properly ( having Prometheus-Operator doing the monitoring for example ) most likely besides the business logic service/s, you would want to expose internal cluster valuable services like Grafana.

That's why even if you have just 1 service, for future extension of such type of need the best would be to use an **Ingress** resource. It providees a way to expose multiple services through a single IP address — HTTP level (network layer 7).

You could play with HTTP headers essentially and do some path/host routing.

You express the definition for your object and helm would do the installation. Another option is to use a tool like ArgoCD, basically sticking to GitOps policy.

A fundamnetal knowledge is to be aware that the Load Balancer/Ingress would forward the traffic to the service, not the pods. Services are abstracted entities by kube-proxy, by default kube-proxy uses a round-robin algorithm.

https://kubernetes.io/docs/concepts/services-networking/service/

Regarding the definition, it follows the standart K8s scheme \[ **apiVersion**, **kind**, **metadata**, **spec** fields\]. Example would be an Ingress created from AWS-Ingress-Controller.

```
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress
  namespace: management
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
spec:
  rules:
  - host: grafana.com
    http:
      paths:
      - backend:
          serviceName: grafana
          servicePort: 80
        path: /*

```
> 1 votes

---
Tags: amazon-web-services, kubernetes, helm, load-balance
---


# Question
Title: Looking for a good book explaining how things work in the industrial system test environment and production environment
Any advice on a good book explaining how things work in the industrial system test environment and production environment? It's my first time in this topic, so I would need a general overview.

Key words: builds, web logic levels, machines, clusters, logs, cache, balancers, UDC

(If you thing I posted this question in the wrong place, please, could you point me the right one? Thanks)

# Answer
> 1 votes
Dude, will depends of what you want to use or follow. Actually cloud + DevOps it's a buzzword in use. So you can start reading about Cloud Architecture, Modern Infrastructure. Refer about one book, maybe these: https://www.springer.com/gp/book/9783030037116

---
Tags: load-balance, builds, logging, cache
---


# Question
Title: I don't fully understand how to use variables in Ansible roles
**Original question:**

I want to roll out a partitioning layout for a home lab setup of GlusterFS and also want to do it using Ansible so I can re-use the configuration for a more stable setup later on. This is also the first time I work with Ansible so please bear with me ;)

How do I use variables across roles correctly?

I created a role for the basic partitioning of a single data disk and I want to have 3 disks on each node.

My nodes are as follows (path = `inventories/staging/hosts.yaml`)

```
all:
  hosts:
    node1:
      ansible_host: node1
    node2:
      ansible_host: node2

storageservers:
  hosts:
    node1:
      ansible_host: node1
    node2:
      ansible_host: node2

```

The base role used by the aggregating role is as follows (path = `roles/gluster-node-partition/tasks/main.yml`):

```
---
# tasks file for gluster-node-partition
- name: 'Create {{ partition_name }} partition'
  parted:
    device: '{{ device_name }}'
    number: 1
    label: gpt
    name: '{{ partition_name }}'
    state: present
  become: true

- name: 'Create filesystem on {{ partition_name }} partition'
  filesystem:
    fstype: btrfs
    dev: '{{ device_name }}1' #should expand to something like '/dev/sdb1'
  become: true

- name: 'Create mount point for {{ partition_name }} partition'
  file:
    path: '/{{ partition_name }}'
    state: directory
  become: true

- name: 'Mount {{ partition_name }} partition'
  mount:
    path: '/{{ partition_name }}'
    src: 'LABEL={{ partition_name }}'
    fstype: btrfs
    state: present
  become: true

```

The aggregating role is as follows (path = `roles/gluster-node/tasks/main.yml`):

```
---
# tasks file for gluster-node
- include_role:
    name: gluster-node-partition

- name: Create data1 partition
  include_role:
    name: gluster-node-partition
  vars:
    device_name: '/dev/sdb'
    partition_name: 'data1'

- name: Create data2 partition
  include_role:
    name: gluster-node-partition
  vars:
    device_name: '/dev/sdc'
    partition_name: 'data2'

- name: Create data3 partition
  include_role:
    name: cluster-node-partition
  vars:
    device_name: '/dev/sdd'
    partition_name: 'data3'

```

I invoke the following command `ansible-playbook -i inventories/playground/hosts.yaml site.yml -K`

The output is as follows

```
BECOME password:

PLAY [storageservers] *****************************************************************************************************************************************

TASK [Gathering Facts] ****************************************************************************************************************************************
ok: [node2]
ok: [node1]

TASK [include_role : gluster-node-partition] ******************************************************************************************************************

TASK [gluster-node-partition : Create {{ partition_name }} partition] *****************************************************************************************
fatal: [node1]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'device_name' is undefined\n\nThe error appears to be in '/home/user/ansible/roles/gluster-node-partition/tasks/main.yml': line 3, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n# tasks file for gluster-node-partition\n- name: 'Create {{ partition_name }} partition'\n  ^ here\nWe could be wrong, but this one looks like it might be an issue with\nmissing quotes. Always quote template expression brackets when they\nstart a value. For instance:\n\n    with_items:\n      - {{ foo }}\n\nShould be written as:\n\n    with_items:\n      - \"{{ foo }}\"\n"}
fatal: [node2]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'device_name' is undefined\n\nThe error appears to be in '/home/user/ansible/roles/gluster-node-partition/tasks/main.yml': line 3, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n# tasks file for gluster-node-partition\n- name: 'Create {{ partition_name }} partition'\n  ^ here\nWe could be wrong, but this one looks like it might be an issue with\nmissing quotes. Always quote template expression brackets when they\nstart a value. For instance:\n\n    with_items:\n      - {{ foo }}\n\nShould be written as:\n\n    with_items:\n      - \"{{ foo }}\"\n"}

PLAY RECAP ****************************************************************************************************************************************************
node1                      : ok=1    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0
node2                      : ok=1    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

```

**Solution by w00te:** Thanks to w00te, it works now :) added the vars like this (`roles/gluster-node/tasks/main.yml`)

```
# tasks file for gluster-node
- include_role:
    name: gluster-node-partition
  vars:
    device_name: '/dev/sdb'
    partition_name: 'data1'

```

# Answer
> 1 votes
Your very first role include does not have variables provided with it, so it will fail.

Here's an excerpt from above showing this. Notice lines 1-2 have no variables after them (unlike the include following that which does correctly provide the variables).

```
- include_role:
    name: gluster-node-partition

- name: Create data1 partition
  include_role:
    name: gluster-node-partition
  vars:
    device_name: '/dev/sdb'
    partition_name: 'data1'

```

As `roles/gluster-node-partition/tasks/main.yml` uses device\_name twice, and it is not defined for that first call, it will fail. The two usages are noted on line 2 and on the last line below.

```
  parted:
    device: '{{ device_name }}'
    number: 1
    label: gpt
    name: '{{ partition_name }}'
    state: present
  become: true

- name: 'Create filesystem on {{ partition_name }} partition'
  filesystem:
    fstype: btrfs
    dev: '{{ device_name }}1' #should expand to something like '/dev/sdb1'

```

# Answer
> 1 votes
You might want to use the group vars feature, so that your playbooks stay clean and re-usable. You define a `storageservers` group in your inventory; associated with this group, Ansible will look for `group_vars/storageservers.yml` for the vars to associate with this group. Here, you can set the variables:

```
# group_vars/storageservers.yml
---
partitions:
 - device_name: /dev/sdb
   partition_name: /data1
 - device_name: /dev/sdc
   partition_name: /data2
 - device_name: /dev/sdd
   partition_name: /data3

```

Now, you have a list of partitions which will be picked up when you target `hosts: storageservers`, which you can loop over with one statement instead of 3:

```
# playbook.yml
---
hosts: storage
become: true
roles:
  - gluster-node-partition

```

Updated tasks:

```
# roles/gluster-node-partition/tasks.yml
- name: "Create {{ item['partition_name'] }} partition"
  parted:
    device: "{{ item['device_name'] }}"
    number: 1
    label: gpt
    name: "{{ item['partition_name' }}"
    state: present
  loop: "{{ partitions }}"

- name: "Create filesystem on {{ item['partition_name'] }} partition"
  filesystem:
    fstype: btrfs
    dev: "{{ item['device_name'] }}1" #should expand to something like '/dev/sdb1'
  item: "{{ partitions }}"

- name: "Create mount point for {{ item['partition_name'] }} partition"
  file:
    path: '/{{ item['partition_name'] }}'
    state: directory
  loop: "{{ partitions }}"

- name: "Mount {{ item['partition_name'] }} partition"
  mount:
    path: "/{{ item['partition_name'] }}"
    src: "LABEL={{ item['partition_name'] }}"
    fstype: btrfs
    state: present
  loop: "{{ partitions }}"

```

Doing it this way separates variables from tasks. If you want to add or change a filesystem, you only need to change the *variable* and not the *playbook*.

You also have a much cleaner playbook -- just a few lines. It's clear that this *all* of the filesystem stuff is contained in the role.

If you have a different set of machines where you want different mounts, you only have to add a new group, give that group a new `partitions` variable, and target it in the same way by setting `hosts: other_storage_servers`.

For reference, see the Ansible docs on :

---
Tags: ansible, glusterfs
---


# Question
Title: Google Cloud SQL: Can I change machine type with zero downtime on my database clients?
I want to update the machine type of my Google Cloud instance, but this takes several minutes to update (second generation instance). The instance will be unavailable until the instance has restarted. Because of this downtime, we have to update the machine type at night time, so our visitors are the least troubled by this update.

Is there a workflow how we can minimise this downtime to zero or maybe a few seconds? I already thought about possible solutions like adding a temporary failover or maybe make use of read replica.

# Answer
> 3 votes
I contacted the support of Google Cloud about this question and they told me that Cloud SQL isn't build to perform this change without downtime. If I want to be able to make these changes, I should look at Cloud Spanner which is a horizontal scalable SQL solution provided by Google.

---
Tags: google-cloud-platform
---