# Question
Title: Terraform foreach multiple map variables
I need to do the following in Terraform but can't work out how

Here is what I have

```
local {
app_config = {
    test-web = {
      name            = "test-web-${local.environment}"
      websockets      = false
      subnet          = "backend"
      audience_subnet = "frontend"
      url             = "https://test-web-${local.environment}.mydomain.com"
    }
    test-api  = {
      name            = "test-api-${local.environment}"
      websockets      = false
      subnet          = "backend"
      audience_subnet = "frontend"
      url             = "https://test-api-${local.environment}.mydomain.com"
    }
  }

keyvault_secrets = {
    aat = {
        test-api = {
            application_id = "1111"
            client_id = "2222"
            administrator_pass = "3333"
        }
        test-web = {
            application_id = "4444"
            client_id = "5555"
            administrator_pass = "6666"
        }
    }
    Demo = {
        test-api = {
            application_id = "1212"
            client_id = "2323"
            administrator_pass = "3434"
        }
        test-web = {
            application_id = "4545"
            client_id = "5656"
            administrator_pass = "6767"
        }
    }
    Dev = {
        test-api = {
            application_id = "9999"
            client_id = "8888"
            administrator_pass = "7777"
        }
        test-web = {
            application_id = "9898"
            client_id = "8787"
            administrator_pass = "7676"
        }
    }
}

resource "azurerm_key_vault_secret" "app_id" {
  for_each = var.apps_config

  name         = var.apps_config
  value        = each.value.application_id
  key_vault_id = data.azurerm_key_vault.mykv.id
}

resource "azurerm_key_vault_secret" "client_id" {
  for_each = var.apps_config

  name         = var.apps_config
  value        = each.value.client_id
  key_vault_id = data.azurerm_key_vault.mykv.id
}

resource "azurerm_key_vault_secret" "admin_pass" {
  for_each = var.apps_config

  name         = var.apps_config
  value        = each.value.administrator_pass
  key_vault_id = data.azurerm_key_vault.mykv.id
}

```

What I can't work out is how you do the following: For every app\_config defined, for example: test-web test-api

A Key Vault secret are created by the environment defined when running:

Terraform plan or apply -var="environment=dev"

I want it to create the 3 secrets for each app\_config

# Answer
> 2 votes
Is something like the below what you are looking for?

```
locals {
  apps = keys(var.app_config)
}

resource "azurerm_key_vault_secret" "app_id" {
  for_each = toset(local.apps)

  name         = each.key
  value        = var.keyvault_secrets[var.environment][each.key]["application_id"]
  key_vault_id = data.azurerm_key_vault.mykv.id
}

```

---
Tags: terraform
---


# Question
Title: jenkins build overview page not showing stages in webui
I'm using the image: jenkinsci/blueocean from Dockerhub

The build overview page isn't showing the stages of a build. i.e. the square boxes that represent the stages are not appearing.

Am I missing a plugin?

# Answer
> 1 votes
Added a pipeline-stage-view plugin

---
Tags: jenkins, jenkins-pipeline, jenkins-plugins, jenkins-blueocean
---


# Question
Title: Retrieve files deleted after Docker service rm
I had to, so I removed a service with docker service rm but I forgot to get the log files of the service. Now I am not able to find those. Can anyone help me or suggest to me if and how can I get those log files. I was using json-files driver for logging via yml.

# Answer
> 1 votes
There's no special feature to undelete files on the host provided by docker. So if you delete your files using docker, you are left with the same option as when you delete on Linux without using docker:

1. Learn from your mistake and don't do it again (I'd recommend forwarding logs to an external log aggregator).
2. Restore from backup.
3. Do a forensic recovery of the hard drive contents.

---
Tags: docker, docker-swarm
---


title: Separation of CI and CD processes

text:
# Question
Title: Separation of CI and CD processes
Is it appropriate to keep the DevOps resources in a different repo from the git repos where the project source codes are placed for the execution of DevOps processes?

some benefits i expected

* Versioning files to a different repo such as Dockerfile, Jenkinsfile or Kubernetes deployment
* Ensuring the security of deployment processes
* Being able to manage the deployment processes of these projects since some projects consist of more than one repo

I know the concepts of monorepo and multirepo, but what I say is a little different. What I mean is to abstract the repos of deployment files from the source code repos.

Could this be a method of use? have you done something like this before? is this method useful? Does this method have a name or does it mean multirepo?

I am waiting for your advice

thanks everyone

# Answer
You're basically describing GitOps. Here's how I've followed that for a containerized microservices build and deploy pipeline:

1. Each service repo includes it's own Dockerfile, Kubernetes manifests, and anything else needed to locally build and test the project. The base image for the Dockerfile will likely be managed from another repo but built and hosted by the organization.
2. CI steps are mostly externalized to a set of common libraries or templates. So if there's a Jenkinsfile or equivalent, it sets a few variables and calls the common template to build and generate the artifacts.
3. Artifacts include a docker image, pushed to a registry, and also the deployment manifest (compose file, Kubernetes manifest, Helm templates), along with any other variables for those manifests. These deployment manifests/variables are pushed to a separate deployment repository that includes all of the microservices' manifest. There is only one deployment repo for all of the microservices in a single application stack, so this commit becomes a fan-in step. The push to this repository is scripted/automated during the CI of the microservice repo when a pull request is approved by enough trusted people to a protected branch. This provides the needed security checks.
4. The deployment repo commits triggers it's own pipelines that are effectively CD and following something close to a GitOps design. This could also be a GitOps tool monitoring the repo and applying the changes, but presently I've been sticking with a single CI tool that does both, simplifying the workflow and allowing some scripting to run on the deploy. That scripting for me loads the variables that were committed, applies additional variables (e.g. which environment is this being deployed to), renders the templates (currently a Helm umbrella chart), and deploys.
5. Deploys to other environments becomes a pull request between branches of the deploy repository. It could also be separate repositories to enforce security controls. Importantly, rendering the manifest should point to an immutable image tag or sha hash to ensure what was pushed to dev and tested there is the same as what's pushed to production. I include these image tags in a values file that is committed to the deploy repo.

One big advantage of this, multiple microservices that interact with each other are tested together and get deployed together as a unit, all services are fanning into a single deploy repo that contains their deployment manifests and what image tag to use. There's never the situation where service A gets a new feature, but it's buggy, service B gets built depending on that new feature, and service B gets released to production before service A (a common issue when each service is built and deployed in isolated pipelines). If A is blocking the deployment of the stack, then a pull request can be submitted to the deploy repo reverting the service A commit. Then if service B passes tests against an older service A, service B gets deployed to production.
> 3 votes

---
Tags: deployment, continuous-delivery, methodology, process, monorepo
---


# Question
Title: Can I run an API gateway on the same instance as my API?
Is there any reason why an API gateway should be on a separate server to the API? Other than ensuring enough resources for both. I am using krackend API gateway and hosting a python API.

# Answer
It should be fine to run the API Gateway on the same server as API-apps - for smaller setups. For larger setups when you need more servers may make sense to use dedicated nodes for API Gateways. API Gateways are increasingly playing a dual role as an application load balancer, or Level 7 Proxy for your APIs - those have a **specific role when it comes to network addresses and routing**, this is the main reason to why it makes sense to use dedicated servers for this part of your stack.

> why an API gateway should be on a separate server to the API?

what is more important for this part of the stack is that you would like to avoid single-point-of-failure, so you should use **more than one instance** for API Gateways, and also for the API apps. When using multiple instances, it is often easier to separate the load balancer/Gateway from your app workload due to the specific needs for network addresses for the gateway, but in the end this depends on how your whole architecture looks.
> 1 votes

---
Tags: api, api-gateway
---


# Question
Title: is a secrets-provider-container a good idea?
I am looking for a way to provide secrets to a container running on Google Cloud Run responding to the following criteria:

* secure
* centralized & pulled based
* free
* simple
* working with a team, with various trust scopes
* host agnostic: changing 1 config will be the same on aws, gcp, azure

When starting a new project with containers, I always come to the point where I need to manage secrets, and I often find it difficult. **My current solution** is to store my secrets encrypted on a public git repository, and decrypt them inside the container, in the entry point script.

But I don't like so much installing gpg along with Django and it's dependencies. I am now thinking of running another container on the same network, which sole responsibility will be serving the config to the first container.

**Is it secure ? Is there a better way ?**

# Answer
> 1 votes
There is probably a better way if you are working with Google Cloud Run. I have not tried this but I see that they have "Secret Manager" that you can use. They also mention a project Berglas that can be used to store secrets in GCS cloud storage.

In my personal experience, it find it best to leverage widely adopted solutions and avoid creating a unique solution for security. Getting it right is non trivial and it is less code you have to support.

This FAQ has doc links to the solutions I mentioned. https://github.com/ahmetb/cloud-run-faq#how-to-configure-secrets-for-cloud-run-applications

---
Tags: docker, docker-compose, dockerfile, configuration
---


# Question
Title: Is there an organization that offers a Hashicorp Vault SaaS solution?
There is a number of recommendations to harden a Hashicorp Vault deployment, but is there a SaaS solution, like CloudBees offers Jenkins to prevent that the hardening should be done manually?

# Answer
It seems that Hashicorp itself offers an enterprise solution including 24x7 gold support called Hashicorp Vault premium enterprise.
> 3 votes

# Answer
To my knowledge there are no services today that offer Vault SaaS. However, if you're looking for a better UI/UX, there are great options out there for a Vault UI (A full Hashicorp supported list can be found here). I've personally tried out both Cryptr and Goldfish and have found both to be robust and complete.
> 1 votes

# Answer
I know this is a bit late (like by 2 years lol), but akeyless.io is a SaaS secrets management tool. You can see more info compared to other tools here.
> 0 votes

---
Tags: hashicorp-vault, software-as-a-service
---


# Question
Title: Command "git [command] --help" opens as HTML in my code editor - no output to terminal. How do I revert to default behavior?
When I type `git reset --help` I don't get output in the bash terminal, but a new tab opens in my code editor to reveal some dense HTML. How can I change Git's behavior to display the help message in the bash terminal? I'm not sure how to fix this.

**Update:**

.gitconfig file:

```
[core]
    editor = 'C:\\Program Files\\Sublime Text 3\\sublime_text.exe' -w
[user]
    name = name
    email = 'email
    name = 'name

```

```
$ type -a git
git is /mingw64/bin/git
git is /mingw64/bin/git
git is /cmd/git

```

**Update 2:**

It looks like `help.format` is set to html, but after my user.name and other info, `help.format` is set to web (I did this). I tried setting the browser using multiple browser names, but git fails to launch the help command in the browser and still opens in my code editor.

```
$ git config -l
core.symlinks=false
core.autocrlf=true
core.fscache=true
color.interactive=true
help.format=html
user.name=name
help.format=web

$ git reset --help
git config option web.browser set to unknown browser: edge
Resetting to default...

```

# Answer
> 1 votes
This is not the default behavior of `git` so you have made some local config change to cause this to happen. Look in your `~/.gitconfig` to see if there's anything that doesn't make sense to you. Try commenting it out. The `core.pager` setting is one of the first things I would check. This stackoverflow question has a variety of options for dealing with this sort of thing.

## update 1

I haven't tried to get this sort of thing to work, but according to the docs you can also specify a `man.viewer` to define the program that is going to run. Do you get any errors if you try `git help -w reset`?

One work around would be to get the help to show up in the terminal by using `git help -m reset`. By using the `help` subcommand instead of `subcommand --help` you can specify whether you want it to try man, info, or web.

# Answer
> 0 votes
Please check this out to reset your git config setting

https://stackoverflow.com/questions/35853986/easiest-way-to-reset-git-config-file

second thing, if you want output of help then you can use the "git reset -h" command which shows you minimal option of usage in your bash terminal whereas "git reset --help" command will be direct you to html format which is more descriptive explanation of command.

I hope this will help you.

---
Tags: git, bash, versioning
---


title: How to set up WordPress CI/CD pipeline

text:
# Question
Title: How to set up WordPress CI/CD pipeline
I am planning to create 3 environments for my website i.e. dev, stage, and production. I am using GitHub to manage the code. So for URLs **dev.example.com**, **stage.example.com**, and **example.com**, the branches are **dev**, **stage**, and **main**. My knowledge says **dev** is for the developer to play around with new codes, tests, and do whatever he wants, the **stage** is like production-ready where QA personal can test codes, functions, etc.

I have modified wp-config.php to take environmental variables. For example, I have changed

```
define( 'DB_PASSWORD', 'password' );

```

to

```
define( 'DB_PASSWORD', getenv('DB_PASS') );

```

# Questions

## Database

* How should I move a database from one environment to another? Should I let the developer do anything from the dashboard in production and later move the database from production to dev/stage if require?

## Plugins

* I know I can avoid including plugins inside the `wp-content/plugins` folder and use wpcli to install the plugins which are free but what about premium plugins? How can I integrate them?

## Update

* What should be the update strategies? Should I let the developer update from the dashboard? I think it will make changes in the codes and later will conflict with `git merge`. what is your thought on this? Also, do you take a backup of all environments or just production?

## Tags

* How should we configure the pipeline for new releases in GitHub? Should I manually configure the pipeline to deploy on each release?

For now, I only have these challenges. If I will have more questions then I will add.

* I am planning to run WordPress in the docker container, alpine as a base image. I am thinking of separating Nginx and php-fpm container. Doing so, we need to copy WordPress files in both containers, how can I overcome this? or should I use php-fpm and Nginx in the same container? is it a good idea?
* what about the cronjobs?

# Answer
It has been a while since I have run a few Wordpress based sites. The database stores some of the content of the site. The file system stores media that is uploaded into the site. More significantly Wordpress is a Content Management System. The intention is that your site users can add content and media to the live site. When someone uses the WYSIWYG editor to add or edit the content you will find that the sites domain name leaks into the database. So if you move the Staging database into Live you would get links appearing on the Live that that point back to the Staging site.

I quick bit of research to see if things have improved since I last used it leads to this article on putting Wordpress into git. Please read the first article in that series that explains the challenges. You will need to understand a little of how Wordpress works to understand the recommendations I make below.

That article talks about turning off automatic updates so that you have to manually security path. I am pretty sure that I remember that updates can write things into the database. I would be very worried about turning off updates as there are a huge amount of attacks against Wordpress as it is so widely deployed. You really should avoid deploying many third party plugins as they might not get security patches in time. This isn't a theoretical problem I had quite a time dealing with Wordpress being used as a spam relay. Yet even if you follow that article you won't have solved the database migrations issue.

My recommendation is that you take a step back and try to figure out what problems you are trying to solve and how to adapt the tools and process:

1. What is the developer developing? A custom Theme and/or Plugin? If so then this article covers that. Put the Theme/Plugin into its own repo.
2. Who actually writes the site content or uploads media? The general public or members of staff? If it is staff then maybe they need minimal training in a Test environment and can happily edit Live after that.

Given that Wordpress does not let you aim for perfection due to the way that it is built, yet it is hugely successful, carefully pick which of the following you are aiming to optimise:

* A Specific Service Level Agreement
* Meantime to recovery
* Mean time between failures
* Reducing the commercial impact of errors
* Something else?

There are quite a few other specifics that may affect what set of compromises might work best for your situation.

Based on my experiences without actual knowledge of your specifics:

* Give up the idea of promoting changes through Dev, Staging and Live. I would simply have a Test site and a Live site. I would periodically refresh the Test site from the Live site. To do this I would dump out the entire Live database a run a `sed -i` over it to replace every instance of "www.mysite.com" with "test.mysite.com".
* Give up on using Docker as the software distribution mechanism for the actual site. I would use Docker to supply the PHP+Apache runtime and mount a volume that contained the actual site php+content. This because the disk is a combination of the code and the content of the site so you need to be able to backup and restore it independently of changing the Docker image.
* Try "git checkout vX.Y" to update the live site in a persistent volume mounted into your generic PHP+Apache docker image. I would investigate the ideas in this series of articles.
* Automate the backup of the media content volume every night and keep as many backups as you can. This is why it needs to be a volume mounted into Docker.
* Automate the backup of the database every night and keep as many as you can.
* Automate the restore of the site as a script that restores the database to the last midnight backup then restores the last nightly snapshot of the media content folder.
* Avoid any party plugins that are not commercially supported. They likely won't work at the next major upgrade of Wordpress and will pin you onto a version that will become obsolete.

Then they workflow that I would go for would be the to add "experimental features" into the Test site before applying them to the Live site. This would be based on git checkouts in the best case. Yet I would also consider doing Wordpress security updates via the CLI. Those can update the database and the state on disk underneith Docker hence I do not recommend you try to mange the state of your site as a Docker image.

You then have to worry about the Test site drifting away form the Live site. So I would on a periodic base (at least monthly) overwrite the Test site with the Live site. This would involve:

1. Backing up the Lite sites media content on the persistent volume and restoring it over the Test sites media content on its persistent volume.
2. Backing up the Lite sites database, doing a `sed -i` to rewrite any content links pointing at the Live to point at the Test site. Then restoring it over the Test database.

A key part of this approach is that you would be running your Live site like every other Wordpress site in the world. You don't do "brain surgery" on the Live site. You do frequently do "brain surgery" on the Test site. If the brain surgery goes wrong it's annoying and painful to fix but it is only the Test site that needs some new step to bring it into sync with Live. It is likely that this only happens when you move to a new Wordpress version. If you try a custom approach of moving change into a Live site they might break at a future Wordpress release and leave you with a multiple-day outage in Live.

I completely recognise that the approach outlined above isn't at all how "normal" software is developed. Yet it is the approach that I have scripted in the past. Even if you did the standard dev/test/live approach you do not guarantee 100% perfection and 0 bugs but you might have a longer mean time between failures. If you accept that perfection isn't possible then what you want is very fast rollback to reduce the impact of bugs. You do this by focusing on reducing the meantime to recovery to the speed of a restore script and the time taken to get permission/agreement to run it.

Being able to rehearse things in a Test environment should mostly avoid a total outage on the live site. Being able to do a fully automated restore can then bring you back to a good state at the loss of some recent edits to the site. That is normally completely acceptable compared to having an extended outage. Consider this scenario:

* Staff are continually editing content in production including uploading new media into the content folder on disk (e.g., uploading a PDF and linking to it within content where the link is in the database).
* You have run some Wordpress updates in Test a day ago and they seemed good. Today you run them at 9 am in the Live and something odd happens (maybe a timeout, whatever) and the site becomes corrupted.
* You have automated the rollback. You run the roll-back script which computes the last nightly full backup point then restores the database, then restores last night's media content folder backup over the live folder to revert any lost or corrupted files.

To my knowledge that is as good as you can get. Your meantime to recovery should be the speed of your database restore plus the time take to spot the problem and agree to do the restore. This is exactly why you shouldn't have to speak to editing Staff about whether or not they are happy with the roll-back as the site is broken so of course, they would agree to restore the site automatically, rather than have the site down for hours while a possible point fix is investigated and attempted. You can send out a prepared email telling Staff that there was an outage at 9 am such that the site has been restored to the previous nights backup and that they should double-check and repeat any work that they have undertaken in the past twenty-four hours and redo their work as necessary. IMHO that is as close to perfection as an opensource PHP content management system such as Wordpress allows.
> 2 votes

---
Tags: wordpress, pipeline
---


# Question
Title: Is it possible to containerize using docker without a local setup?
I have a Windows machine and I am just starting to learn docker. I have gone through the internet that the best way to start is by installing docker-desktop. Is there a way i can use docker remotely on any cloud providers without having to install it on my machine ? From what I understand I may need to upload my jar files to the remote servers and everything else should be able to be done on the cloud server. Is there any such subscription I could purchase to use docker remotely ? Please let me know all the possibilities.

# Answer
Checkout Play with docker here. If you just started then it is best to install docker on your local machine to get more of a native experience and understanding of docker networks and overlays etc... and after that you can move on to any cloud like aws ECS service, azure etc....
> 1 votes

---
Tags: docker, dockerhub
---


# Question
Title: How to connect to a kubernetes cluster that is inside of a docker container?
For CICD purposes, I need to run a docker container and then create a kubernetes cluster inside of it using kind. Here is my Dockerfile:

```
FROM ubuntu:18.04

RUN apt-get update
RUN apt-get install curl -y

RUN apt-get install -y gnupg2

RUN curl -fsSL https://apt.releases.hashicorp.com/gpg | apt-key add -
RUN apt-get install -y lsb-release
RUN apt-get install -y software-properties-common
RUN apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
RUN apt-get update && apt-get install terraform -y

RUN apt-get install -y wget
RUN wget https://golang.org/dl/go1.15.6.linux-amd64.tar.gz
RUN tar -C /usr/local -xzf go1.15.6.linux-amd64.tar.gz
RUN GO111MODULE="on" /usr/local/go/bin/go get sigs.k8s.io/kind@v0.9.0

RUN curl -fsSL https://get.docker.com -o get-docker.sh
RUN sh get-docker.sh

```

It's basically an ubuntu image with kind installed.

I run this image using docker run commad:

```
docker run -v /var/run/docker.sock:/var/run/docker.sock -it ef33f1a0ac5e bash

```

And then I create a kind cluster inside of it:

```
kind create cluster --name kind

```

Then I install kubectl on it.

When I try to connect to my kind cluster using kubectl (`kubectl get all`, for example), I get the following error:

```
The connection to the server 127.0.0.1:37583 was refused - did you specify the right host or port?

```

Here is my kubeconfig (`kind get kubeconfig`):

```
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: CERTIFICATE-AUTHORITY-DATA
    server: https://127.0.0.1:37583
  name: kind-kind
contexts:
- context:
    cluster: kind-kind
    user: kind-kind
  name: kind-kind
current-context: kind-kind
kind: Config
preferences: {}
users:
- name: kind-kind
  user:
    client-certificate-data: CLIENT-CERTIFICATE-DATA
    client-key-data: CLIENT-KEY-DATA

```

# Answer
> 2 votes
While you are running a container linked to your PC/Server Docker, the Kind Kubernetes port is available on your PC/Server not on the container.

So if you run `kubectl` commands from your machine it will work (including the kubeconfig).

Solution: Using `--network="host"` in your docker command, the container will act as your localhost (eg. `docker run --network="host" -v /var/run/docker.sock:/var/run/docker.sock -it ef33f1a0ac5e bash`

---
Tags: docker, kubernetes
---


# Question
Title: Why is my website costing approximately £20 per day?
I have an ASP.NET website hosted on S1 which estimates costs to be £32.64 per month.

Looking at Application Insights I see nearly 45,000 requests in a 24 hour period - which is an average of 1 request every 2 seconds.

Over a period of about 36 hours I was charged nearly £24. The configuration is literally an App Service + Application Insights, nothing else, no databases or anything.

Surely this can't be right? How can I see why this is costing so much money?

# Answer
A Microsoft employee has confirmed there is a bug in the "Current balance" notification that pops up at the top of the page.

The value in that notification continues to go down at the same rate for up to 48 hours after you have deleted a resource (in my case a database).
> 2 votes

# Answer
If you turn on logging to file system first. Then open the log streaming blade. You'll be able to see the requests hitting the app service. Will help narrow down what is causing so many. Potentially the alwaysOn feature of web app, will send 1 request a minute to keep app alive. Do you have anything else probing the app for health. An application gateway, front door. Traffic manager etc. Or in app insights do you have some health probe configured. Or some perf testing. Either way, the log streaming will show you where all these requests are originating from.
> 1 votes

---
Tags: azure, cost-management
---


title: Dockerize a windows GUI application

text:
# Question
Title: Dockerize a windows GUI application
Is it possible to run a Windows GUI executable (notepad.exe for example) into a container ?

I want to do this because i do not trust some software i install on my personal computer and i want to be sure they are runned into a sandbox

Thanks

# Answer
I'd have to say that I don't think it's possible and if you understood windows containers you really wouldn't want to do it anyway. Last time I checked, a windows container weighs in at gigabytes++ and that's huge to run notepad.

You can run xwindows apps in a container, but windows is, basically, complete shite in a container.

If you want a sandbox then you should checkout VirtualBox or Hyper-V. You can script either with Hashicorp Vagrant.
> 3 votes

---
Tags: docker
---


title: CORS in Docker Engine REST Api

text:
# Question
Title: CORS in Docker Engine REST Api
I have activated my Docker Engine V1.41 REST API in docker with this article.

After that in my browser it works fine like http://localhost:port/container/list will show me all lists of my containers in json.

But now, when I try to reach the API with axios in react I get an CORS error:

So I think I have to setup cors for docker?

I found a bunch of articles about how to enable CORS for a docker image / container but nothing about enabling it for the docker agent / host itself.

Any answer?

# Answer
CORS is applied at the application layer, not the infrastructure (docker) layer. So, this has nothing to do with docker. It also has nothing to do with the axios client. Rather, your server needs to allow requests from your client by returning the correct headers via the OPTIONS method.

If you're using nodejs and express, you can add something like this to your service:

```
app.use((req, res, next) => {
  res.header("Access-Control-Allow-Origin", "*");
  res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept, Authorization");
  res.header("Access-Control-Allow-Methods", "GET");

  if (req.method === "OPTIONS") return res.status(200).json({});

  next();
});

```

Or, there are plenty of CORS libraries that can help you out.
> 2 votes

---
Tags: docker, cors
---


# Question
Title: Mass remove/add to multiple security groups cross region
Would it be possible to create a script or use something like terraform to mass remove/add security group entries across several regions under the one account based on the security group name? Specifically we are allowing remote access to a few instances based on public addresses that are changing frequently, so updating manually has become tedious. Thanks!

# Answer
Do you already manage your resources with Terraform or are you looking for something to iterate through your AWS environment to change those settings?

If you're currently not managing your environment with Terraform, you'll have to import your resources into Terraform before you can begin making changes to your environment.

Your request sounds more like something you'd use the AWS CLI or perhaps Ansible for. Terraform isn't intended to perform scripting tasks on an environment.
> 1 votes

---
Tags: amazon-web-services, terraform
---


# Question
Title: how to update .env file in production while Jenkins deployment
how to update the .env file in production while Jenkins deployment? we have not committed the .env file in git for security reasons.

# Answer
You can store secret file at Jenkins and use it for deployment. It can be achieved by using Credentials and Credentials Binding plugins.

1. Install Credentials Binding and Credentials plugins
2. Select your user name in the top navigation bar.
3. Select Credentials in the left pane.
4. Under Stores from parent, select (global) in the Domains column.
5. Select Add Credentials in the left pane.
6. insert into your pipeline script

```

withCredentials([file(credentialsId: 'env', variable: 'mySecretEnvFile')]){
    sh 'cp $mySecretEnvFile $WORKSPACE'
}

```
> 4 votes

---
Tags: jenkins, jenkins-pipeline
---


# Question
Title: How do I SSH into an EC2 instance from AWS gitlab-runner?
I have setup my own gitlab-runner on AWS following GitLab's walkthrough (Autoscaling GitLab Runner on AWS EC2).

One step in my deployment is to SSH into another EC2 instance. The SSH connection fails every time (see below for the one exception).

I have added an Inbound rule to my security group for the EC2 instance that allows SSH connections from the security group for both the gitlab-runner manager and the gitlab-runner machines. Still SSH connections are refused. Allowing SSH traffic from 0.0.0.0/0, however, causes the SSH connection from the gitlab-runner to the EC2 instance to succeed.

Why doesn't allowing my gitlab-runner security group in the inbound rules work, and how do I fix this so that I don't have to open my EC2 instance to the world to get SSH access from gitlab-runner?

# Answer
When you use public IP the traffic exits your VPC and comes back in, at which point it is no longer identified as coming from that security group. So you can adjust your ssh connection and use private IP
> 2 votes

---
Tags: amazon-ec2, gitlab, gitlab-ci-runner
---


title: Terraform variable Validation

text:
# Question
Title: Terraform variable Validation
is it possible to use validation of a variable type equals list(string)?

My example:

**main.tf**

```
  name            = "tf-prd-sg-foo"
  image_id        = "ami-00e788542ee66c64f"
  instance_type   = "m5.xlarge"
  security_groups = ["sg-000000", "sg-00000000"] # Example of SG only to contextualize
}

```

**vars.tf**

```
variable "security_groups"  {
    type        = list(string)
    default     = []
    description = "(Optional) A list of associated security group IDS."

    validation {
      condition = can(regex("^sg-", var.security_groups))
      error_message = "The Security Groups need starts to \"sg-\"."
    }
}

```

When I try execute returns the following error:

```
[terragrunt] 2021/01/05 17:12:10 Running command: terraform plan

Error: Invalid value for variable

  on vars.tf line 21:
  21: variable "security_groups"  {

The Security Groups need starts to "sg-".

This was checked by the validation rule at vars.tf:26,5-15.

[terragrunt] 2021/01/05 17:12:12 Hit multiple errors:
exit status 1

```

# Answer
> 4 votes
When validating an input variable of a collection type, we typically want to write a validation rule that tests if something is true for each value within the collection, rather than true of the collection as a whole.

You can write a "for each" sort of condition using Terraform's `for` expressions. If you are using Terraform v0.14 or later then you can write it concisely using the `alltrue` function along with a `for` expression that produces a true or false result for each of the items, like this:

```
  validation {
    condition = alltrue([
      for id in var.security_groups : can(regex("^sg-", id))
    ])
    error_message = "All security group ids must start with \"sg-\"."
  }

```

---

If you are using Terraform v0.13 then you won't have the `alltrue` function available to you yet, but you can get a similar result in a clunkier way by using the optional `if` clause of a `for` expression to filter out all of the valid ids and then checking whether the resulting list is empty -- that is, whether there are any remaining items that were *not* valid:

```
  validation {
    condition = length([
      for id in var.security_groups : id
      if !can(regex("^sg-", id))
    ]) == 0
    error_message = "All security group ids must start with \"sg-\"."
  }

```

This second example is a bit more cryptic to understand because it's using some language features in an atypical way, so I'll break it down into smaller parts to explain:

* `[for id in var.security_groups : id if !can(regex("^sg-", id))]` filters the elements of `var.security_groups` to keep only the ones where the expression is true.
* `!can(regex("^sg-", id))` uses the `!` operator to turn your original expression that decides if the value is *valid* into an expression which decides if the value is *invalid*. Consequently the result of the `for` expression will be empty unless there's at least one invalid value.
* Finally, the `length( ... ) == 0` around the `for` expression says that the value is only valid if filtering out all of the valid values produces an empty list, which conversely means that the list didn't contain any invalid values.

---
Tags: terraform, terragrunt, aws-security-group
---


# Question
Title: What's the difference between travis-ci.org and travis-ci.com?
A few of the GitHub projects I contribute to use Travis CI for build testing. However, I've noticed that some projects use travis-ci.org for build testing, while others use travis-ci.com. Both sites seem to function identically, even down to using the same UI.

**What's the difference between travis-ci.org and travis-ci.com? Why do some projects use one over the other?**

# Answer
> 82 votes
Travis CI originally created two separated platforms to differentiate between private repos / paid (travis-ci.com) and public repos / free (travis-ci.org).

However, as of May 2018, new users and projects (both private and public) should only use travis-ci.com (see this blog post). Note that **travis-ci.org will be closed down completely on December 31st, 2020** (see this newsletter).

Although still in beta—which is a bit weird since travis-ci.org will be shut down soon—Travis CI provides a well-documented migration guide.

# Answer
> 17 votes
As of May 2018, travis-ci.org is scheduled to shut down and should no longer be used. Please refer to beatngu13's answer for updated information.

---

### ~~Aside from pricing model, the two sites are identical.~~

~~* travis-ci.org is a free service for public open source projects.
* travis-ci.com is a paid service for private commercial projects.

From the FAQ page for Travis Pro
~~
> ~~Why can’t I find information on pricing on travis-ci.org? Travis CI is, and always will be, free for open source projects.
~~
>
> ~~For a list of plans and prices for private repositories, look at travis-ci.com/plans.~~

# Answer
> 2 votes
There is one profound difference which is a bit hard to find:

Unfortunately the free OSS public repos at `travis-ci.com` are restricted to 1000 free build minutes after which one will have to beg for extra build minutes for the 'trial plan'. Travis CI announced this new pricing model at 2020-11-02. Effectively this ends generous open source offering.

Read Jeff Geerlin's comprehensive experience of migration to `travis-ci.com`.

---
Tags: travis-ci
---


title: How to test a cluster of microservices?

text:
# Question
Title: How to test a cluster of microservices?
I took a look into Server Spec and it says it clearly:

> Remark: serverspec test suites are meant to be run against a single machine (or docker container). In other words, you should not try to issue a single rspec command that would harvest and run tests against multiple machines or containers. You need to issue one rspec command for each of them.

So I can do test per-server, which is good for some cases. But the problem is the following: my microservice architecture has a auto-discovery service, so some services are known after querying it. Is there any project to express this? I know I can tool around with ruby (or python if I choose testinfra or other)

Something integrates with Consul with serverspec or similar will be great, in 2014 people were looking for this, does anyone know of any project that tackled this issue?

# Answer
> 1 votes
If nothing exists to easily mock Consul, have you considered spinning up a Consul service along with your other microservices? Consul can run in Docker, so this should be fairly straightforward if you use docker-compose and your apps can be configured to point to a different Consul endpoint.

Once your docker-compose is configured to spin up all the infrastructure/microservices required for your test, you will be able to use whatever tools you're familiar with to test your microservices, e.g. Selenium.

# Answer
> 1 votes
I like SaltStack because the configuration management provides node testing if you want in the states, per node, and the results are coalesced in the job status and fully accounted for in the returners.

SaltStack states also allow you to generate events on successes or failures, and it allows nodes to advertise (mines) arbitrary endpoint data (ie. which ip:port is it listening on), available to other nodes' configuration states.

The combination of the two gets you what you probably want from Consul.

# Answer
> -1 votes
locally you can test against kind (kubernetes in docker) cluster or k3d which is tiny k3s cluster in docker.

Just spin up the cluster, install bundle of microservices there and run the tests.

Have a look on https://github.com/AbsaOSS/k3d-action which demonstrates spinning such cluster.

---
Tags: testing, microservices
---


# Question
Title: How to install curl in conjunction with --no-install-recommends in Debian correctly?
I am building a Docker image based on `debian:buster-slim` and installing cUrl in this image. According to the recommendations made by Hadolint, I install all packages via `apt-get` with the option `--no-install-recommends` to avoid the installation of unneeded packages.

Installed in this way, cUrl is not able to get any file via HTTPS due to it is unable to find the "local issues certificate", as shown in the extract from Docker output below.

Does someone know which additional package I have to install to solve this problem?

```
Step 15/19 : RUN set -ex;     curl --output distribution.zip $FULL_URL
 ---> Running in 9aa176a788d7
+ curl --output distribution.zip https://www.host.domain/file.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

```

This is my current version of the use Dockerfile:

```
FROM debian:buster-slim

RUN set -ex;
    apt-get -y update

RUN set -ex; \
    apt-get -y install --no-install-recommends \
        curl=7.64.0-4+deb10u1 \
        unzip=6.0-23+deb10u1 \
        coreutils=8.30-3 \
        lsb-release build-essential ssh-client apt-transport-https \
        python gnupg

RUN set -ex;
    curl --output distribution.zip https://www.host.domain/file.zip

```

# Answer
> 2 votes
you need just to install `ca-certificates` so update your apt-get command to :

```
RUN set -ex; \
    apt-get -y install --no-install-recommends \
        ca-certificates \
        curl=7.64.0-4+deb10u1 \
        unzip=6.0-23+deb10u1 \
        coreutils=8.30-3 \
        lsb-release build-essential ssh-client apt-transport-https \
        python gnupg

```

---
Tags: docker
---


# Question
Title: 403 forbidden error on calling jenkins post api from python script
I am trying to build a python script that will build jobs externally but when I am executing it, throws 403 forbidden error for build jobs call other get APIs are working fine. Below is the python snippet.

```
j=jenkins.Jenkins('http://localhost:8080/login?from=%2F', 'admin', 'admin@123' )

j.get_jobs();

print(j.jobs_count());

print(j.get_jobs());

j.build_job('demo1');

```

Jenkins details: I am able to create & build jobs from UI successfully. My Jenkins version is 2.237.

Other settings.

* logged in user can do anything. Csrf protection is disabled. On the other hand my friend having the same script and version of Jenkins is able to build job.

# Answer
> 1 votes
As recommended by Jenkins (\>2.2xx) you may benefit by creating an API token for the user, I did so, to trigger Jenkins job in my local Jenkins server from NODEJS code.

Please refer below for further clarification:

1. https://www.jenkins.io/doc/book/system-administration/authenticating-scripted-clients/#shell-with-curl
2. https://www.jenkins.io/doc/book/using/remote-access-api/#RemoteaccessAPI-CSRFProtection

And for details: https://www.jenkins.io/doc/book/managing/security/

---
Tags: jenkins, python
---


# Question
Title: Isolate service networks with Docker Compose
I have two containerised services, A and B, with which I wish to isolate network communications with docker compose. There is also a shared service C which is not isolated from A or B and is free to communicate with either.

I have tried to achieve this with the following docker-compose.yaml:

```
version: "3.3"
services:
  C:
    build: C
    depends_on:
      - A
      - B
    expose:
      - 8080
    networks:
      - A
      - B
  A:
    build: A
    expose:
      - 8080
    networks:
      - A
  B:
    build: B
    expose:
      - 8080
    networks:
      - B

networks:
  A:
    internal: true
  B:
    internal: true

```

With this, C is able to connect to both A and B on the hosts A:8080 and B:8080 respectively and A and B are isolated as required. However, neither A nor B is able to connect to C on C:8080. Removing the network options from the compose file, all services are able to talk to each other. What am I doing wrong?

# Answer
I don't experience same issue with similar setup to what you describe. 'A' & 'B' can ping & access services on 'C' just fine. Few things to look at:

* Port aside, can you ping 'C' from either of the other containers?
* You probably shouldn't (can't?) name containers capital 'A', 'B', 'C' if you're going to build them in docker-compose. If that's what's actually used, try more meaningful names.
* Does name resolution work if you do `nslookup c` from either a & b? It should resolve via `nameserver 127.0.0.11`
* firewall on 'C' or service not up on `c:8080`?
> 1 votes

---
Tags: docker, docker-compose, networking
---


# Question
Title: How to calculate disk space required by Prometheus v2.2?
We are trying to calculate the storage requirements but is unable to find the values needed to do the calculation for our version of Prometheus (v2.2).

Prometheus(v2.2) storage documentation gives this simple formula:

```
needed_disk_space = retention_time_seconds * ingested_samples_per_second * bytes_per_sample

```
* **retention\_time\_seconds**: We took our retention time of 720 hours and converted to `2 592 000`seconds.
* **bytes\_per\_sample**: We used `rate(prometheus_tsdb_compaction_chunk_size_bytes_sum[1d]) / rate(prometheus_tsdb_compaction_chunk_samples_sum[1d])` based on the formula from this site, except we did not have `prometheus_tsdb_compaction_chunk_size_bytes_sum` so used `prometheus_tsdb_compaction_chunk_size_sum` instead. This gives us ~`1.3`, which sounds about right.
* **ingested\_samples\_per\_second**: We cannot find a way to get this value. We tried the following:

+ This site suggest `prometheus_local_storage_chunk_ops_total`, but we do not have this metric.

    > Another useful metric to query and visualize is the prometheus\_local\_storage\_chunk\_ops\_total metric that reports the per-second rate of all storage chunk operations taking place in Prometheus.
    + This uses `rate(prometheus_tsdb_head_samples_appended_total[1h])`, but the result is ~`18600` suggesting `68TiB` storage.

    > Combined with rate(prometheus\_tsdb\_head\_samples\_appended\_total\[1h\]) for the samples ingested per second, you should have a good idea of how much disk space you need given your retention window.
    + As above, this issue mentions `rate(tsdb_samples_appended_total[5m])`, but probably meant (or what we got working is) `rate(prometheus_tsdb_head_samples_appended_total[5m])`
    > You can still get samples per second via rate(tsdb\_samples\_appended\_total\[5m\]).
    + This uses `prometheus_local_storage_ingested_samples_total`, but again, it is not available to us.
    > sample rate = rate(prometheus\_local\_storage\_ingested\_samples\_total{job="prometheus",instance="$Prometheus:9090"}\[1m\])
    + Finally this issue also says to use `rate(prometheus_local_storage_ingested_samples_total[2m])`, and once again, it is not available to us.
    > The other side is ingestion, which is way easier to reason with capacity-wise. You can find out how many samples your server is ingesting with this query: rate(prometheus\_local\_storage\_ingested\_samples\_total\[2m\])

Any assitance or nudge in the right direction would be much appreciated.

**\>\>\> What worked in the end \<\<\<**

As was pointed out, our math was broken, so the below metrics worked for the calculation.

* **ingestion\_samples\_per\_second**: `rate(prometheus_tsdb_head_samples_appended_total[2h])`
* **bytes\_per\_sample**: `rate(prometheus_tsdb_compaction_chunk_size_sum[2h]) / rate(prometheus_tsdb_compaction_chunk_samples_sum[2h])`

# Answer
> 4 votes
I think your math may be off. If you use the ~18600 number you found I get a very different result from 68TiB:

`2,592,000 (seconds) * 18600 (samples/second) * 1.3 (bytes/sample) = 62,674,560,000 (bytes)`. This is ~62.675 Gigabytes (divide the digital storage value by 1e+9 ) and I'm guessing a more reasonable number for your infrastructure.

# Answer
> 8 votes
To calculate disk space required by Prometheus v2.20 **in bytes**, use the query:

```
retention_time_seconds *
rate(prometheus_tsdb_head_samples_appended_total[2h]) *
(rate(prometheus_tsdb_compaction_chunk_size_bytes_sum[2h]) / rate(prometheus_tsdb_compaction_chunk_samples_sum[2h]))

```

Where `retention_time_seconds` is the value you've configured for `--storage.tsdb.retention.time`, which defaults to `15d` = `1296000` seconds.

---
Tags: prometheus
---


# Question
Title: Best practices for app and infrastructure code repositories
We're looking for guidance on how to structure application code versus infrastructure code (IaC). More specifically, are there typical advantages/disadvantages to storing the two types of code in the same repository versus in different repositories?

A bit more background: We're on the journey to improve our ability to deliver systems more effectively. At the moment, we have application code in one repository and infrastructure code in a separate repository. We've newly begun having the app team change app-level infrastructure code. With that separation of repos, keeping the changes in sync introduces overhead. We think that bringing in the app-specific infrastructure code into the app repository would be advantageous.

Primarily, we're looking for info that helps to answer the question of which is the more effective option, one repo or two.

# Answer
> 50 votes
The correct answer, like almost everything in IT, is, "it depends." It depends on the way you work, the type of company you are working for, your requirements, your non-functional requirements, and possibly a lot of other factors.

*tl;dr -- read the final section, it's the most important.*

## Layers

In this admittedly long answer, I'll talk about the 3 layers of an organisation's IT operations -- *infrastructure*, *platform*, and *application*. These 3 layers are blurred at the boundaries and there is a continual debate about their delineations, but if you're not familiar with this model, roughly:

* *Infra*: machines and VMs, security and resource groups, auto-scaling groups and elastic load-balancers -- stuff you typically provision with Terraform
* *Platform* is Kubernetes, OpenShift, IIS, etc. -- stuff you typically provision with Packer, kubectl, or helm (and, this is the first blurring, because EKS, AKS, GKE, are often provisioned with terraform too)
* *Application* is the business logic and software that runs your business, confusingly this is also often provisioned with kubectl or helm, and possibly packer if you're into the wonderful *immutability* model

## Monorepos

I think the question you're asking relates to the concept of 'monorepo', where all the code needed to build and deploy your application is contained in a single repo and versioned together. In this answer, I'll mainly address the benefits, techniques, and drawbacks of monorepos, though you don't have to go deep into monorepo culture for this to apply. Simply maintaining a single version for all your applications, infra, and platform will have the same effect.

When making your decision you need to factor in whether you are working with an app per cluster, as is becoming more common in modern K8S systems, or a shared cluster. Another factor is which team owns the various layers of the architecture and how they will distribute their output.

## Teams & Architecture

My experience comes mainly from large banks. In these organisations, we have hundreds (even thousands) of development teams and it is impractical and inefficient to expect each team to develop its own infrastructure and platform layers as well as their application. Distributing this effort to each team would be a governance and compliance nightmare and hiring that many people with the right skills would be impossible.

Instead, we have centralised teams for infra and platform, and they distribute their output to the development teams and we typically work with the cluster-per-app model. In modern orgs we work with a cross-functional team that has some skills in each of the areas we need, including DevOps, and each team member is also a member of a group that is focused on their speciality where they receive groupwide instruction.

We have 2 main models to provide access to the output of the centralised infra and platform teams:

1. The infra and platform teams can provision clusters on behalf of a dev team and provide the pipelines needed to deploy applications to it. This works well when a team has little or no DevOps capability, but it is far from being the best solution as it creates a dependency from the dev team to the infra and platform teams which usually causes impedance and added complexity.
2. The infra and platform teams can provide access to their git repos, and the dev teams can fork them into git submodules within their own mono-repos. This model is by far the most common and currently represents the best solution for highly regulated industries. One of the biggest benefits here is that a dev team can make the changes they need to make (to the infra and platform) and issue pull-requests for the application teams to manage, without having to coordinate the schedules of the teams and thus reducing impedance.

Both solutions have their problems and complexities, but both are manageable given good engineering leadership and tool support.

Solution 1 would require separate repos for infra, platform, and app, and this usually results in a need to synchronise the backlogs of the respective teams, which quickly becomes a logistical nightmare as the number of teams increases. Solution 2 would require the implementation of 'initial trust' base infra with something like *Open Policy Agent* (OPA) rules to ensure that dev teams are complying with organisational governance requirements (e.g. verifying that a dev team isn't running a bitcoin mining operation, or publishing pro-trump/fascist/racist propaganda to our public-facing websites). OPA is an amazing tool when decentralising responsibility for the lower layers of an app while maintaining central control and I can't recommend it highly enough.

The app per cluster model allows you to use a monorepo that can completely rebuild your app from the ground up with minimal human interaction. The alternative architecture, multi-tenanted, completely disallows the monorepo approach and you must have separate repos for each of the layers.

## Versioning & Configuration Management

The benefits of a monorepo are manifold, but one of the major benefits is having a single version for all layers of your application. One common problem in IT is called *Configuration Management* -- I refer here to the ITIL meaning, where we talk about which version of what software/hardware work together, rather than the folk-meaning which relates to configuration files.

This is such a hard problem to solve that many books have been written on the subject and ITIL has an entire discipline dedicated to it. Many of the larger banks I have worked for have entire departments dedicated to this discipline and to do it right usually requires the Enterprise Architecture department to keep continually updated models of every component and relationship in an organisation's entire IT estate.

The configuration management problem is largely solved with a monorepo. You'll never hear anyone say "I tried to deploy your software but you've written a helm 3 chart and our cluster only has helm 2 installed," or, "we patched the OS with the latest security updates and your application crashed." With a monorepo, as long as you have sufficient testing in place, your helm version will always be in sync with your helm files and your software will have always been tested with the latest security patches.

## Configuration files

I think the OP mentioned config files in one of the comments so I'll address that briefly here too.

If you are building software that a client will run, then configuration files per client or installation are often necessary. In all other cases, they are not needed and should be avoided. Service discovery should be the responsibility of a service-mesh (my preference is *Consul*), not a configuration file. Secrets should be provided via dynamically generated properties files housed in an in-memory filesystem only accessible to a single instance of your service. All other 'configuration' data should be managed by the platform, and it should be exactly the same for every environment. The days of placing the database name and port, and the database credentials, in your `database.properties` file were over a long time ago and you should move on now.

It isn't always possible to completely avoid configuration files. Where this is the case the files should preferably be 'owned' by the environment layer, not the application layer. These can be used to enable/disable platform services and control feature flags so as to customise a platform for your team's needs. These should mostly be interpreted by the deployment mechanism to ensure that only the things you actually use are deployed, rather than having things enabled/disabled at runtime, which is a big security no-no. You could use these configurations to down-scale a test environment so that its not as large as a prod environment, or to scale one up for load testing, or even to deploy to a less than six-sigma HA configuration for basic dev work.

Application-specific configuration is considered by many to be a code smell. It is often used to reference database names, ports, and the like. Your app should run in a container and should always run on port 80, so that doesn't need to be configured. Your database should always have a meaningful name and service discovery (DNS) should route you to the appropriate instance for your environment. Your memory and CPU constraints belong to the container, which is in the platform layer, not the app layer.

You may have some truly app-level configuration points but that is normally only true when you are deploying many instances of your apps for different clients (even if they are in your own organisation), in which case this is probably ok, but even then you should work on deriving them from the context in which your app runs, rather than hard-coding them in a config file unless absolutely necessary.

## Environments

Another major benefit from the monorepo approach is that you gain the ability to 'print' new environments. An application that is designed to take advantage of this (autoscaling, auto-healing, service-mesh, etc.) will allow you to have a test environment that is 100% aligned with your production environment. Further, if time-to-market is a prominent enough factor of your business plan, you can spin up multiple test environments and run your tests in parallel rather than sequentially.

This also means that developers can have a production-like environment to experiment with whenever they need one. They simply have to run the provisioning scripts for the app and away they go.

Of course, there is a cost to 'printing' environments and not everyone has the unlimited resources of a major bank to fall back on, so this capability must be managed, but if you engineer your environments to be automatically torn down when they're not being actively used then you will go a long way towards reducing these costs. OPA rules can also be used to ensure that boundaries are not overstepped and costs are managed efficiently without human interaction.

## Team Dependencies

Personally, I believe that the major factor in the productivity of individual teams is the relationships they necessarily have with their peers in different layers. If there are 300 dev teams, an infrastructure team will be continually inundated with requests for changes that must be understood, managed, prioritised, and scheduled -- this is a major effort in large orgs and can leave a team waiting months, or longer, for changes that support their needs, leading to project delays.

There are no winners here -- if the centralised team is working on your problem then they're not working on someone else's, so either you are delayed or the other 300 teams are. There are always 299 teams waiting to be serviced. It is a lose/lose situation and it simply does not work well at all. Monorepos with git's submodules (or similar structures) help fix this problem. Dev teams can make their own changes to their forked codebase and simply have it reviewed by an authorised representative of the centralised team and pull-requests can be used to manage this process.

## Feature Flags

Of course, nothing is perfect. We don't yet have a silver bullet in software engineering and monorepos are no exception -- here be dragons. One fatal but unfortunately common misconception centres around the concept of feature-flags. These flags show what is finished, production-ready, and can be released, and what is a work in progress and should not yet be released to production. Many articles written by engineers suffering from the Dunning-Kruger effect suggest that these flags should be used to ensure that unfinished code should not be executable in a production environment. In reality, these flags should stop code from being released to production at all if it is not yet ready. Non-production code should not be in production -- it is a huge security hole if it is released and this must be avoided.

## No choice really

As far as I understand it though, the structure of your repos and software architecture tends to follow the structure of the organisation you work for (see: Conway's Law). Ultimately, the deciding factor may not be related to software at all, but instead is related to your organisation's politics and maturity/sophistication/nature. You may experiment with counter-structures, but usually, these experiments fail to gain traction and are destined for the scrapheap.

If your organisation is split into separate teams with separate responsibilities then your repos will be split the same way. If you go for a cross-functional team approach then your repos will tend to be merged. If you just have one big team that's responsible for the success of the organisation, even if it has specialities then you will naturally end up with a monorepo and life will be good.

If your organisation is poorly structured, following the Harvard method espoused by various MBAs, of splitting the organisation into departments and assigning each of them a vice-president and then grouping them whenever there are more than 7 under a senior VP, then your path to working software will be fraught with problems and progress will be slow. I'm not trying to bash MBAs here, it's just that their models conflict with the primary activity of a lot of modern organisations, which is software development, and I feel that they need to modernise their approaches to take into account the opportunities provided by modern technology and big-data, rather than continuing a model that was developed by candlelight.

Wow, this has turned into a bit of a lecture. Sorry, I don't have time to make it shorter, but as I said up-front, this is a complex subject with a lot of variables. The last section is the most important though, so read that first -- it could save you a lot of pain.

# Answer
> 16 votes
Structuring application code, application configuration, and infrastructure in Git repositories is sometimes not the easiest thing to do, and there is not a single correct solution, instead it sometimes depends on how you want to work and what tools you use, e.g. monorepo with monorepo tools or one repo per app.

The classic book Continuous Delivery touches on this topic. From my own experience working full time with this problem for a few years on a Kubernetes platform, I can confirm that it helps to follow the structure from that book to some extent, but again it also depends on how you want to work. If you use Kubernetes, the 2nd edition of Kubernetes Up & Running has a new chapter 18 about how to structure code and manifests in repositories.

**Repositories**

The Continuous Delivery book use a repository for *application source code* and a separate repository for *application configuration*. The repo for *application configuration* contains the environment-specific configuration for the app, for each environment (e.g. test, staging and prod). When apps are deployed using some form of IaC e.g. Terraform or Kubernetes manifests, this lives in the repo for *application configuration*. If you are using Kubernetes, Kustomize with manifests and overlays for each environment is useful to use.

The reason to the separation of the *source code* and *configuration* here, as explained in the book, is that they have different *lifecycles*, e.g. they change at different points in a pipeline. An example, you typically don't want to run all your integration tests if you only want to change a value in the configuration repo, e.g. a feature-toggle or when you do rollback to a previously working version.

## App Build Pipeline

* A repository for application source code
* Every `git-push` to the repository initiates a *Build Pipeline*-run
* The *build pipeline* typically builds the app, tests it, builds docker images, and perhaps runs integration tests. If everything has succeeded, it also pushes the image to an image registry. The last task in the pipeline would be to `git-clone` the *application configuration repository* and update the image reference to the newly built image.
* The result of this pipeline is that it either failed on a certain task, or that it succeeded. And if it succeeded on the mainline (not a feature-branch) you probably want to initiate a *Deployment Pipeline*, typically with a `git-push` to the *application configuration* repo.

## App Deployment Pipeline

* A repository for application configuration (and IaC)
* The *Build Pipeline* typically writes to this repo, for successful builds on the mainline. But also Developers or Operations may want to write to this repo for e.g. configuration changes like feature-toggles or change of URLs, perhaps with a PullRequest reviewed by team members.
* The *Deployment Pipeline* typically applies the configuration with e.g. `kubectl apply` or `terraform apply` using a deployment strategy of choice e.g. Rolling Deployment, A/B-testing deployment or a Canary Deployment - some of these strategies includes two parallel deployments.
* In a more advanced *Deployment Pipeline* you might want to automatically monitor metrics after the deployment and perhaps rollback if it looks bad (typically a part of Canary Deployment).

## Infrastructure Pipeline

Above I have explained how custom-developed applications are deployed using IaC. But there are also other kinds of infrastructure and IaC, like e.g. Kubernetes clusters, load balancers, or Kafka clusters. These are typically pre-built software, where your responsibility mostly is about *configuration*. Pipelines for this kind of infrastructure are similar to *App Deployment Pipelines* but you may want to run tests and integration tests on Pull Requests to this repository. The book Infrastructure as Code: Dynamic Systems for the Cloud Age is good about this kind of systems, tests, and pipelines.

* A repository for infrastructure configuration (possibly Terraform, Terragrunt, or possibly Kubernetes manifests).
* Tests (e.g. linting) and possibly integration tests (provision the infrastructure in an isolated environment and validate properties)
* Apply configuration (to one ore more environments) e.g. using `terragrunt apply` or `terraform apply` or maybe `kubectl apply`

## Terraform Module Pipeline

If you use Terraform, it is common that you want to compose your own modules. This may perhaps be similar for AWS Cloud Formation stacks. Changes to these modules should be tested in a pipeline and result in a unique version of the module using a `git tag`.

* A repository for your Terraform module (or perhaps you want to have multiple modules in the same repository).
* A pipeline running Terratest code validating your Terraform module in an isolated environment (e.g. a separate AWS account).
* Also validation and security scans with e.g. TfSec and TfLint.
* If your test is running on the mainline (e.g. after an approved PR and merge to mainline) you would typically end your pipeline with a task that creates an unique `git tag` \- that you later use with e.g. Terragrunt when you want to apply this version of the module to your infrastructure in the *Infrastructure Pipeline*.

*In the end, it all depends on how you want to work. Maybe you want to work with less process and PRs. Or maybe all your app source code is in a big monorepo, then you might want to work in a workflow and structure that fits with the tools you use.*

# Answer
> 3 votes
It is definitely advantageous for CICD since you can ship dependent infra and app code changes together, instead of having to manually synchronize it. This means you can run tests that also cover the infrastructure and reduce the likeliness of shipping a breaking change.

But you have to be wary of possible downside, especially pipeline times since infra changes can lead to longer feedback time.

Certain technologies such as pulumi can make this a lot easier, so you might need to research if it makes sense to invest in them, or make do with the current ones in use for your projects.

# Answer
> 2 votes
Personally I would always separate out the pure IaC code in a different repository.

The only "Ops" related thing that should reside in the application repository should be Makefiles, Docker-Compose or pure Dockerfiles to bring up test environments for local or CI/CD tests.

Additionally, you should follow the Principle of Least Privilege, and therefore you may have developers in the team who should not be able to tear down the whole infrastructure because they can work and look into all the code, states, and secrets in the same repository.

If you need to build up Infrastructure after you ship application features like: "We add this new app / microservice which then creates a Pub/Sub environment, a bucket for storage and some other stuff" I would consider looking into Terraform Modules (or as Michael Pereira already mentioned Pulumi). Maybe cdk8s could also fit, but this all depends on what "IaC" you are using and how.

---
Tags: infrastructure-as-code, repository
---


title: Jenkins connect to Windows 10 agent

text:
# Question
Title: Jenkins connect to Windows 10 agent
I have a Jenkins master on a Linux machine and I would like it to be able to connect to a Windows 10 agent, how do I do that?

I can't find much in the official documentation about agents in general, and as for searching the web, the results show how to do it using JNLP, which only works on Java 8 and not on 11, and I don't want to downgrade from 11 to 8.

I've been able to set up Linux agents, to which I connect using ssh (like this), but the same process doesn't seem to work for the Windows agent.

I am at wits' end here, do you know or have any up-to-date resources on connecting to Windows agents? Thanks.

# Answer
> 0 votes
Have you checked the following link: https://wiki.jenkins.io/display/JENKINS/Step+by+step+guide+to+set+up+master+and+agent+machines+on+Windows I advise you to configure a node on the master ( manage --\> manage nodes ) and set the `launch method` to `launch agent by connecting it to the master`. After that you have three methods to connect the Windows agent to Jenkins master, the easiest way will be to lanch it from a browser.  This should work enven with jdk 11.

# Answer
> 0 votes
I've added a Jenkins windows slave to a Jenkins master running on Linux not too long ago and wrote the steps I've taken for future use.

Here's what needs to be done:

Jenkins service on the windows slave needs to be started with a dedicated user rather than local system account.

Then I had some problems when I tried connecting from the master jenkins to this windows-jenkins-slave of "Access denied" - when the master tried to start the local service.

In order to avoid "Access denied" errors when the master tries connecting to windows slave, follow these steps:

* Create a dedicated user for jenkins which is a member of the local administrative group on the windows-jenkins-slave.
* Allow the local administrator group full control over the following registry values:

* HKEY\_LOCAL\_MACHINE\SOFTWARE\Classes\Wow6432Node\CLSID{72C24DD5-D70A-438B-8A42-98424B88AFB8}
* HKEY\_CLASSES\_ROOT\CLSID{76A64158-CB41-11D1-8B02-00600806D9B6}

And then reboot the windows agent to apply the changes to the registry.

Then, Configure the node in master Jenkins like so:

this jenkins user is the local user I've created.

* Add this local jenkins user to the jenkins service as the log on user.

---
Tags: jenkins
---


# Question
Title: IaC Deployment in Azure with Terraform + Chef vs. Puppet vs. Ansible for Data Engineers
I am an *Azure Data Engineer* and try to leverage the IaC benefits in this area of *Azure Data Factory, Azure Databricks, Azure SQL & Synapse, Azure Power BI, Azure IoT*. After some research I figured out that I need two essential parts:

```
1.Provisioning part
2.Configuration management part

```

Breaking down to technology I decided to use *Terraform* for the provisioning of infrastructure like (RG, Network, DBs etc). For the configuration management I am still not sure what to choose between *Chef, Puppet and Ansible*. There is another idea to use *Docker/Packer* \+ *Kubernetes*.

What are the best practices of IaC and automation when it comes to the Area of Data Engineering/Big Data?

# Answer
Cloud setup could be done completly by Terraform. Check out terraform provider resources for required parameters ex. Azure Data Factory

Configuration management stuff mostly in use for OS tuning inside Virtual Machines. So maybe you don't need them at all.
> 1 votes

---
Tags: ansible, terraform, chef, infrastructure-as-code, puppet
---


# Question
Title: How to add a static slave to Jenkins running in Kubernetes?
I've installed a Kubernetes cluster and deployed Jenkins in it which dynamically starts slave machines using Kubernetes plugin for Jenkins.

Some builds require a specific build machine with GNU make and specific kernel packages and for that I have a dedicated static virtual machine.

How can I automatically add the static virtual machine to Jenkins as a permanent slave? so when a Jenkins pod goes down and another starts it would automatically connect to the static slave?

# Answer
> 1 votes
Since I'm using Jenkins official helm chart which contains JCasC plugin (Jenkins Configuration as Code), I was able to configure the static slave attachment using it including the creation of relevant credentials.

Here's the relevant section of the values.yaml file I use to install/upgrade the Jenkins helm chart:

```
JCasC:
    defaultConfig: true
    configScripts:
      credentials: |
        credentials:
          system:
            domainCredentials:
              - credentials:
                  - usernamePassword:
                      scope: GLOBAL
                      id: "CentOS_7.1_k4.14.11_ssh_password"
                      username: 'jenkins'
                      password: ${decodeBase64:BASE64ENCODEDPASSWD}
                      description: "Username/Password Credentials for CentOS_7.1_k4.14.11"
      permanent-nodes: |
        jenkins:
          nodes:
            - permanent:
                name: "CentOS 7.1 k4.14.11"
                remoteFS: "/home/jenkins"
                numExecutors: 2
                labelString: "CentOS_7.1_k4.14.11"
                mode: EXCLUSIVE
                retentionStrategy: "always"
                launcher:
                  SSH:
                    jvmOptions: "-Xms256m -Xmx1024m -XX:MaxPermSize=1024m"
                    credentialsId: "CentOS_7.1_k4.14.11_ssh_password"
                    host: "10.1.66.120"
                    port: 22
                    sshHostKeyVerificationStrategy:
                      manuallyTrustedKeyVerificationStrategy:
                        requireInitialManualTrust: false

```

Now whenever I re-deploy or upgrade Jenkins helm chart, it automatically configures these credentials and add the static slave.

---
Tags: jenkins, kubernetes
---


# Question
Title: How can I make artifacts available to the QA team with a user-friendly UI?
Currently developers push to master, jenkins picks that up, builds it, and then I go to artifacts, download the APK files and upload them in a Microsoft teams group so that the QA team can download either the latest or any previous version I had uploaded before.

I want to change this process to make it more efficient. I want something with a UI that the QA team can go to and download whichever version of the app (APK files) they want.

I want it all to be automated, so as soon as I push the master, Jenkins will build and the files will eventually show in this portal, webpage or section of Jenkins and the QA team can download which ever version they want. How can I achieve this? Is there a Jenkins plugin for this?

# Answer
> 4 votes
It seems to me that you're looking for an artifact store such as Nexus or Artifactory.

Both solutions have Jenkins plugins which will simplify the process of uploading your artifacts to them.

You can find an example as to how to upload artifacts from your Jenkins pipeline to Nexus in this link and an example for Artifactory in this link.

Both solutions are very common and relatively easy to manage and use and are known by the community for a long time so you can find many tutorials that explain how they can be used and benefit you.

Then, you just need to allow your QA team read access to the relevant artifacts repo and they can download whatever version of the artifacts they need.

# Answer
> 0 votes
The answer @itai-ganot provides is the "best approach".

A "simpler" alternative is to leverage Jenkins "Archive Artifacts" option ; example here and provide your QA people Read access to Jenkins instance. Intro to Access Control

Be aware you should implement strategy to manage the builds and artifacts or you'll quickly run out of space (or accumulate garbage).

You can also copy those artifacts to another job workspace or elsewhere.

---
Tags: jenkins, continuous-integration, gitlab, jenkins-plugins
---


# Question
Title: Troubles with mapping ports on a very simple Docker container
I'm new to Docker and following a tutorial on Udemy.com. The turtorial has me setting and running a very simple container.

**Docker File**

```
# Specify a base image
FROM node:alpine

# Putting files into a specific directory
WORKDIR /usr/app

# Install some dependencies
COPY ./package.json ./
RUN npm install
COPY ./ ./

# Default command
CMD ["npm", "start"]

```

**Index.js**

```
const express = require('express');

const app = express();

app.get('/', (req, res) => {
    res.send('Hi there');
});

app.listen(5000, () => {
    console.log('Listening on port 5000');
});

```

**package.json**

```
{
    "dependencies": {
        "express": "*"
    },
    "scripts": {
        "start": "node index.js"
    }
}

```

I'm working on a laptop running Windows 10 Home. Using the Docker Quickstart Terminal and PowerShell.

I've edited my HOSTS file as to point localhost to 192.168.99.100, which is the ip the container gets.

I've tried hitting the server by both 192.168.99.100:5000 and localhost:5000. No matter what, when I try to access via browser(I.E. and Chrome) I get the following message:

This site can’t be reached

192.168.99.100 refused to connect.

---

> curl http://192.168.99.100:5000

returns

> curl: (7) Failed to connect to 192.168.99.100 port 5000: Connection refused

---

However, If I run `'docker-machine ls'`, I get:

> tcp://192.168.99.100:2376

Why port 2376? I cannot figure out where/how this port is being assigned.

Trying the browser on that port I get a different message:

This page isn’t working

192.168.99.100 sent an invalid response.

ERR\_INVALID\_HTTP\_RESPONSE

---

This would suggest(at least to me) that the server is there on that port, but not doing something right.

```
curl http://192.168.99.100:2376 --output -

```

Gives me some garbled output in the terminal, and writing it to a file gives garbled output as well. Not much, just a few garbage characters.

I suspect this is the "Hi There" message that is not being formatted correctly.

So my questions would be, am I doing something wrong in my dockerfile? Why/how is the container getting this port 2376 when i'm specifically mapping to 5000?

Any insight would be appreciated.

# Answer
> 1 votes
May be you are using bridge network mode. In that case, you need to bind your host port to container port to use same port. So when you execute docker run, you may need to add -p 5000:5000 as a flag to docker run command. Also I don't see any expose in Dockerfile. So you may want to add EXPOSE 5000 in Dockerfile just above cmd.

# Answer
> 0 votes
### Containers have to expose ports

A container will not listen to outside traffic unless the Dockerfile has an `EXPOSE 5000` command. This tells the container to listen for external traffic on that port.

That being said, the Docker engine will only listen to external ports that you tell it to. By default that is none. Setting `EXPOSE 5000` means that the container will listen to external traffic that the docker engine routes to it. By default this means only internal docker engine traffic will be able to access that port.

### Expose the port to the host

Depending on how you are starting the container (either cli or docker-compose), you will need to expose the port to the host. From CLI, this will be done with the `-p` flag in the format `-p hostPort:containerPort`. You can then map the port as you wish, like `-p 50000:5000` with 5000 being the same port in your `EXPOSE` command in the Docker file and 50000 being the `localhost` port you want to use.

### Putting it together

Update your docker file to something like this:

```
# Specify a base image
FROM node:alpine
...

EXPOSE 5000

# Default command
CMD ["npm", "start"]

```

Then run it with:

```
docker run --rm imageName -p 5000:5000

```

From there you should be able to `curl http://localhost:5000` and get the expected response.

---
Tags: docker, dockerfile
---


# Question
Title: How to redirect the request to a different node when there is an unexpected exception and one of the pods gets restarted
We are deploying the application in a Kubernetes cluster and when we face an unexpected error and the pod gets restarted or goes down we are getting a 502 Gateway error even though there is another pod that should start accepting the requests without getting the 502 gateway error.

We are making sure that the application is cluster enabled and the request is redirected to the another pod automatically when we stop/kill the pod gracefully but doesnot happen on a exception.

# Answer
> 1 votes
You should configure a Readiness Probe in the `Deployment` for your app. When using a *Readiness Probe*, no traffic will be sent to instances that are not prepared to receive traffic.

---
Tags: docker, amazon-web-services, kubernetes, containers
---


title: Provisioning meaning

text:
# Question
Title: Provisioning meaning
> Provisioning is the process of setting up IT infrastructure. It can also refer to the steps required to manage access to data and resources, and make them available to users and systems.
>
> Provisioning is not the same thing as configuration, but they are both steps in the deployment process. Once something has been provisioned, the next step is configuration.

I find the following lines confusing

"It can also refer to the steps required to manage access to data and resources, and make them available to users and systems."

"Provisioning is not the same thing as configuration, but they are both steps in the deployment process"

How can you make your resources available to the user without setting them up and **configuring them**. **So why isn't configuration a part of provisioning?**

Also what **exactly do you mean by setting up the resources**?

# Answer
As per my understanding on cloud native infra.

Provisioning involves creating resources like VMs/instances, setting up IAM roles and policies, Firewalls, DBs, Load balancers, any clusters like EKS, GKE, etc.

And, configuration part involves, preparing your instances to run actually your apps by installing necessary softwares and packages, setting up tables in DBs and populating data, RBAC controls on resources, etc.
> 2 votes

# Answer
The term is used in different ways by different people and in different contexts. I like to think of each layer of 'the whole thing' as being provisioned individually, so I provision machine or container images by installing software on them using docker or packer, then I provision the infrastructure using terraform, and finally, I provision my application using kubectl or helm. Though, this is only a working model as these layers and tools tend to blend together at the edges.
> 0 votes

---
Tags: terminology, provisioning, term-disambiguation
---


# Question
Title: Service account permissions error when provisioning instance group from template
I'm trying to create a GCE Instance Group Manager using terraform

```
resource "google_compute_region_instance_group_manager" "frontend_group_manager" {
  name = "frontend-igm"

  base_instance_name = "frontend"
  instance_template  = "${google_compute_instance_template.frontend_instance_template.self_link}"
  region             = "${var.region}"

  depends_on = [ "google_compute_instance_template.frontend_instance_template" ]
}

```

When I try I get the following error:

> Error: Error waiting for Creating InstanceGroupManager: The user does not have access to service account 'One of \[serviceAccount:frontend@vpc-challenge-lab-245114.iam.gserviceaccount.com\]'. User: 'terraform@vpc-challenge-lab-245114.iam.gserviceaccount.com'. Ask a project owner to grant you the iam.serviceAccountUser role on the service account

When I validate the IAM policy on the account it appears to be configured correctly:

```
> gcloud iam service-accounts get-iam-policy frontend@vpc-challenge-lab-245114.iam.gserviceaccount.com
bindings:
- members:
  - serviceAccount:terraform@vpc-challenge-lab-245114.iam.gserviceaccount.com
  role: roles/iam.serviceAccountUser
etag: BwWMpQvtA3w=
version: 1

```

The template is configured as follows:

```
resource "google_compute_instance_template" "frontend_instance_template" {
  name = "frontend-template"

  instance_description = "Frontend Instances"
  machine_type         = "f1-micro"
  can_ip_forward       = false

  scheduling {
    automatic_restart   = true
    on_host_maintenance = "MIGRATE"
  }

  disk {
    source_image = "${data.google_compute_image.compute_image.self_link}"
    auto_delete  = true
    boot         = true
  }

  network_interface {
    network = "${google_compute_network.frontend_network.name}"
  }

  service_account {
    email  = "serviceAccount:${google_service_account.frontend_service_account.email}"
    scopes = ["default"]
  }

  depends_on = [
    "google_compute_network.frontend_network",
    "google_service_account_iam_member.frontend_service_account_permission",
    "google_project_iam_member.frontend_service_account_role_binding"
  ]
}

```

And the Service Account is configured as follows:

```
resource "google_service_account" "frontend_service_account" {
  account_id   = "frontend"
  display_name = "Frontend Service"
}

resource "google_service_account_iam_member" "frontend_service_account_permission" {
  service_account_id = "${google_service_account.frontend_service_account.name}"
  role               = "roles/iam.serviceAccountUser"
  member             = "serviceAccount:${data.google_service_account.terraform.email}"

  depends_on = ["google_service_account.frontend_service_account"]
}

resource "google_project_iam_member" "frontend_service_account_role_binding" {
  role   = "projects/${data.google_project.project.project_id}/roles/${google_project_iam_custom_role.base_compute_role.role_id}"
  member = "serviceAccount:${google_service_account.frontend_service_account.email}"

  depends_on = [
    "data.google_project.project",
    "google_project_iam_custom_role.base_compute_role",
    "google_service_account.frontend_service_account"
  ]
}

```

Finally, the custom role is as follows:

```
resource "google_project_iam_custom_role" "base_compute_role" {
  role_id = "base_compute_role"
  title   = "Base Compute Role"
  permissions = [
    "logging.logEntries.create",
    "monitoring.metricDescriptors.create",
    "monitoring.metricDescriptors.get",
    "monitoring.metricDescriptors.list",
    "monitoring.monitoredResourceDescriptors.get",
    "monitoring.monitoredResourceDescriptors.list",
    "monitoring.timeSeries.create"
  ]

  depends_on = ["google_project_service.iam"]
}

```

Why when everything appears to be in place does that error keep getting thrown by the GCE API.

# Answer
> 1 votes
Try adding permissions on your project level:

`gcloud projects add-iam-policy-binding <your-project-id> --member=serviceAccount:terraform@vpc-challenge-lab-245114.iam.gserviceaccount.com --role='roles/iam.serviceAccountUser'`

# Answer
> 1 votes
In the google\_compute\_instance\_template resource, the email property of the service\_account block should not start with "serviceAccount:". Change from:

```
service_account {
  email  = "serviceAccount:${google_service_account.frontend_service_account.email}"
  scopes = ["default"]
}

```

To this:

```
service_account {
  email  = "${google_service_account.frontend_service_account.email}"
  scopes = ["default"]
}

```

# Answer
> 0 votes
I fumbled upon this question today as i was struggling with this error, the way credentials are get created in azure is way much better than this google, however i had added service account user through the gcp portal and my terraform was able to create test vm, hope this help.

---
Tags: terraform, google-cloud-platform, google-compute-engine
---


# Question
Title: How to do blue-green deployment with PL/SQL code?
Is it possible to do blue-green deployment when your software is mostly a set of PL/SQL packages?

I don't really want to duplicate the database, the financial risks in case of any desyncs are too big. I understand that step 1 is to decouple schema changes from package changes, but how to proceed further?

Is splitting packages into a separate DB that I can spawn multiple copies of the only option?

# Answer
Yes, it is possible, but it takes a bit of work. I assume that the concern with duplicating the database is 1) time and 2) size/storage, otherwise it seems like it would be trivial to spawn databases for testing against. Based on the fact that you are targeting PL/SQL i'm guessing you are working with Oracle.

You will either need: 1) a smaller-scale dataset that can act as a representative sample of you data which is sufficient for testing all of your corner cases or 2) to clone the *whole* production database. Honestly, I recommend option #2. There are numerous cases where things operate correctly at a smaller scale, but run into issues once scaled up - especially with database data. Ideally, you could store and replay a sample of queries for load testing using a database proxy like ScaleArc (if Oracle won't let you store your queries).

Saying you need to clone your whole dataset may make me sound like a kook and like it will take forever, but you may find this answer in which I detail a methodology for cloning multi-terabyte databases in seconds without significantly increasing your storage costs due to duplication and delta-snapshotting useful. This may be harder if you are using Oracle's proprietary storage system. Be forewarned, the tech isn't cheap, but if you wanted cheap, you wouldn't be using Oracle. The best way to sell that to your upper management is to prove that the tech is cheaper than the outage/data mangling it would prevent. It's your insurance policy against outages - so how much insurance are you willing to buy?
> 3 votes

# Answer
This is very easily possible, with no change to any code (neither the PL/SQL code nor your application code, if there is some other application like Java or whatever):

1. Put all PL/SQL in one (or more) separate Oracle schema. If your application today uses only one "myapp" schema for tables and PL/SQL, then in the future you will have "myapp" (tables, indexes etc.) and "mycode" (or whatever) which *only* contains PL/SQL procedures.
2. In "myapp", create synonyms for all the objects that now live in "mycode".
3. In "mycode", you can create stub tables - i.e., the same tables as in "myapp", but without any data. You do not need any indexes or whatnot, just the tables. These are only used at compile time; they are especially not used by the optimizer, later.

That's it for separating your code and data. The next step obviously is to create "mycode2" as a clone of "mycode", and from then on you simply need to switch the synonyms in "myapp" between those two. You can have as many code schemas as you want, and switch between them freely with some rather trivial tooling.

(Setting the proper options, grants, making sure that the PL/SQL uses the tables of the caller instead of the compile unit left as an exercise to the reader... - feel free to ask if anything is really unclear.)

Practical hints: It is true that you have to change 5000 synonyms (well, hopefully you do not have 5000 PL/SQL packages :) ), but that is an uncritical operation insofar as there is nothing that can go wrong (both `drop synonym` and `create synonym` can not go wrong for any reason I am aware of; unlike e.g. `drop table` which can lock if the table is in use). You will obviously script those drops; and you will also probably query the synonyms you have to set by `select object_name from all_objects where user_name='MYCODE2' and object_type in ('PACKAGE', 'PROCEDURE')` or something like that to automatically catch all of them.
> 1 votes

# Answer
For non-data components such as DDLs or PL/SQL I would recommend using Liquibase to manage them. However, to do Blue Green deployment with sub-second downtime is likely an impossibility because you need to stop the requests to the old database and route them to an exact copy of the new database at the time with your delta being applied.

The closest I can think about doing it is as follows. For the purposes of the sequence I have the liquibase migration scripts as part of the application startup to avoid having multiple steps for application startup.

## Verification

1. Copy Green DB and other data artifacts such as file systems etc. to Blue. This is a time and size factor.
2. Deploy and start Green application blue.
3. Deploy and start the new application on Blue which applies liquibase scripts.
4. Verify

## Switch over

1. Pause the Green application (this prevents any future access to the data). This begins the downtime.
2. Copy Green DB and other data artifacts such as file systems etc. to Blue (again). This is a time and size factor.
3. Deploy and start Green application blue.
4. Deploy and start the new application on Blue which applies liquibase scripts.
5. Switch route to Blue

## Switch over and Fallback (note new data loss)

(from above)

6. Restart green application
7. Switch route back to Green

## Reducing cost

* Instead of a new database or ElasticSearch or Kafka instance, just use a different schema and credentials between blue and green.
* Instead of a new storage, use a different root folder between blue and green.
> 0 votes

---
Tags: deployment, databases
---