title: Combine Lists of Objects in Ansible

text:
# Question
Title: Combine Lists of Objects in Ansible
I'm really trying to merge lists of objects along with defaults so I can loop over them and create resources in kubernetes. I've gotten it down to a simple(ish) playbook and will post it here.

```
- hosts: localhost
  vars:
    default_thing_1: 123
    default_thing_2: 456
    default_transport: train
    list_of_things:
      - name: foo
        common_thing_1: "sam"
      - name: bar
        common_thing_1: "sam"
        common_thing_2: "ham"
      - name: biz
        common_thing_1: "eggs"
  tasks:
    - name: Set the Things Facts
      set_fact:
        thing:
          i_am: "{{ item.common_thing_1 | default(default_thing_1) }}"
          eggs_ham: "{{ item.common_thing_2 | default(default_thing_2) }}"
      loop: "{{ list_of_things }}"
      register: things_fact_results
    - name: Set Optional Things Facts
      set_fact:
        thing:
          transport: "{{ item.transport | default(default_transport) }}"
      when: "item.common_thing_2 is defined and item.common_thing_2 == 'ham'"
      loop: "{{ list_of_things }}"
      register: transport_fact_results
    - name: Debug the Facts
      debug:
        msg: "{{ item.0 | combine(item.1|default({})) }}"
      loop: "{{
        things_fact_results.results |
        map(attribute='ansible_facts.thing') | list |
        zip(
          transport_fact_results.results |
          map(attribute='ansible_facts.thing') | list
        ) | list
      }}"

```

The output is as follows...

```
PLAY [localhost] ***********************************************************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************
ok: [localhost]

TASK [Set the Things Facts] ************************************************************************************************************************************************************************
ok: [localhost] => (item={'name': 'foo', 'common_thing_1': 'sam'})
ok: [localhost] => (item={'name': 'bar', 'common_thing_1': 'sam', 'common_thing_2': 'ham'})
ok: [localhost] => (item={'name': 'biz', 'common_thing_1': 'eggs'})

TASK [Set Optional Things Facts] *******************************************************************************************************************************************************************
skipping: [localhost] => (item={'name': 'foo', 'common_thing_1': 'sam'})
ok: [localhost] => (item={'name': 'bar', 'common_thing_1': 'sam', 'common_thing_2': 'ham'})
skipping: [localhost] => (item={'name': 'biz', 'common_thing_1': 'eggs'})

TASK [Debug the Facts] *****************************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"msg": "Invalid data passed to 'loop', it requires a list, got this instead: [({'i_am': 'sam', 'eggs_ham': '456'}, AnsibleUndefined), ({'i_am': 'sam', 'eggs_ham': 'ham'}, {'transport': 'train'}), ({'i_am': 'eggs', 'eggs_ham': '456'}, AnsibleUndefined)]. Hint: If you passed a list/dict of just one element, try adding wantlist=True to your lookup invocation or use q/query instead of lookup."}

PLAY RECAP *****************************************************************************************************************************************************************************************
localhost                  : ok=3    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

```

I think if I'm reading the loop item that's what I'm looking for. Though I'm not sure why ansible is refusing to perform the loop.

Any help about how to merge lists of objects together with optional attributes that could or could not be set because k8s is pretty picky about what's being set and not when passing in objects.

Thanks.

# Answer
`loop` requires a valid list. But we got here is,

```
[({'i_am': 'sam', 'eggs_ham': '456'}, **AnsibleUndefined**),
 ({'i_am': 'sam', 'eggs_ham': 'ham'}, {'transport': 'train'}),
 ({'i_am': 'eggs', 'eggs_ham': '456'}, **AnsibleUndefined**)]

```

Since no value is assigned to `transport_fact_results` when `common_thing_2` is not defined.

if you remove the `when` condition it will work.

CODE:

```
- hosts: localhost
  vars:
    default_thing_1: 123
    default_thing_2: 456
    default_transport: train
    list_of_things:
      - name: foo
        common_thing_1: "sam"
      - name: bar
        common_thing_1: "sam"
        common_thing_2: "ham"
      - name: biz
        common_thing_1: "eggs"
  tasks:
    - name: Set the Things Facts
      set_fact:
        thing:
          i_am: "{{ item.common_thing_1 | default(default_thing_1) }}"
          eggs_ham: "{{ item.common_thing_2 | default(default_thing_2) }}"
      loop: "{{ list_of_things }}"
      register: things_fact_results
    - name: Set Optional Things Facts
      set_fact:
        thing:
          transport: "{{ item.transport | default(default_transport) }}"
      when: "item.common_thing_2 is defined and item.common_thing_2 == 'ham'"
      loop: "{{ list_of_things }}"
      register: transport_fact_results
    - name: Debug the Facts
      debug:
        msg: "{{ item.0 | combine(item.1|default({})) }}"
      loop: "{{
        things_fact_results.results |
        map(attribute='ansible_facts.thing') | list |
        zip(
          transport_fact_results.results |
          map(attribute='ansible_facts.thing') | list
        ) | list
      }}"

```

OUTPUT:

```
PLAY [localhost] ************************************************************************************************************************************************************************************************

TASK [Gathering Facts] ******************************************************************************************************************************************************************************************
ok: [localhost]

TASK [Set the Things Facts] *************************************************************************************************************************************************************************************
ok: [localhost] => (item={u'common_thing_1': u'sam', u'name': u'foo'})
ok: [localhost] => (item={u'common_thing_1': u'sam', u'common_thing_2': u'ham', u'name': u'bar'})
ok: [localhost] => (item={u'common_thing_1': u'eggs', u'name': u'biz'})

TASK [Set Optional Things Facts] ********************************************************************************************************************************************************************************
ok: [localhost] => (item={u'common_thing_1': u'sam', u'name': u'foo'})
ok: [localhost] => (item={u'common_thing_1': u'sam', u'common_thing_2': u'ham', u'name': u'bar'})
ok: [localhost] => (item={u'common_thing_1': u'eggs', u'name': u'biz'})

TASK [Debug the Facts] ******************************************************************************************************************************************************************************************
ok: [localhost] => (item=[{u'eggs_ham': u'456', u'i_am': u'sam'}, {u'transport': u'train'}]) => {
    "msg": {
        "eggs_ham": "456",
        "i_am": "sam",
        "transport": "train"
    }
}
ok: [localhost] => (item=[{u'eggs_ham': u'ham', u'i_am': u'sam'}, {u'transport': u'train'}]) => {
    "msg": {
        "eggs_ham": "ham",
        "i_am": "sam",
        "transport": "train"
    }
}
ok: [localhost] => (item=[{u'eggs_ham': u'456', u'i_am': u'eggs'}, {u'transport': u'train'}]) => {
    "msg": {
        "eggs_ham": "456",
        "i_am": "eggs",
        "transport": "train"
    }
}

PLAY RECAP ******************************************************************************************************************************************************************************************************
localhost                  : ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

```
> 1 votes

# Answer
So the way I figured it out was to put the `when` condition into Jinja.

```
- hosts: localhost
  vars:
    default_thing_1: 123
    default_thing_2: 456
    default_transport: train
    list_of_things:
      - name: foo
        common_thing_1: "sam"
      - name: bar
        common_thing_1: "sam"
        common_thing_2: "ham"
      - name: biz
        common_thing_1: "eggs"
  tasks:
    - name: Set the Things Facts
      set_fact:
        thing:
          i_am: "{{ item.common_thing_1 | default(default_thing_1) }}"
          eggs_ham: "{{ item.common_thing_2 | default(default_thing_2) }}"
      loop: "{{ list_of_things }}"
      register: things_fact_results
    - name: Set Optional Things Facts
      set_fact:
        thing: |
          {% if item.common_thing_2 is defined and item.common_thing_2 == 'ham' %}
          { "transport": "{{ item.transport | default(default_transport) }}" }
          {% else %}
          {}
          {% endif %}
      loop: "{{ list_of_things }}"
      register: transport_fact_results
    - name: Debug the Facts
      debug:
        msg: "{{ item.0 | combine(item.1) }}"
      loop: "{{
        things_fact_results.results |
        map(attribute='ansible_facts.thing') | list |
        zip(
          transport_fact_results.results |
          map(attribute='ansible_facts.thing') | list
        ) | list
      }}"

```

This is kinda sloppy but does allow me to specify the else condition to set the result over every iteration to something instead of `AnsibleUndefined`.

I don't really like this solution but it works.

```
$ ansible-playbook playbook.yml
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [localhost] *******************************************************************************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************************************************************************
ok: [localhost]

TASK [Set the Things Facts] ********************************************************************************************************************************************************
ok: [localhost] => (item={'name': 'foo', 'common_thing_1': 'sam'})
ok: [localhost] => (item={'name': 'bar', 'common_thing_1': 'sam', 'common_thing_2': 'ham'})
ok: [localhost] => (item={'name': 'biz', 'common_thing_1': 'eggs'})

TASK [Set Optional Things Facts] ***************************************************************************************************************************************************
ok: [localhost] => (item={'name': 'foo', 'common_thing_1': 'sam'})
ok: [localhost] => (item={'name': 'bar', 'common_thing_1': 'sam', 'common_thing_2': 'ham'})
ok: [localhost] => (item={'name': 'biz', 'common_thing_1': 'eggs'})

TASK [Debug the Facts] *************************************************************************************************************************************************************
ok: [localhost] => (item=[{'i_am': 'sam', 'eggs_ham': '456'}, {}]) => {
    "msg": {
        "eggs_ham": "456",
        "i_am": "sam"
    }
}
ok: [localhost] => (item=[{'i_am': 'sam', 'eggs_ham': 'ham'}, {'transport': 'train'}]) => {
    "msg": {
        "eggs_ham": "ham",
        "i_am": "sam",
        "transport": "train"
    }
}
ok: [localhost] => (item=[{'i_am': 'eggs', 'eggs_ham': '456'}, {}]) => {
    "msg": {
        "eggs_ham": "456",
        "i_am": "eggs"
    }
}

PLAY RECAP *************************************************************************************************************************************************************************
localhost                  : ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

```

Now the `transport` key is defined in the one object but not the other. So I can pass the item off to a complex object that's part of an API call.
> 0 votes

---
Tags: ansible
---


# Question
Title: Restart docker container from another container
I'm in a planning phase for a little tool for myself. I have to say that I have close to 0 experience with docker.

**My setup**

* 1 x Nodejs App in a Container (lets call it *App*)
* n x Instances/Containers of the same image (lets call it *Agent*)

**What I want**

I'm looking for a way to restart one of the *Agents* from within my *App*.

**The problem**
I just managed to create my first docker-compose. From here I'm stuck. I just have no idea how to cross-access the container from my *Apps* container (by script) or if this is even possible and then force to restart it (like `docker restart #hash` does).

**My current docker-compose.yml** (if that helps in any way)

```
version: '3'
services:
    app:
        container_name: app
        volumes:
            - .:/usr/src/app
        restart: always
        build: .
        ports:
            - '8080:3000'
        links:
            - agent

    # no control over this as this is an existing image from docker hub
    agent:
        restart: always
        image: image/from/dockerhub
        ports:
            - '1234'

```

... which I would run as `docker-compose up --scale agent=n`.

I'm not necessarily looking for a complete solution. If someone could guide me into the right direction I would really appreciate it.

Thanks

# Answer
In majority of circumstances I wouldn't recommend this in production, but to restart docker container from another container, you need to install docker in that container and then run that "master" container with

```
docker run -v /var/run/docker.sock:/var/run/docker.sock

```

as described here: https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/ (look for "The solution" part of the article).

But check your architecture and see if you really need to restart one container from another or if there is some other way.
> 2 votes

# Answer
You can open a socket to have the docker command available inside a container:

https://forums.docker.com/t/how-can-i-run-docker-command-inside-a-docker-container/337

But...even if technically possible docker experts suggest to not do because of security reasons.
> 0 votes

---
Tags: docker, docker-compose
---


# Question
Title: Terraform fails to modify DNS settings a recently created VPC Peering connection because it is not yet active
I have the following configuration file:

`environment/main.tf`:

```
resource "aws_vpc_peering_connection" "sb_vpc_peering" {
  vpc_id      = var.vpc_id
  peer_vpc_id = var.peer_vpc_id
  peer_region = var.peer_region

  #Other vars
}

resource "aws_vpc_peering_connection_accepter" "sb_vpc_peering_accepter" {
  vpc_peering_connection_id = aws_vpc_peering_connection.sb_vpc_peering.id
  auto_accept               = true
}

resource "aws_vpc_peering_connection_options" "sb_vpc_peering_options" {
  vpc_peering_connection_id = aws_vpc_peering_connection.sb_vpc_peering.id

  accepter {
    allow_remote_vpc_dns_resolution = var.accepter_dns_resolution
  }

  requester {
    allow_remote_vpc_dns_resolution = var.requester_dns_resolution
  }
}

```

The problem here is that last block in `resource "aws_vpc_peering_connection_options" "sb_vpc_peering_options"`:

```
  accepter {
    allow_remote_vpc_dns_resolution = var.accepter_dns_resolution
  }

  requester {
    allow_remote_vpc_dns_resolution = var.requester_dns_resolution
  }

```

This causes the following error:

```
Error: error modifying VPC Peering Connection (pcx-084fe8578b2935b6a) Options: OperationNotPermitted: Peering pcx-084fe8578b2935b6a is not active. Peering options can be added only to active peerings.
    status code: 400, request id: 2aa0a163-e9db-4c55-aee5-4f7ffbbf8b9f

  on ../../../aws/vpc/peering-connection-accepter/main.tf line 15, in resource "aws_vpc_peering_connection_options" "sb_vpc_peering_options":
  15: resource "aws_vpc_peering_connection_options" "sb_vpc_peering_options" {

```

If I give it a couple of minutes and run `terraform apply` again, it works as the VPC has been given time to become active.

I originally had that DNS resolution bit within the `aws_vpc_peering_connection` resource itself, but the same error occurred so I separated them like this but that did not fix the issue.

Do I need to somehow have some sort of sleep before applying the options?

# Answer
Figured it out! The problem is that those DNS settings set in `aws_vpc_peering_connection_options` cannot be set until the peering connection is active (approved). It was only depending on `aws_vpc_peering_connection` existing, therefore was running at the same time or before `aws_vpc_peering_connection_accepter`.

This was simply fixed in `aws_vpc_peering_connection_options` by getting the `vpc_peering_connection_id` from `aws_vpc_peering_connection_accepter` instead of `aws_vpc_peering_connection`, so that the terraform dependency tree would automatically have the dependency work in the correct order.

Before:

```
resource "aws_vpc_peering_connection_options" "sb_vpc_peering_options" {
  vpc_peering_connection_id = aws_vpc_peering_connection.sb_vpc_peering.id

  accepter {
    allow_remote_vpc_dns_resolution = var.accepter_dns_resolution
  }

  requester {
    allow_remote_vpc_dns_resolution = var.requester_dns_resolution
  }
}

```

After:

```
resource "aws_vpc_peering_connection_options" "sb_vpc_peering_options" {
  vpc_peering_connection_id = aws_vpc_peering_connection_accepter.sb_vpc_peering_accepter.id

  accepter {
    allow_remote_vpc_dns_resolution = var.accepter_dns_resolution
  }

  requester {
    allow_remote_vpc_dns_resolution = var.requester_dns_resolution
  }
}

```
> 4 votes

---
Tags: amazon-web-services, terraform
---


# Question
Title: How can I best deliver read only data assets to a Kubernetes application?
I'm migrating an HPC app to Kubernetes, and am trying to determine the best way to provide read only data assets as a configuration managed snapshot.

Previously, my team had delivered our application as a set of RPMs, but as we move to Kubernetes, we're delivering Docker images. This works fine for our application binary, as instead of a pile of RPMs that all have to agree, we can just deliver a known working image.

The problem, however, comes with our read-only data assets (similar to a game's asset files). Several different Docker images might rely on a single set of data assets, and so we'd prefer not to bake them into the actual Docker image itself (plus we want the ability to change the assets without having to recompile the application images).

We're unsure as to the best approach for this. The first idea would be to create a "data container" that simply runs NFS and serves out the data. This successfully isolates the data from the application and allows us to collapse a set of data RPMs into a single tagged docker image, but to me it seems like it might be overkill.

I know that we're essentially looking for a persistent volume for Kubernetes, but the problem for us is bundling all of the data into a single package that has the same delivery convenience as a Docker image.

**Is there a better way to provide this read-only data as a version controlled snapshot?**

# Answer
> 1 votes
> we'd prefer not to bake them into the actual Docker image itself (plus we want the ability to change the assets without having to recompile the application images).

This is indeed good properties to have. You are on the right track.

> The problem, however, comes with our read-only data assets (similar to a game's asset files).

This can be solved with multiple solutions, it's hard to tell what is the best for you. Here are some alternatives.

**Persistent Volume - ReadOnlyMany**

> I know that we're essentially looking for a persistent volume for Kubernetes, but the problem for us is bundling all of the data into a single package that has the same delivery convenience as a Docker image.

Yes, you could create a Persistent Volume with access modes:

```
accessModes:
- ReadWriteOnce
- ReadOnlyMany

```

then (by an automatic process)

1. Load your data onto the volume with a Job using access mode `ReadWriteOnce`
2. Load the volumes to your pods/services using access mode `ReadOnlyMany`

and keep the volume immutable, you create a new when you need to change content. You can also use multiple volumes this way.

**CDN**

> our read-only data assets (similar to a game's asset files).

This inherently also sounds what typically is served by Content Deliver Networks. But it depends on your use case if this is a good solution or not.

**Buckets**

Data for e.g. CDN is usually stored by "buckets" at cloud providers, that may also be an option, then your applications load data using HTTP typically. Minio may be an alternative if you want to do this in your cluster.

---
Tags: kubernetes, data
---


title: One Jenkinsfile or multiple?

text:
# Question
Title: One Jenkinsfile or multiple?
So I have a Jenkinsfile defining a build pipeline and then a Jenkins job (not pipeline) with a very simple deployment script for our Docker stacks.

Seeing that Jenkinsfiles can become as complex and powerful as one's coding skills are (seeing that they can groovy fairly well), would I have everything in one Jenkinsfile or have multiple Jenkinsfiles?

Consider this theoretical process:

```
Development
> Build artefact > Build Docker image
> Deploy Docker container to testing > tests & QA
> deploy to staging > have client check it
> deploy to production

```

Would all of that go into one Jenkinsfile and run different steps of the pipeline according to user input like

```
"which stages should be performed?
Please specify:
Build artefact,
build Docker Image, ..."

```

This would result in a complex Jenkinsfile but you'd only have one single file.

Or should this process be split into multiple Jenkinsfiles with multiple Jenkins jobs/pipelines for each Jenkinsfile to do a part of the process?

# Answer
It really comes down to personal preference.

One additional tool you might not be aware of are shared libraries for Pipeline. These allow you to quickly write custom Pipeline steps or factor out common Pipeline code without writing a Jenkins plugin in Java.

Between multiple Pipeline jobs and shared libraries, there are many ways to split up your job's code and there's no one right answer on how to do it. One suggestion I have is to determine what steps in your process are "atomic" - that is, what are the smallest group of steps that you envision admins/ops/devs/etc. might need to run on their own? Each "atomic unit" should then become its own job.

For instance, let's say you're automating a deploy process, and your process in a general sense looks like this:

* build
* deploy to dev
* deploy to prod

Then I would create three jobs, one for each bullet point. This gives you some nice advantages:

* You can re-run each job individually in case something fails instead of having to restart the entire process over from the very beginning. Deploy to prod failed? You only need to re-run the deploy to prod job. You don't need to rebuild or re-deploy to dev.
* Each stage in the process is separated out so you can see how often it succeeds or fail. Some stages in your process may be more robust or fragile than others; this kind of separation gives you insight into that.
* This level of abstraction makes it easy for non-ops people to perform ops tasks (if that is something you want/need at your org). When everything is monolithic, the only thing non-ops people can do is run the entire process from the beginning. When everything is split up into tiny jobs, you need intimate ops knowledge to know which pieces to run in what order. By separating your jobs into independent, easy-to-understand, appropriately bite-size chunks, a non-ops person only needs to press a single button to kick off an automated ops process.
> 8 votes

# Answer
Having been down a similar road, here are my suggestions:

Break your tasks apart by function; build is different to deploy, which is different to release. You can get more granular, but let's stick to the basics for now...

Secondly, if you want to consider who is permitted to do certain actions, you might break these functions apart by team (dev, test, release).

**Build pipeline:**

* This is probably your dev team.
* A single job to check out sources, compile and package.
* Perhaps run unit tests against this package
* Perhaps send the code to SonarQube for analysis
* If successful, publish this build artifact somewhere, like Artifactory/Nexus
* I automatically run this job whenever someone pushes to the repo
* Additionally, I have some rules which limits behaviour based on the branch (I don't really want to publish builds for every single feature branch, but always for release branches)

**Deploy pipeline:**

* This job should be be generic, and accept parameters that gives it context
* Are we deploying to a dev server? Or to Test/Pre-rel/Live?
* Create a deploy-dev job, and a deploy-prod job. Both use the same Jenkinsfile
* Who has permission to perform this action?
* What are we deploying? This is a dropdown of artifacts available in Artifactory.
* I use JobDSL to generate multiple jobs that use the same pipeline. One for dev/test/pre/live. You could manually create these jobs at first with appropriate deploy targets/users

The above lets me write highly targeted and specific pipelines, yet gives the teams the flexibility they need for day to day stuff. Dev and Test are self-service. Only the release team has access to deploy to pre-rel/live.

**Other pipelines:**

* Once that's working, you'll probably look at jobs that can bounce servers (self-service for teams) and promote builds from snapshot/staging/RC repos to gold repos in Nexus/Artifactory, etc.
* Are you using git-flow? How about jobs to create/finish release/feature branches, bump maven versions, tidy up stale branches, perform integration builds, etc. All good fun.

Hope this helps.
> 5 votes

# Answer
I recently built a build-test-publish-deploy-rollback pipeline and struggled with the question of one-or-many pipes. I decided on a single pipe to do all tasks, with the reason that it would be easy to go to one place, and tell the pipe what you want to do by entering parameters. It seemed like a good idea at the time, but I have smelled a few drawbacks since releasing it:

1. A success or failure of a given pipeline build doesn't mean the application code is necessarily bad. For example, we had a push to production fail due to a bug in the pipe code, but that doesn't mean the app build is bad. But the Build Status Badge shows `Failed` in our ReadMe. Which isn't true (the way we see it).
2. It is more complicated. There are several paths through the pipe, determined by the parameter values. More chance for error, harder to review, etc.
3. Blatantly breaks the principle of single responsibility, as well as the principle of least amazement. It would not necessarily do what one might expect based on some rules i'd built in, for the purposes of protecting the users. With a simpler pipe, there'd be less chance of error.
4. Our org isn't evolved to true Continuous Deploy yet, so there will always be a delay and a person pushing the "deploy" button, even to QA. We may move to CD into QA sooner rather than later, but we have to prove the safety and reliability of all of this before the higher-ups will be comfortable with CD into production.

All of which speaks to separate pipes with clearly delineated functions.

I am now building another pipeline for a service of the same type, and I plan to refactor to use separate pipes to build and deploy - same as @jayhendren listed above:

1. Build, Test (barely exists at this point), Publish Artifacts
    1. I am trying to build cookbooks to spin up temporary machines to deploy to give devs siloed test sandboxes.
2. Deploy to QA
3. Deploy to Prod

One thing worth mentioning is we are definitely using Multibranch pipes, at least for the build pipes, and Declarative syntax wherever possible. Thinking about it as I write this, the Build pipe can stay as it is now, as a multibranch pipeline job built off of the Jenkinsfile living in the app's repo, and building on any push to github. Then I can manually build additional pipelines for deploying - templated so they're generic, and only pass the unique variables to each one.

**Update**:

I want to come back to this and contradict myself. After more internal discussion on the design, and my own growth as a Jenkins user, I went back to a **single pipe for all tasks**.

The difference was using fewer parameters, and controlling the flow with the branch/PR that triggered the build. This way, although the tasks may vary, a pipe's output is based on its properties only.

Obviously this reduces the overall complexity by only having one pipeline, although the single pipeline's conditional checks may become more complex to handle all the conditions.

I should also add that doing the non-code design work like flow charts and diagrams helped expose the possibilities - going straight to code hid the big picture from me at first.
> 2 votes

---
Tags: docker, jenkins, jenkins-pipeline, jenkinsfile
---


# Question
Title: Azure DevOps pipeline upload to FTP after create directory on remote server
I developed a .net core console application and I want to deliver it to a FTP server.

I managed it in Azure Devops with FTP Upload task. All good here.

But before upload content to FTP, I want to make sure that remote directory exists, if not then create.

I put command `mkdir test` in FTP command but I got error that command is unknown and deployment is not done

How to achieve that ?

# Answer
> 1 votes
I'm just about positive that this Task is buggy and custom FTP Commands do not work at all. However, if the directory you specify in "Remote directory" does not exist it will be created so don't worry about doing a `mkdir`.

---
Tags: azure-devops
---


# Question
Title: How do we encrypt credentials in Jenkins console output
I can see credentials in Jenkins console output. How can we encrypt them? The "Mask Passwords plugin" is considered as not safe in my case.

# Answer
> 5 votes
Here's an example of using them in a pipeline safely. In this case, they are injected as environment variables and their value should never have to be shown unless you explicitly choose to print out the variable content for some reason or other.

Environment variables are generally seen as the best way to store secrets at this point and are heavily relied on by many modern deployments (e.g. often used in kubernetes apps).

https://www.jenkins.io/doc/book/pipeline/jenkinsfile/#handling-credentials

```
Jenkinsfile (Declarative Pipeline)
pipeline {
    agent {
        // Define agent details here
    }
    environment {
        AWS_ACCESS_KEY_ID     = credentials('jenkins-aws-secret-key-id')
        AWS_SECRET_ACCESS_KEY = credentials('jenkins-aws-secret-access-key')
    }
    stages {
        stage('Example stage 1') {
            steps {
                //
            }
        }
        stage('Example stage 2') {
            steps {
                //
            }
        }
    }
}

```

You can also avoid Jenkins altogether by having your pipeline/etc retrieve them from some other tool like HashiCorp Vault, AWS SSM, Azure Vault, etc.

# Answer
> 0 votes
John's answer works just fine. An alternative is to use the `withCredentials` block, which is built right into core Jenkins.

**Example**

```
pipeline {
  agent any

  stages {
    ...
    stage('Publish') {
      steps {
        withCredentials([usernamePassword(credentialsId: 'sonar-publisher', usernameVariable: 'SONAR_USER', passwordVariable: 'SONAR_PASSWORD')]) {
          // maybe do stuff with variables, echoing them prints *****
          sh './gradlew sonar'
        }
      }
    }
  }
}

```

The only requirement in this scenario is that the credentials `sonar-publisher` exists on the Jenkins master and your job has access to them.

---
Tags: jenkins, credentials
---


# Question
Title: Doing map and filter of a list of dictionaries in ansible
Given a list of users:

```
# Python Syntax
users = [{'name': 'alice', 'sudo': True}, {'name': 'bob'}, {'name': 'charlie'}]

# YAML Syntax
users:
  - name: alice
    sudo: yes
  - name: bob
  - name: charlie

```

How would I do the following in an ansible playbook:

```
# Python Syntax
[ dict(user, home="/home/{}/".format(user['name'])) for user in users if not 'sudo' in user]

# more verbose
for user in users:
  if not 'sudo' in user:
    user['home'] = '/home/' + user['name'] + '/'

```

resulting in:

```
# Python Syntax
[{'name': 'bob', 'home': '/home/bob/'}, {'name': 'charlie', 'home': '/home/charlie/'}]

# YAML Syntax
users:
  - name: bob
    home: /home/bob/
  - name: charlie
    home: /home/charlie/

```

Is this even possible/intended use of playbook language?

# Answer
> 4 votes
There's a lot that you can do, but I'm not sure with the existing filters you easily augment a dict.

Creating a new filter to do it was easier.

```
$ cat filter_plugins/addhomedir.py
class FilterModule:
    @staticmethod
    def add_home(_val):
        return [dict(user, home='/home/{0}/'.format(user['name'])) for user in
                _val if 'sudo' not in user]

    def filters(self):
        return {
            'add_homedir': self.add_home
        }
$ cat test1.yml
---
- hosts: localhost
  vars:
     users:
         - name: alice
           sudo: yes
         - name: bob
         - name: charlie
  tasks:
      - debug:
          msg: "{{ users | add_homedir | list }}"

```

You can use set\_facts and loop to get the values you want, but the question becomes what format and use of the data is really needed. I'm not sure creating a new structure is the Right Way (TM). It could be easier to just calculate the individual value for each item in the list when it is used.

```
$ cat test2.yml
---
- hosts: localhost
  vars:
      users:
        - name: alice
          sudo: yes
        - name: bob
        - name: charlie
  tasks:
     - user:
         name: "{{ item['name'] }}"
         create_home: true
         home: "{{ '/home/{}/'.format(item['name'] }}"
       when: "'sudo' not in item"
       loop: "{{ users }}"

```

---
Tags: ansible
---


# Question
Title: What are the daily job duties of a devops?
I have been working for linux management over 10 years. In the last 5 years, I have added a lot of coding experience through software development like php, ruby, go, etc. I wanna be a devops, but I have no idea how to get into it.

# Answer
> 1 votes
Keep learning other requirements for devops engineering. such as:

**Operating System concepts (Process management, threads and concurrency, POSIX basics, I/O management, file systems, memory storage, ...)**

as a devops engineer you need to know everything about the operating systems and how they work. to know everything about the operating systems, first you need to be a system administrator or sysadmin.

**managing servers**

server performance management helps in increasing agility and optimizes the performance of both physical and virtual servers.

**service management**

IT service management a cultural mindset to ensure that the desired outcome for the business is achieved. It incorporates principles and practices from various management approaches, such as lean manufacturing, organizational change management, system analysis and risk management.

**bash scripting (vim, emacs, ...)**

every sysadmin needs to have a good knowledge and experience in bash scripting.

**working with terminals**

every sysadmin needs to live entirely on the command line.

**web servers (nginx, apache,...)**

every sysadmin needs to know how servers work and how web servers send data.

**CI/CD tools (jenkins, circle ci,...)**

automation and automated testing improves agility and speed in scale.

**infrascructure as code (docker, ansible, kubernetes,...)**

IaC can be a key attribute of enabling best practices in devops.

**cloud services (aws, azure, google cloud platform, heroku)**

***Here's the complete 2020 devops engineer roadmap:*** https://roadmap.sh/devops

---
Tags: continuous-integration, linux, configuration-management, infrastructure-as-code, sysadmin
---


# Question
Title: Deploying a statefulSet with static volume provisioning but controller insists to use dynamic
I am trying to deploy a statefulSet that will use *static* volume provisioning (claim existing PVs) based on their label. However, it seems that the persistentvolume-controller is trying to provision storage *dynamically*, as when I `kubectl desribe pvc db-data-db-N` the Events log states that:

```
Events:
  Type       Reason              Age               From                         Message
  ----       ------              ----              ----                         -------
  Warning    ProvisioningFailed  9s (x3 over 21s)  persistentvolume-controller  Failed to provision volume with StorageClass "db-data-storage": claim.Spec.Selector is not supported for dynamic provisioning on AWS

```

I have pre-provisioned PVs with this format (I am using Helm):

```
kind: PersistentVolume
apiVersion: v1
metadata:
  name: db-data-{{ $volIndex }}
  labels:
    restoredFromBackup: "true"
spec:
  capacity:
    storage: 30Gi
  accessModes:
  - ReadWriteOnce
  awsElasticBlockStore:
    volumeID: {{ $volId }}
    fsType: ext4

```

Here is my PVC template:

```
  volumeClaimTemplates:
  - metadata:
      name: db-data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 30Gi
      selector:
        matchLabels:
          restoredFromBackup: "true"

```

# Answer
> 2 votes
Did you try by specifying a storageClassName since Kubernetes will be using default storage class when it's not specified? See Dynamic Provisioning of the blog post and Dynamic from Kubernetes documentation? Or try with by removing the default storageclass in the cluster to explicitly use given PV and PVC configuration.

---
Tags: kubernetes, kubernetes-volume, persistent-volumes
---


# Question
Title: Accessing remote servers through a VPN tunnel from within docker container
My laptop has an IPsec VPN tunnel set up to access a remote site (with my side being the VPN client).

As an experiment, I started up a simple docker container on my laptop using `docker run --rm -it alpine:3.7 /bin/sh` and from within the container tried to ping one of the remote machines on the other side of the tunnel.

I expected the ping to not work, but I was surprised that it actually did.

I expected that in order to be able to access one of the remote machines from within a docker container, I'd have to first set up the VPN connection from within the container.

If not a full-fledged VPN connection from within the container, I thought that I would *at least* need to set up some `iptables` rules on the host machine. (My laptop's private IP address is `192.168.10.1`, that of the remote server is `10.2.5.100` and that of my docker container is `172.17.0.2`.) But even this wasn't needed...

So my question is how is it that communicating with the remote machines through the tunnel *just works* without any additional setup?

p.s. in case it matters, my laptop is running Arch Linux and I'm using strongswan to set up the IPsec connection.

# Answer
> 3 votes
If you run `docker run --rm -it alpine:3.7 /bin/sh`, created docker container will use default `bridge` network.

You can see the properties of this network using the following command: `docker network inspect bridge`. You can see there to which host interface (usually `docker0`) the network is associated and also what is the subnet for this network (usually `172.17.0.0/16`).

Additionally, if you run `docker run --rm -it alpine:3.7 ip addr` command, you should see the IP address of your container, which should be part of previously mentioned subnet.

Then, Docker by default creates NAT iptables rules, which allows the containers to communicate with outside world (as the container IP address is not known to the internet). You can see the rules using `sudo iptables -t nat -nvL POSTROUTING` command. It should look similar to:

```
Chain POSTROUTING (policy ACCEPT 144K packets, 15M bytes)
 pkts bytes target     prot opt in     out     source               destination
  392 75916 MASQUERADE  all  --  *      !docker0  172.17.0.0/16        0.0.0.0/0

```

Finally, for your container, there is no difference, if you try to reach remote site of your VPN or the internet, as the container only knows how to reach Docker network. You can see that using `docker run --rm -it alpine:3.7 ip r` command:

```
default via 172.17.0.1 dev eth0
172.17.0.0/16 dev eth0 scope link  src 172.17.0.2

```

So when traffic from the container goes via default route, it hits host routing table (I'm not 100% sure if this is right), it is being routed to your tunnel and then iptables NAT rule applies, changing the source IP of your traffic to IP of your end of the tunnel, so returning traffic will come back over the tunnel as well and then back to your container.

---
Tags: docker, vpn
---


# Question
Title: I want to scale up/down as fast as AWS Lambda but also be able to allocate the vCPU (minimum 8) per task, any advice?
## Goal

So I'm looking for an AWS product or combination of products to accomplish the following:

* Ability to do CPU heavy (non-parallel) calculations (min 8 vCPU's per node)
* Minimum timeout limit of 1800 seconds (AWS Lambda has a limit of 900 seconds)
* Ability to automatically scale up and down to 0/1
* Ability to scale up really fast (\<30 seconds)
* Event-driven execution model (one task per node, node gets destroyed after the job is finished)
* Assign the desired vCPU's per "type" (3 predefined types) event/task/job

I'll try to be as detailed as possible in my description of what the job is, the two setups I've tried and what I don't like about them.

---

## Definitions

* ROE: Route Optimisation Engine
* VRP: Vehicle Routing Problem

A single JSON containing all the information regarding the stops, vehicles, time windows, start/end addresses) which needs to be optimised. VRP's can be classified into three different complexities \[easy, medium, hard\] based on:

+ number of stops
    + restrictions per stop (time window, capacity)
    + number of vehicles
    + restrictions per vehicle (time window, maximum range, capacity, breaks)

Easy classified VRP's will need less CPU resources and can be solved quicker than for example high classified VRP's.
* Solution: The best solution for the VRP (the most efficient routes for the VRP)

## Setup 1 - AWS Lambda

### Diagram

### AWS SQS

Standard type AWS SQS queue for optimisation request messages. Every message that enters triggers the ROE (AWS Lambda).

### AWS Lambda

The Lambda (3008 MB memory) is triggered by the AWS SQS queue for optimisation request messages and processes alles messages when they are added to the queue. The maximum timeout limit of all AWS Lambda's is 15 minutes.

### Problems

* AWS Lambda does not allow us to select more CPU resources and therefore optimisations of medium complexity VRP's take a long time.
* AWS Lambda maximum timeout settings of 15 minutes makes the usage of a Lambda for doing optimising high complexity VRP impossible

## Setup 2 - AWS Batch

### Diagram

### Steps

* WA sends optimisation request to API
* API creates a VRP and stores the VRP in S3
* API evaluates the complexity of the VRP (low, medium or high)
* API creates an AWS Batch Job (with the parameter problemId=123456789)
* API allocates the correct number of vCPU's and Memory (predefined) to that job based on the VRP complexity
* API adds the AWS Batch Job to the correct AWS Batch queue (the queue is defined by the VRP complexity)
* API returns a 200 - OK response to the WA
* AWS Batch Job is taken from the AWS Batch Queue for execution
* AWS Batch Job fetches the VRP from S3 based on the problemId it received during step 4
* AWS Batch Job solves the VRP
* AWS Batch Job stores the solution in S3
* WA requests the solution from the API
* API gets the solution from S3 and returns it to WA

### Problems

* Scaling up takes unacceptably long
* Going from 0 to 1 node takes too long (300+ seconds)
* Going from 1 to X nodes takes too long (60/300+ seconds)

# Answer
You can check out AWS Fargate, I've just used it for a similar use case.

Setup:

Jobs are posted to an SQS queue where a scheduler lambda picks them up and launches an AWS Fargate task.

Caveats:

* Lambda handles the SQS messages so you can't hand over the messages to Fargate (Lambda will auto-delete the messages from the queue once the lambda returns successfully), instead you'll need to handle retries via a separate lambda that gets notified whenever a task ends.
* Fargate only supports up to 4 vCPUs
* You need to pass any data to the container either via environment variables or as command line arguments

Launch times are not as fast as lambda, but much better than Batch and EC2 - I typically get my containers within 30 seconds of the lambda launching them.

Dead-letter/retry handling is a bit of a nuisance, but it's not that hard to implement as long as your process is well-behaved. You can easily get the process exit status from the API once the task has finished and you can store the number of retries as a tag on the task since you'll be launching a new task for each retry.

The scheduler and retry handler should be fairly trivial to write and if you need to pass larger amounts of data you can store it on S3 and send the location as an argument to your container.

A similar option - if you don't mind paying for the extra, unused capacity is to use ECS instead of Fargate and have unused capacity in the cluster so you can spawn new containers more or less instantly. This removes the limit on vCPU but you'll instead need to manage the cluster size (can be done via Autoscaling) and pay for the unused capacity.
> 1 votes

---
Tags: amazon-web-services, amazon-ecs, aws-lambda, java, serverless
---


# Question
Title: Should I give an assessment to DevOps Candidate
We're a software team currently trying to hire a DevOps Engineer.

We have an assessment for Software Developer candidates and we're currently evaluating the idea of having an assessment for our DevOps Engineer candidates. Since we have not hired DevOps people before and barely know a few, it would help us greatly.

What would be your advice ? If you support there should be an assessment, then what would be the content ?

### EDIT(Additional Info):

* We need to hire a DevOps Engineer to bootstrap our all DevOps process/culture.
* We have more than 100 servers. Around 20 different apps on them.
* We want him/her to;

+ Install a main devops tool like Jenkins or Teamcity
    + Build the pipelines for all this apps and servers
    + CIntegration, CDelivery, CDeployment depending on what app does
    + help migrate to Linux and/or docker

### About assessment

* It must be about building and deploying a few apps interacting with each other to different targets with different configurations.

### EDIT 2(answers):

Thanks for answers. While reading them, I'll answer the questions to make the backstory clearer.

@taleodor

* Developers including me are doing provisioning/build/deployments. We're very unhappy about that.
* Unfortunately, we can only hire one person for all our expectations.
    + We have plenty of time to put everything together, so it should not be problem for anyone who's willing to do all the stuff.
* Assignment will be take-at-home and candidates will be given a few days to complete. No problem about that. After all assessment is complementary to interview.
* I know a few DevOps people. I may get their help for evaluating assessments.

@030

I agree the devs automating their stuff is much better than another person doing that. But we have a big backlog about devops. We need someone to show us the way while doing the bulk of it himself/herself. We are not considering to totally forget the process. We want exact opposite: we want to do things right but we don't know where to start and don't have time.

I installed Jenkins. Set up pipelines for some of the projects. But I am a senior developer. Neither me nor my executive wants my all time spent on this.

### EDIT 3(What I came up with):

This is roughly the assignment I brought together. What do you think ? What else can be here ?

* Install Jenkins on a platform on your choice
* Dockerize .net project given.
* Build two pipelines with unit test automation.
* Build and host database.
* You must be delivering us your scripts and an account to jenkins.

# Answer
Yes, there absolutely should be an assessment, however it would be a good idea to agree upfront with one of the few DevOps people you know to validate the content and results.

Regarding the contents of the assessment, there are some questions still remaining after your edits. Key one is how your operations are done. Possible options are: you already have someone doing your server infrastructure and operations and looking for a purely CI/CD person, or you do not have good operations support (i.e. developers doing that and you're unhappy) and you need DevOps person to do server management too. Note, that if it's the later case, you'd probably need more than person over time but you need somebody senior to start with.

Note, that for larger organizations you'd find more specialization within DevOps (i.e. things like Release Management, DevSecOps and others), but for smaller companies those two - CI/CD vs Infra Operations are two main ones in my experience. You still need to keep those differences in mind if you're considering taking someone from a larger company.

Now, after answering those questions, you need to first establish the seniority level of a person you're looking to hire. If it's your first DevOps person - you would probably want somebody more senior, but it depends on other people in your existing team. Next you scope your assessment to the on-job tasks. My take - always give take-at-home assignments, I usually give it to candidates over weekend and I make sure they take no more than 3-4 hours.

For CI/CD case you may ask them for a simple CI pipeline in their tool of choice, built from scratch over some sample project. You should add extra questions about failure handling, parallelizing, caching, handling of automation tests.

For CI/CD + operations case you should add tasks about shell scripting, cloud security, terraform.

Also for the first DevOps / senior person it's always a good idea to add some questions about interacting with other teams. I.e. what developers would need to do and how the interaction process with them should be built.

Again, start with one of the few DevOps people you know and get them onboard as a consultant to assess and hire that first person, otherwise you're taking a lot of chances there - and DevOps is a pretty sensitive part of your business.
> 3 votes

# Answer
Yes. Although DevOps is a culture and not a role, one should assess the mindset and technical skills of a candidate.

At my previous work we conducted two interviews. The first one was a conversation about the mindset. If my colleague and I were positive then we offered the candidate the possibility to prepare a demo for the second interview. He or she could choose to demonstrate an autodeploy or the candidate could come up with another proposal.

During the second and technical interview, the candidate had to bring his or her laptop and demonstrate what he or she prepared. During the demo, my colleague and I were asking questions. By doing this we got an impression about the technical knowledge as well.

> We need to hire a DevOps Engineer to bootstrap our all DevOps process/culture

In my opinion, your current Developers should deploy the code they created, themselves. "You build it, you run it.". Although it can be hard to get started with CI, the documentation of various platforms like, Gitlab, Bitbucket, Jenkins is comprehensive and they can implement it. If you hire some person for this, you will create silos and it will be a negative effect on a DevOps culture. It will promote a "throw it over the wall mindset" and "What I am?", "You are IT of Dev" mindset.

The more I think about the question, the more I think you just should hire a freelancer for a couple of sprints that motivates and instructs developers to implement the pipelines themselves. Once a CI is running well, it can be applied to all projects and the person or freelancer you are trying to hire will be superfluous.
> 3 votes

---
Tags: project-management, software-engineering, human-resources
---


title: Production grade methodology for alerts

text:
# Question
Title: Production grade methodology for alerts
# Background

Our **code** is written with:

1. Unit tests
2. End to end tests
3. Code review
4. Staging process
5. Deployment process

On the contrary, our **alerts** are just written and then modified occasionally manually. No quality process at all.

This process is reasonable for simple threshold checks. However, our alerts are sometimes built on complicated queries. Sometimes composed of ~20 lines of a query.

If we accidentally break an alert, it could expose us to production instability since we won't know if some logic or component breaks.

# The question

Is there a recommended methodology for validating the quality of complicated alerts?

## P.S.

We're using Splunk alerts

# Answer
> 0 votes
**DTAP**

As monitoring is part of production it should be part of DTAP as well. This implies monitoring in Development, Testing, Acceptance and Production as well. If somebody modifies a check in production and it will silence some check and if the team does not get reported that there are issues then the customer will be impacted and that could be a huge problem. In short, if monitoring is part of production it should be applied to all stages of DTAP as well.

**Testing**

If custom scripts are used in splunk and these are written in Python or some other language, then you should also apply unit tests and integration tests. In most monitoring systems, a 0, 1, 2 and 3 exit code are checked so this should also be covered in the tests. If a monitoring is written in bash, one could use BATS and Powershell can be tested as well with Pester.

Why testing these scripts? Again, same reason as depicted in the DTAP section. Imagine that somebody breaks some monitoring script by introducing a typo and you do not get notified, then this could have a huge impact on the customer and also on the team. Imagine that you have to debug for a couple of days because some script is not working well, while that could have been prevented. So I advocate to apply unit tests, integration tests and even CI for this "simple" monitoring scripts.

---
Tags: monitoring
---


# Question
Title: AWS MSK: deploying all Kafka brokers in the same availability zone
When using AWS MSK, is it possible to deploy Kafka brokers in the same AWS availability zone?

From what I read, AWS enforces deployments across availability zones. If you use three brokers, it will require three subnets in different availability zones. Is that true or is there a way to bypass that restriction?

---

Background: for most situations, deploying across zones is a good idea, but we are operating a subsystem where availability is not crucial and where all services are running in the same availability zone. Traffic in the same AZ is free, so we would prefer that if possible. If Kafka is unavailable if the whole data center behind that AZ goes down, would be acceptable. Most likely, the whole subsystem would not work in that scenario anyway.

# Answer
I cannot back it up with links to the documentation, but my strong assumption is that it is not possible. AWS MSK forces cross availability-zone deployments. (In most cases, it is a good idea, anyway. I understand that the scenario in my question is not so common.)

The only option to stay in the same availability zone is to use only one broker. However, it is not recommended to run Kafka with a single broker. Therefore it is not an attractive option.
> 0 votes

---
Tags: amazon-web-services, kafka
---


# Question
Title: Jenkins pipeline: do "Declarative: Checkout SCM" step only
I'm newbie in Jenkins. I've created pipeine with parameters in Jenkinsfile in git. It works fine.

Problem is: when I push **new** parameters into `Jenkinsfile`, then press button "Build with Parameters", there are no new parameters. I will see them only on next build, after step "Declarative: Checkout SCM" will be done.

I know workaround: add parameter to skip all steps. Then I can run build and skip all steps (except "Declarative: Checkout SCM" of course) and afterwards run build with new parameters.

Question: is there are some magic URL or other possibility to update parameters from git before build?

# Answer
As implied by the links in comments, for better or for worse, this behavior is an unavoidable consequence of how Pipeline is designed. Pipeline configuration changes in your Jenkinsfile cannot be picked up by Jenkins without executing your Pipeline. This includes things such as adding parameters.

My personal advice is to ensure that your Pipeline jobs are non-destructive, or in other words, to ensure that the worst thing that could possibly happen by running a job is nothing. This allows you to pick up configuration changes by just running your job after committing the changes to your Jenkinsfile, and then you can simply abort the job (or let it run to completion if it's a short job) with the confidence that you won't break anything. Then subsequent runs will have your configuration changes available.
> 2 votes

# Answer
My workaround looks like this:

```
pipeline {
    parameters {
        booleanParam(name: 'skip_all_steps', defaultValue: false, description: 'When changed parameters in pipeline, just update code from git')
    }
    stages {
        stage('Stage1'){
            when { not { expression { return params.skip_all_steps } } }
            steps {
                …
            }
        }
        stage('Install requirements') {
            when {
                allOf {
                    expression { return params.install_django_requirements }
                    not { expression { return params.skip_all_steps } }
                }
            }
            steps {
                sh 'pip3 install --upgrade --user -r src-django/requirements.txt'
                }
        }
        …
    }
    post {
        success {
            script {
                if (!params.skip_all_steps.toBoolean() && params.workspace_cleanup.toBoolean()) {
                    cleanWs()
                }
            }

        }
    }
}

```

Be careful:

* parameters in Jenkins UI will be updated only after **successful** build,
* maybe it is better idea to use `defaultValue: true` for `skip_all_steps` in your particular case (for example, to avoid running any step on first build)
> 0 votes

# Answer
A few years ago I used a similar approach:

```
 parameters {
        booleanParam(
                name: 'ABORT_JOB', defaultValue: false,
                description:    'Set this to true if you need to "dry-run" this job without doing any real work.\n' +
                        'The Jenkinsfile will be read and new parameters added. It will exit in the first stage.')

```

And in the first stage that runs:

```
stage('Initialise') {
            steps {
                script {
                    if (params.ABORT_JOB) {
                        currentBuild.result = 'ABORTED'
                        error('Stopping early...')
                    }

```

I've since started using JobDSL to generate my jobs. This simplifies the pipeline code and removes the need to code as above.
> 0 votes

---
Tags: jenkins-pipeline
---


# Question
Title: Terraform syntax error after V0.12 upgrade
I have just upgraded from terraform v0.11 to v0.12 But i am facing lot of error

complete code @ https://github.com/sivatest82/Terraform1/tree/master/secretsv12

especially in the block

```
 secrets = [

    {
      secret_id = "docker_secret.mysql_db_password.id"
      secret_name = "docker_secret.mysql_db_password.name"
      file_name = "/run/secrets/docker_secret.mysql_db_password.name"
    },

    {secret_id = "docker_secret.mysql_root_password.id"
      secret_name = "docker_secret.mysql_root_password.name"
      file_name = "/run/secrets/docker_secret.mysql_root_password.name"
    }
  ]

```

v0.11 code which had no issue

```
   resource "docker_service" "mysql-service" {
  name = "mysql_db"

  task_spec {
    container_spec {
      image = "${docker_image.mysql_image.name}"

      secrets = [
        {
          secret_id   = "${docker_secret.mysql_root_password.id}"
          secret_name = "${docker_secret.mysql_root_password.name}"
          file_name   = "/run/secrets/${docker_secret.mysql_root_password.name}"
        },
        {
          secret_id   = "${docker_secret.mysql_db_password.id}"
          secret_name = "${docker_secret.mysql_db_password.name}"
          file_name   = "/run/secrets/${docker_secret.mysql_db_password.name}"
        }
      ]

      env {
        MYSQL_ROOT_PASSWORD_FILE = "/run/secrets/${docker_secret.mysql_root_password.name}"
        MYSQL_DATABASE           = "mydb"
        MYSQL_PASSWORD_FILE      = "/run/secrets/${docker_secret.mysql_db_password.name}"
      }

      mounts = [
        {
          target = "/var/lib/mysql"
          source = "${docker_volume.mysql_data_volume.name}"
          type   = "volume"
        }
      ]
    }
    networks = [
      "${docker_network.private_overlay_network.name}"
    ]
  }
}

```

v0.12 code

```
resource "docker_service" "mysql-service" {
  name = "mysql_db"

  task_spec {
    container_spec {
      image = "docker_image.mysql_image.name"

      secrets = [

        {
          secret_id = "docker_secret.mysql_db_password.id"
          secret_name = "docker_secret.mysql_db_password.name"
          file_name = "/run/secrets/docker_secret.mysql_db_password.name"
        },

        {secret_id = "docker_secret.mysql_root_password.id"
          secret_name = "docker_secret.mysql_root_password.name"
          file_name = "/run/secrets/docker_secret.mysql_root_password.name"
        }
      ]

      env = {
        MYSQL_ROOT_PASSWORD_FILE = "/run/secrets/docker_secret.mysql_root_password.name"
        MYSQL_DATABASE           = "mydb"
        MYSQL_PASSWORD_FILE      = "/run/secrets/docker_secret.mysql_db_password.name"
      }

      mounts = {
          target = "/var/lib/mysql"
          source = "docker_volume.mysql_data_volume.name"
          type   = "volume"
        }

    }

    networks = "docker_network.private_overlay_network.name"

  }
}

```

Error

```
[cloud_user@siwgftd9213c secrets]$ terraform validate

Error: Unsupported argument

  on main.tf line 8, in resource "docker_service" "mysql-service":
   8:       secrets = {

An argument named "secrets" is not expected here. Did you mean to define a
block of type "secrets"?

Error: Unsupported argument

  on main.tf line 26, in resource "docker_service" "mysql-service":
  26:       mounts = {

An argument named "mounts" is not expected here. Did you mean to define a
block of type "mounts"?

Error: Incorrect attribute value type

  on main.tf line 34, in resource "docker_service" "mysql-service":
  34:     networks = "docker_network.private_overlay_network.name"

Inappropriate value for attribute "networks": set of string required.

```

I have already have a seperate file for them

Secrets.tf

```
resource "docker_secret" "mysql_root_password" {
  name = "root_password"
  data = "var.mysql_root_password"
}

resource "docker_secret" "mysql_db_password" {
  name = "db_password"
  data = "var.mysql_db_password"
}

```

And variables tf file

```
variable "mysql_root_password" {
  default     = "cDRzU1dvUmQwIQo="
}

variable "mysql_db_password" {
  default     = "cDRzU1dvUmQwIQo="
}

```

---

**After tfvars.tf creation**

* Able to compile code
* But unable to apply the plan

> Error: Error response from daemon: rpc error: code = InvalidArgument desc = malformed secret reference
>
> on main.tf line 1, in resource "docker\_service" "mysql-service":
> 1: resource "docker\_service" "mysql-service" {

**main.tf**

```
    resource "docker_service" "mysql-service" {
  name = "mysql_db"

  task_spec {
    container_spec {
      image = docker_image.mysql_image.name

      secrets {
        file_name = "${local.secrets[0].file_name}"
        secret_id = "${local.secrets[0].secret_id}"
      }

      secrets {
        file_name = "${local.secrets[1].file_name}"
        secret_id = "${local.secrets[1].secret_id}"
      }

      env = {
        MYSQL_ROOT_PASSWORD_FILE = "/run/secrets/${docker_secret.mysql_root_password.name}"
        MYSQL_DATABASE           = "mydb"
        MYSQL_PASSWORD_FILE      = "/run/secrets/${docker_secret.mysql_db_password.name}"
      }

      mounts {
          target = "/var/lib/mysql"
          source = "docker_volume.mysql_data_volume.name"
          type   = "volume"
        }
    }

    networks = [docker_network.private_overlay_network.name]
  }
}

```

**tfvars.tf**

```
    locals {
 secrets = [
        {   secret_id   = docker_secret.mysql_root_password.id
            secret_name = docker_secret.mysql_root_password.name
            file_name   = "/run/secrets/${docker_secret.mysql_root_password.name}"
        },
        {
            secret_id   =    docker_secret.mysql_db_password.id
            secret_name =    docker_secret.mysql_db_password.name
            file_name   = "/run/secrets/${docker_secret.mysql_db_password.name}"
       }
    ]
}

```

# Answer
> 2 votes
It looks like there are two different problems here, so I'll answer them separately.

---

First, we'll talk about the errors of this type:

```
Error: Unsupported argument

  on main.tf line 8, in resource "docker_service" "mysql-service":
   8:       secrets = {

An argument named "secrets" is not expected here. Did you mean to define a
block of type "secrets"?

```

This error is talking about the distinction described in the Terraform documentation section Arguments and Blocks, and is telling you that `secrets` is defined in the provider's schema as using nested block syntax rather than argument syntax.

The syntax for nested blocks and the syntax for literal object values are both similar in that they use `{` `}` as delimiters, but the meaning of these and what is valid inside the braces in each case is distinct.

To fix this, you can use the block syntax to define these nested objects:

```
  secrets {
    secret_id   = docker_secret.mysql_root_password.id
    secret_name = docker_secret.mysql_root_password.name
    file_name   = "/run/secrets/${docker_secret.mysql_root_password.name}"
  }
  secrets {
    secret_id   = docker_secret.mysql_db_password.id
    secret_name = docker_secret.mysql_db_password.name
    file_name   = "/run/secrets/${docker_secret.mysql_db_password.name}"
  }

```

The same is true for the `mounts` block type:

```
  mounts {
    target = "/var/lib/mysql"
    source = docker_volume.mysql_data_volume.name
    type   = "volume"
  }

```

---

The other error message you saw is an unrelated problem:

```
Error: Incorrect attribute value type

  on main.tf line 34, in resource "docker_service" "mysql-service":
  34:     networks = "docker_network.private_overlay_network.name"

Inappropriate value for attribute "networks": set of string required.

```

This error is saying that the `networks` argument requires a set of strings, but the value you gave here is a single string. This is happening because you've put the reference expression in quotes, and so Terraform assumes you are intending to give a literal string rather than a reference here. The first step to address this is to remove the quotes so to make this a reference expression rather than a literal expression:

```
  networks = docker_network.private_overlay_network.name

```

That's not sufficient for this particular situation though, because this `name` attribute is just a single string rather than a set of strings, and so the above change alone will not fully fix the problem. To address this secondary problem we can use brackets `[` `]` to construct a single-element sequence:

```
  networks = [docker_network.private_overlay_network.name]

```

This argument expects a set of strings rather than a sequence of strings, but that's okay here because Terraform will automatically convert the sequence into a set. (A set is an unordered collection of distinct values, whereas a sequence is an ordered collection of values that may contain duplicates.)

# Answer
> 1 votes
These were made as errors in 0.12. You need to put these into tfvars file in a locals definition.

```
locals {
   secret_id = "docker_secret.mysql_db_password.id"
   secret_name = "docker_secret.mysql_db_password.name"
   file_name = "/run/secrets/docker_secret.mysql_db_password.name"

```

}

Update: Using a list:

```
locals {
    secrets = [
        {
            secret_id = "docker_secret.mysql_db_password.id"
            secret_name = "docker_secret.mysql_db_password.name"
            file_name = "/run/secrets/docker_secret.mysql_db_password.name"
        },
        {
            secret_id = "docker_secret.mysql_root_password.id"
            secret_name = "docker_secret.mysql_root_password.name"
            file_name = "/run/secrets/docker_secret.mysql_root_password.name"
        }
    ]
}

```

Then to access an individual value, use indexing with variable evaluation: `${local.secrets[1].file_name}`.

# Answer
> 0 votes
According to the documentation, you should change your secrets handling to following syntax:

```
      secrets = [

        {
          secret_id   = docker_secret.mysql_db_password.id
          secret_name = docker_secret.mysql_db_password.name
          file_name   = "/run/secrets/${docker_secret.mysql_db_password.name}"
        },

        {
          secret_id   = docker_secret.mysql_root_password.id
          secret_name = docker_secret.mysql_root_password.name
          file_name   = "/run/secrets/${docker_secret.mysql_root_password.name}"
        }
      ]

```

So you pass the actual secret ID to the service object and not the Terraform resource reference as a string. The only difference between Terraform 0.11 and 0.12 code is, that in 0.12 you can remove the interpolation syntax (so `${}`), if you don't actually use it, as you only pass the attribute of other resource, with no additional formatting.

---
Tags: docker, terraform
---


# Question
Title: In what way does a new ECS Fargate service not satisfy the task definition's required capabilities?
When creating a new AWS ECS service using a Fargate task, in the AWS Console wizard, it fails with the following error:

```
Failed creation of Service
Service creation failed: The specified platform does not satisfy the task definition’s required capabilities.

```

The service launch type is set to `FARGATE` and the Platform version is set to `LATEST`. The task is attempting to mount an AWS EFS filesystem for persistent storage (introduced in version 1.4.0 of the platform).

# Answer
This is by design because AWS have yet to upgrade LATEST to point to v1.4.0 (as per 2020-06-23, see AWS ECS Developer Guide). Set the service platform version explicitly to `1.4.0` (instead of LATEST).
> 4 votes

# Answer
It doesn't tell the exact error message, however in my case,

I was using firelens log driver with S3 configuration type. Since Fargate only supports local file at this point, I got the same error message. Once I changed it to local file in the task definition json, it proceeded further.
> 1 votes

---
Tags: amazon-ecs, fargate, amazon-efs
---


title: How to version control shared database?

text:
# Question
Title: How to version control shared database?
We have several C# applications using a shared monolithic database (on-premise SQL server) using Stored Procedures as an exclusive interface. The C# code repositories include only Stored Procedures that are relevant to them, which means a single database is split into multiple repositories.

We are currently using a shared DEV environment restored from production (after some modification to sensitive data, etc.)

I want to move to our version control being a single source of truth and put the whole database into one repository so I can generate empty DB for each developer (that would allow me to unit test, etc.)

But that would create problems for developers: they would have to keep a branch of their app and branch of a database in sync and coordinate extensively.

How to solve this without moving the apps back into monolith as well?

# Answer
I personally made the unwise choice of cramming five different apps into a single database about 20 years ago. I then had to deal with the repercussions of that. The bad news is, this is really hard. The good news is, modern tools make it easiser, but not easy.

So, first, absolutely get a single source of truth. You're using SQL Server, so there are a bunch of really good tools that will help you maintain your database in source control. One is provided by Microsoft. It's called SQL Server Data Tools. It only works inside Visual Studio. However, it's a solid tool. If you have an MSDN license to run VS, it's complete free. There are also 3rd party tools available for a cost. They're generally much more sophisticated (Microsoft doesn't give SSDT much love). I work for one of the vendors of such tools, Redgate Software.

Once you're on the path of maintaining the code in source control, the easy part is over. Now you have to figure out how to integrate the teams. Here's what we did. Every team got it's own branch of code (we did this in the old TFS, which sucked, you can use some variant of Git which will be much easier, not easy, just easier). This also means that each team had to have it's own copy of the database (which it sounds like what you're going for anyway). In fact, each developer got their own database and then we had an integration testing database for each team. So the team worked on it's own, independent of the other teams.

Next, we created a master integration database. This is where the teams came together. We had a main branch for the code and teams would have to merge their code semi-regularly into this branch and then we could run automated tests to see if any team had broken the code or structures of other teams.

Then, you have to build a release process that ensures that any one team can't overwrite or supersede the others. This means that teams have to regularly merge from the main branch, changes from the other teams.

It requires quite a bit of discipline and some training to get this all set up. However, it's doable. I'd also suggest implementing a flow control software like Jenkins, Octopus, AWS DevOps or Azure DevOps to assist with automating all of it.

I've worked on tons and tons of documentation around these types of processes. Full disclosure, a lot of the info at that link includes our tooling. However, the processes are what you're looking for and you can find good information there. This is a free ebook on these processes I helped to write that's very product agnostic. Hopefully between these resources you get enough to help out.
> 8 votes

# Answer
Disclaimer: I'm not a RDBMS DBA and I have zero experience with MSSQL. Everything below is a hypothesis.

Each app should have it's own stored procedures in migrations, and shared procedures should be in their own repo with a separate migration list. This adds a necessary isolation and the ability to use benefits of common ones with optional toggling on per-app basis if there are breaking changes, plus a separate repository is an easy way to maintain them.
> 2 votes

---
Tags: databases, versioning, monorepo
---


# Question
Title: Using DevOps pipelines to create distribution bundles of applications
We want to create bundles of our applications in order to distribute to clients etc. An application can have more than one **component**. Ex: an application can have a spring server and a scheduler so in order to distribute this application both jars will have to be given to the client.

We want to use Docker/Docker compose to accomplish this. Ex: we give a client a bundle containing a Docker compose file, and he/she just runs `docker-compose up` to start the application.

We currently have a Jenkins pipeline that when run will produce a zip file containing `.tar` files of all the Docker images of the components. There will also be a `start.sh` (or `start.bat`) file that, when run, will load all the Docker images and run `docker-compose up`.

Some of these components have properties that the client needs to configure according to their environment (ex: SCM URL). If the component is Spring/Java based, we would externally mount the application.properties file into the docker image in the `docker-compose.yml`

This all works, I just want to know if the current process we are using is compliant with industry standards or is there a better way of meeting the requirement? Any open source tools that would simplify the process?

# Answer
Add a config file with parameters used in install/run script. You can look at Harbor (Github, install docs) as an example of doing just that (and they have a configuration migrator).
> 2 votes

---
Tags: docker, jenkins, jenkins-pipeline, docker-compose, continuous-delivery
---


# Question
Title: How can I load multiple paths into the workspace during docker run?
My datasets and codes are in different folders and they don't have immediate parent folder. I want my codes to access my datasets while using docker run. Is there a way to load both codes and datasets folder into workspace?

# Answer
You can use Docker volume mounts at run time. For example:

```
$ mkdir -p app/src logs
$ touch app/src/foo.c logs/bar.txt
$ docker run --rm -v $PWD/app:/app -v $PWD/logs:/app/logs alpine ls -l /app /app/src /app/logs
/app:
total 8
drwxr-xr-x    2 1000     1000          4096 May  6 23:33 logs
drwxr-xr-x    2 1000     1000          4096 May  6 23:33 src

/app/logs:
total 0
-rw-r--r--    1 1000     1000             0 May  6 23:33 bar.txt

/app/src:
total 0
-rw-r--r--    1 1000     1000             0 May  6 23:33 foo.c

```

This mounts two directories on the host into the docker container. Notice in this example that inside the container, they are mount on inside the other.
> 3 votes

---
Tags: docker
---


# Question
Title: Grant access permission to file when mounting from host
Below is except of my docker-compose.yml file

I am trying to mount from host file inside the container, which works i.e. able to mount but then I need to give it permission (my clumsy chown command, below).

I am not sure where to run it as wherever I place my (command: chown,-R,elasticsearch,/usr/share/elasticsearch/config/)

It gives me error that 'no such file or directory' which I think: my code run before volume is attached.

I tried placing it after container\_name as well, same 'no such file or directory'

Any advise how to do it?

```
image: docker.elastic.co/elasticsearch/elasticsearch:7.6.2
container_name: elasticsearch

volumes:
  - ./elastic-stack-ca.p12:/usr/share/elasticsearch/config/elastic-stack-ca.p12
  - esdata1:/usr/share/elasticsearch/data
*command: chown,-R,elasticsearch,/usr/share/elasticsearch/config/*

```

Here is full yml, if it is helpful

```
    version: '3'

services:

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.6.2
    container_name: elasticsearch
    command:
     chown,-R,elasticsearch,/usr/share/elasticsearch/config/
    environment:
      - node.name=elasticsearch
      - discovery.seed_hosts=elasticsearch
      - cluster.initial_master_nodes=elasticsearch
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"

    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - ./elastic-stack-ca.p12:/usr/share/elasticsearch/config/elastic-stack-ca.p12
      - esdata1:/usr/share/elasticsearch/data
    ports:
      - 9200:9200

  kibana:
    image: docker.elastic.co/kibana/kibana:7.6.2
    container_name: kibana
    environment:
      ELASTICSEARCH_URL: "http://elasticsearch:9200"
      ELASTICSEARCH_USERNAME: "kibana"
      ELASTICSEARCH_PASSWORD: "kibana"
    ports:
      - 5601:5601
    depends_on:
      - elasticsearch

volumes:
  esdata1:
    driver: local

```

# Answer
I don't think volumes can be specified with relative paths. Put an absolute path instead of `./elastic-stack-ca.p12` and you should be fine.
> 1 votes

---
Tags: docker, docker-compose, dockerfile
---


# Question
Title: docker-compose multiple services in a loop
Trying to optimize definition of multiple services with `docker-compose.yml`

Here is the original, "dumb" variant of the `docker-compose.yml`:

```
version: '3'
services:
  s1:
    build: .
    ports:
    - "5001:9000"
    volumes:
    - ../www1:/var/www
  s2:
    build: .
    ports:
    - "5002:9000"
    volumes:
    - ../www2:/var/www
  s3:
    build: .
    ports:
    - "5003:9000"
    volumes:
    - ../www3:/var/www

```

Is there some sort of an iterator to compact this to just one block like the following?

```
version: '3'
services:
  for i in 1,2,3:
    s$i:
      build: .
      ports:
      - "500$i:9000"
      volumes:
      - ../www$i:/var/www

```

# Answer
> 5 votes
Not in pure Docker Compose. If you have a plain-text templating engine you like (Mustache, Jinja, m4, ...) you can ask it to do this for you.

You tagged this as , and Compose can take the YAML configuration on stdin, so in principle one option is to write a shell script, and then pipe that into `docker-compose`:

```
#!/bin/sh

buildComposeYaml() {
  cat <<HEADER
version: '3'
services:
HEADER
  for i in $(seq 3); do
    cat <<BLOCK
  s$i:
    build: ../www$i
    ports:
      - "500$i:9000"
BLOCK
  done
}

buildComposeYaml | docker-compose -f- "$@"

```

(Note that YAML is indentation-sensitive, and the script is mixing indentation for the script itself, the YAML embedded in the script, and the end-of-heredoc markers; so this will work for a simple tool but it wouldn't be my long-term choice.)

# Answer
> 3 votes
Using `docker-compose` version 3.3 extensions and with the long variants for `ports` & `volumes`:

```
version: '3.3'
services:
  s1: &s
    build: .
    ports:
    - published: 5001
      target: 9000
    volumes:
    - source: ../www1
      target: /var/www
  s2:
    <<: *s
    ports:
    - published: 5002
    volumes:
    - source: ../www2
  s3:
    <<: *s
    ports:
    - published: 5003
    volumes:
    - source: ../www3

```

Not really more compact, but native :-)

Version 3.4 introduced further improvements into making docker-compose.yml more DRY.

Perhaps someone more experienced will point to even greater developments yet!

---
Tags: docker, docker-compose, yaml, shell-script, templating
---


# Question
Title: Why do I need tools like minikube or microk8s to run k8s locally?
I'm just getting started on k8s and got stuck on running k8s locally (it's running, but I don't understand exactly)

Pretty much all the tutorials on running k8s locally use minikube and I'm a little bit puzzled.

Isn't docker + k8s combination enough to run k8s locally?

Why do I need an extra layer with a virtual machine in order run k8s locally?

I've read about minikube and followed tutorials on the offical k8s website, but I'm still puzzled.

Can anyone shed light on me?

# Answer
Kubernetes has several system-level parts. It needs a functioning etcd cluster, the API server proper, and a per-node service called the kubelet. It also generally expects to be able to use 100% of the resources available on any given node for its own use. Just having the `kubectl` binary on its own isn't enough to use Kubernetes; you also need the rest of the cluster that goes with it.

There's no specific reason you can't directly install all of the Kubernetes cluster parts on the host (using `kubeadm`, or Minikube's "none" VM driver), but it will be trickier to maintain. It may or may not conflict with local Docker use, and it will be tricky to uninstall or upgrade it. If you run Minikube in a VM (or Kind in a Docker container) then all of the parts are preinstalled for you, and you can easily stop or delete the VM (or container) when you don't need it any more.
> 6 votes

---
Tags: kubernetes, minikube
---


# Question
Title: Jenkinsfile script to waitUntil a folder is created
With this Jenkinsfile code, I am trying to test A/B folder creation, and also testing if waitUntil works till the folder is created, I expect it to wait for 0 secs as the folder is created before `waitUntil` gets called.

```
stage('other job') {
                    steps {
                        script {
                            timeout(10) {
                                def folder = new File( 'A/B' )
                                println "Waiting for " + folder
                                println "fe==" + folder.exists()
                                waitUntil {
                                    if(folder.exists()) {
                                        return(true)
                                    }
                                }
                            }
                        }
                    }
                }

```

It fails with below error, what's wrong here? the folder is not gerrit created in workspace.

```
Timeout set to expire in 10 min
[Pipeline] {
[Pipeline] // stage
[Pipeline] }
[Pipeline] echo
Waiting for A/B
[Pipeline] echo
fe==false
[Pipeline] waitUntil
[Pipeline] {
[Pipeline] }
[Pipeline] // waitUntil
[Pipeline] }
[Pipeline] // timeout
...
Failed in branch other job
...
[Pipeline] End of Pipeline
java.lang.ClassCastException: body return value null is not boolean
    at org.jenkinsci.plugins.workflow.steps.WaitForConditionStep$Callback.onSuccess(WaitForConditionStep.java:167)
    at org.jenkinsci.plugins.workflow.cps.CpsBodyExecution$SuccessAdapter.receive(CpsBodyExecution.java:377)

```

# Answer
Your block expression doesn't return a value in all cases.

Try this instead:

```
waitUntil { folder.exists() }

```
> 1 votes

---
Tags: jenkins-pipeline, jenkinsfile, groovy
---


# Question
Title: How to set Deployment Type using ECS First Run Wizard
The AWS ECS First Run Wizard provides no option for choosing the 'Deployment type' of the ECS Service:

Whereas the ECS Service creation screen does:

Why does the First Run Wizard not allow for setting the same properties as the regular ECS creation screen does? There's also no way to edit an ECS Service created via the First Run Wizard to change the Deployment Type.

**How do I set the Deployment type of an ECS Service created via the First Run Wizard?**

# Answer
> 1 votes
I think you want to setup Blue/Green deployment type for ECS. As Blue/Green deployment, there should be one environment (**Blue**) should exist before deploying another environment called **Green**. As it's your first project or first ECS services, there is no prior environment thus we cannot choose Blue/Green deployment in deployment type.

---
Tags: amazon-web-services, amazon-ecs
---


title: How to keep track of broken releases?

text:
# Question
Title: How to keep track of broken releases?
I spent considerable time googling around, but found nothing that would answer my question, so here it is.

If you do a canary release, and you learn the the release is somehow broken: how to you denote that a certain tag is broken and should not be deployed anywhere else, but removed from your productions ASAP?

I had several ideas:

1. Using GitLab, I could add something like `**broken**` to tag release note. Deploy jobs would call GitLab API to ensure that the tag I want to deploy is not annotated as broken.
2. I could have a text file in repo, where broken tags would be listed. In this case deploy job checks if the tag you want to deploy is not listed in the file.
3. Do no save information about release problems: just deploy a new stable, patched version to production and never look back.

How do/would you handle this? Any tips or hyperlinks are welcome.

# Answer
> 2 votes
I am not sure about other approaches, but the one that I use is a modification of #3.

There's usually a classification of releases (for example think of Ubuntu LTS and regular release).

1. For the regular release, I would just release a new version and add a new tag.
2. For LTS/Stable releases I would delete a tag pointing to a faulty release or change a commit this tag points to

# Answer
> 2 votes
I'm building a product to solve this problem among others - https://relizahub.com (note it's pretty early stage at the moment).

Idea goes as following - for all your releases you stream their details to Reliza Hub. You would then have a client that would pick up releases you need based on combination of tags, approvals, etc. So it is somewhat similar to your #1 approach. For broken releases you either do not approve them or mark them as 'REJECTED' and this would stop them from propagating in your pipeline.

# Answer
> 1 votes
I think that it might help to distinguish between a "release" and a "release candidate". If you really have shipped a version of software to other people and it is 'totally broken' then something went really very badly wrong. To avoid this opensource projects have milestone release and snapshot release (aka "nighly releases" or "freshly releases"). These try to flush out critical failures through pre-release public testing. Once the risk of shipping something totally broken has passed then the software is tagged as the final release. After that, if there are bugs, and there always are bugs, then it is a matter of issuing advisories and warning folks about bad old versions within release notes.

For example, spring.io releases are versioned something like "5.2.5.RELEASE". We might expect that they have a "5.2" branch and that they had many release candidate builds on that branch. In fact they have actually have two special repos for milestone releases and snapshot releases as described here. These approximate to beta releases and alpha releases.

If you are not releasing software to other people to run, but are running the software as a service yourself, then your option 3 may work well:

> (3) Do no save information about release problems: just deploy a new stable, patched version to production and never look back.

This approach can work well if you are a building a web-app or mobile backend based product. The internal software at large firms like global banks will typically use this approach. In fact, I have worked on many systems at both banks and startups over two decades and we only ever used this approach in practice.

I should mention that your Git server might have a "latest release feature". Both GitHub and GitHub Enterprise have this. Even if you release via a build pipeline to some artefact repository it can still be helpful to make a "release" within the Git server so that an API call to something like `GET /repos/:owner/:repo/releases/latest` API will return details of the latest version including the git tag that should match the release version. Your deployment piplien can then pull the corresponding prebuilt code from the artefact server to deploy.

---
Tags: release-management
---


# Question
Title: How to execute an ansible playbook using a jenkins pipeline script?
We developed declarative pipeline code to execute an Ansible playbook. We tested the playbook from CLI and it is working properly. My playbook actually performs some shell commands on remote server.

SSH configuration between two servers has been done: I have generated an ssh key and copied it to the remote server, and I am able to ping the server.

The Error msg:

```
ansible-playbook /var/lib/jenkins/workspace/test1/playbook2.yml -i
/var/lib/jenkins/workspace/test1/inventory.ini -b --become-user root
[WARNING] Ansible is in a world writable directory
(/var/lib/jenkins/workspace/test1), ignoring it as an ansible.cfg source

fatal: [17.14.69.21]: UNREACHABLE! => {"changed": false, "msg": "Failed to
connect to the host via ssh: Permission denied (publickey,gssapi-
keyex,gssapi-with-mic,password).\r\n", "unreachable": true}
[17.14.69.21]:
FATAL: command execution failed
hudson.AbortException: Ansible playbook execution failed.

```

I have given sudo access to Jenkins user and Jenkins is running as Jenkins user.

# Answer
If you have Jenkins user public key, on your target hosts authorized keys file, for the user you want to login as, ssh connection authorization should work.

This is an example of Jenkinsfile step.

```
ansiblePlaybook colorized: true, installation: 'Ansible', inventory: 'inventory', playbook: 'playbook.yml',tags: "" , skippedTags: "", extras: "--extra-vars ' hst=targetHost "

```

This is Ansible Inventory example. You can define the args at the host level as well.

```
[targetHost]
17.14.69.21 ansible_ssh_user=sshuser

[all:vars]
ansible_connection=ssh
ansible_ssh_common_args='-o StrictHostKeyChecking=no -oHostKeyAlgorithms=+ssh-dss'

```
> 1 votes

# Answer
> --become-user root

I think you are doing ssh connection between two servers as a root user. I would like to know why. I would never do this.

Assuming those servers are Unix/Linux . Since there is no details how you test with CLI.I am also curious about it.

This might help. Enable root login over SSH

> Command line options --ask-become-pass, -K ask for privilege escalation password; does not imply become will be used. Note that this password will be used for all hosts. --become, -b run operations with become (no password implied) --become-method=BECOME\_METHOD privilege escalation method to use (default=sudo), valid choices: \[ sudo | su | pbrun | pfexec | doas | dzdo | ksu | runas | machinectl \] --become-user=BECOME\_USER run operations as this user (default=root), does not imply –become/-b

Understanding Privilege Escalation

I think you just want it working :) Ansible documentation is short and brief.

> Working with Inventory ansible\_ssh\_pass
> The ssh password to use (never store this variable in plain text; always use a vault. See Variables and Vaults) ansible\_ssh\_private\_key\_file Private key file used by ssh. Useful if using multiple keys and you don’t want to use SSH agent. ansible\_ssh\_common\_args This setting is always appended to the default command line for sftp, scp, and ssh. Useful to configure a ProxyCommand for a certain host (or group).

Working with Inventory
> -2 votes

---
Tags: jenkins, ansible, jenkins-pipeline
---


# Question
Title: It is not allowed to use stages more than once [Jenkins_Declarative_Pipeline]
I am completely new to the jenkins pipeline. I'm trying to create a project with multiple stages but in my `Import` stage , I want to execute `parallel`. For each `stage` in `parallel` , I have again multiple `stages`. I'm trying the following way, but getting syntax error as `It is not allowed to use stages more than once`.Can someone correct me how to achieve this, I have tried some online resources but unable to get clear syntax.

```
pipeline {
  agent any
  stages {
    stage('Build') {
      steps {
        echo 'from build'
      }
    }
  }

    stage('Test_A') {
      parallel {
        stages("GUI") { stage("Tests_A") { steps {echo 'from A'}} stage("Archive") {echo 'from Publish' } }
        stages("API") { stage("Tests_B") {steps {echo 'from B'} } }
        stages("CLI") {  stage("Tests_C") {steps {echo 'from C'} }}

      }
    }

  }

```

I want to create something like this

# Answer
> 2 votes
Got answer from StackOverflow and here is the link to the answer.

```
stage('Test') {
    parallel {
        stage("Test_A") {
            stages {
                stage("Tests_A") { steps { echo 'from A' } }
                stage("Archieve") { steps { echo 'from Archieve' } }
            }
        }
        stage("Test_B") {
            ...
        }

        ... and so on
    }
}

```

---
Tags: jenkins-pipeline, build-pipeline, groovy
---


title: Strange behaviour with docker-py

text:
# Question
Title: Strange behaviour with docker-py
Consider the following setup.

Precondition: have a healthy image i.e. service starts up.

E.g.

```
docker run -d IMAGE

```

Then, with the Python plugin docker-py, consider the following code which should be equivalent, only with added port mapping.

```
ingreedy = client.containers.create("openfoodbroker/ingreedy.service",
                                     name="ingreedy_service",
                                     detach=True,
                                     ports={'5000': 5000})
print(ingreedy.id)

```

This code prints out the container ID and I can remove the container using `docker rm -f`, but:

* port 5000 is not open;
* the container does not show up on `docker ps`

Any ideas?

# Answer
`docker.containers.create()` is documented to:

> Create a container without starting it. Similar to `docker create`.

Either call `docker.containers.run()` instead, or given the container object returned from `create()`, call `ingreedy.start()`.

(`docker ps` by default only shows running containers; `docker ps -a` should show the container in a "created" state.)
> 3 votes

---
Tags: docker, python
---


# Question
Title: EKS - created cluster from console with federated IAM admin - how to access
I have no programmatic access for the user who created the cluster only graphical. I tried a lot of things and I always get access denied on kubectl as I have to use another IAM identity that has programmatic access. I assume there is some trick with STS but I don't have the access keys of the root account. I guess if I give graphic access to the IAM identity that I want to use, it will work out, but I want to know if there is a way with my current situation. Also using the aws-cli is not an option for creation in the given scenario. The admin who created my user won't allow my federated IAM to have access keys.

# Answer
The only way to manage `IAM` users - using appropriate account with `IAM Administrator` privileges.

See AWS Identity and Access Management If you lost your root account, you may contact an `AWS` support, see Recover Your AWS Account Password
> 1 votes

# Answer
As said in the question summary, the issue is that I have created an EKS cluster from IAM user that doesn't have programmatic access. I have tried what I have suggested as scenario that I would like to avoid. Only solution in this case was to create another IAM identity that will create the cluster and give it programmatic access. As I am a federated user from OKTA, I have admin rights but using the console on the created identity is a security bridge, so I have allowed it only to login, create the cluster and then disabled the console access. The from the newly created identity I could use kubectl to control the cluster and add other IAM identities from the auth configmap.
> 0 votes

---
Tags: amazon-web-services, kubernetes, eks
---