# Question
Title: azure activity log, how to find releases
My security department has asked to ingest logs from Azure with the goal being to identity releases. The security team wants to be able to say:

> Hey, there was a release on mm/dd/yyyy at noon (because we see it in the Azure logs), show me the corresponding internal approval for that?

I've been looking around in the Azure Activity Log, but I can't seem to find anything related to releases, overwrites, or anything else that's clearly new-code being dropped onto a PaaS instance.

Like I'm pretty sure there was a release (devops build/release, pushed the code to a Paas) during this timeframe but I'm not seeing it.

Is there a way to do this? Am I barking up the wrong tree?

Edit: my little highlight there on the "Write Event" isn't related, just something that happened to a database at the time.

# Answer
Activity Logs can only able to show the resource level logs like creating a resources,deleting a resources modify the SKU of the resources etc,

Assuming from your question you are saying web app deployment from azure devops,which is more like a deployment of code to an existing webapp which will not be covered by activity logs. You need to go to that azure app service then you can that deployment details in `Deployment Center`
> 1 votes

---
Tags: azure, azure-devops, logging
---


# Question
Title: How to Block Traffic From a Single IP in AWS
Our server being hammered by traffic from some particular IP. It was causing a 20x increase in traffic to some URLs. So I wanted to block all traffic from that single IP.

# Answer
Yes, we can block traffic from single IP or IP range. Please follow below steps to do this

> Open **VPC** dashboard

> Open the **Network ACLs** view

> Open the ACL editor

1. Select the subnet to which your EC2 instances or load balancers are connected.
2. Click **Inbound Rules**
3. Click **Edit**

> Add a rule to block the traffic/IP

You will now see the ACL editor. On the last row, you can add a new rule.

Here is how you should fill out the fields:

# Rule

Use any number less than 100, which is the number of the default accept-all rule. This is important because rules are evaluated in order, and your rule needs to come before the default.

# Type

Select “All traffic” or Particular Protocol which you want to Block

# Source

The CIDR you want to block. To match a single IP address, enter it here and append /32. For example, I blocked `22.87.45.187/32`

# Allow/Deny

Select “DENY”

Now **click Save** and you should see the updated rules table.

Fore more details visit https://www.serverkaka.com/2018/05/block-traffic-from-single-ip-in-aws.html

---

### UPDATE: How to do it in CLI

> NOTE: This example command includes `--dry-run` option, so it is safe to run. It will not create *NACL* rule. Run it without this option, if you want to apply it.

```
aws ec2 create-network-acl-entry --dry-run --network-acl-id acl-146adb6f --ingress --rule-number 20 --protocol tcp --port-range From=80,To=80 --cidr-block 22.87.45.187/32 --rule-action deny

```

More examples: https://docs.aws.amazon.com/cli/latest/reference/ec2/create-network-acl-entry.html#examples
> 3 votes

---
Tags: amazon-web-services, aws-vpc
---


# Question
Title: How much I will pay for AWS spot: my bid or current spot price?
AWS EC2 Spot. My Spot Max Bid Price - 2$. Current Spot Price - 1$. How much I'll pay for instance? 2$ per hour or 1$ (and increasing|decreasing), equal to Current spot price, while my bid price is less than current (and get an error if current is more than my bid). Thanks!

# Answer
> 1 votes
At the beginning of each hour that a spot instance is running (instance hour, not wall-clock hour), the current spot price for instances of that type and availability zone is captured, and you pay that price for that hour. The next hour, and each hour after that, the process repeats. If the market price rises above your bid while the instance is running, the spot instance is terminated, but not sooner than 2 minutes after the price shifts. What happens after that depends on whether the spot request was persistent and how you configured termination/hybernation behavior.

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-instances.html#spot-pricing

Spot instances can also be termimated at any time without a price change, if the capacity is no longer available, such as can happen due to an increase in the number of on-demand instances, causing the size of the available spot pool to be reduced. Available capacity changes do not necessarily correlate with an immediate change in the market price. This did happen prior to the change in the spot price algorithm in late 2017 but spot price fluctuations are now dampened -- more loosely coupled to demand -- to prevent the sometimes dramatic price volatility (and presumably some market gaming) that occured with the old algorithm.

---
Tags: amazon-web-services
---


# Question
Title: How to setup multiple build environments in Azure DevOps/TFS2018?
So I have an application that contains an ASP.NET Core backend and a React frontend (pure static).

In our TFS 2018, we have created a Build that does the following:

* Get sources from the repo
* Get Nuget packages (backend)
* Build the backend
* Run backend tests
* Get NPM packages (frontend)
* Build the frontend
* Run frontend tests
* Generate and publish artifacts

Then, a Release pickups those artifacts and deploy them to an Azure AppService (backend) and an Azure Storage v2 account (frontend).

We currently have only one environment, but we need to support multiple ones. The problem is that the frontend build depends on some environment variables which are not present during the build stage. The build doesn't know where is it going to be deployed yet.

So there are a few options here:

* Generate the frontend build in a Release. This needs a way for the Release to get access to the sources, either via Git or by getting it from an artifact. Latest is really slow because of all the npm\_modules.
* Build should produce builds artifacts for all posible environments and each Release environment will pick up the proper one.

What's the recommended approach to handle this on TFS 2018?

# Answer
This is a general issue with react apps that is independent of what CI/CD stack you use that is discussed at https://stackoverflow.com/questions/49975735/rendering-an-environment-variable-to-the-browser-in-a-react-js-redux-production

With the Azure family of CI/CD products it is expected you create a release artefact then release that same artefact into environments using a release workflow. That is a typical best practice and expected by ”enterprise” release management processes. The general manifesto of native cloud apps is 12factor.net, and it also advises to ”build once, deploy many times, configured by environment variables”. So its a bit of a surprise to me that the best practices isn't really known in the react community so you have to hunt around for solutions.

In my case at that link above I didn't want to make deployments of our react apps different from our backend microservices where we have ”build once, run anywhere”. So we chanced the js code to fetch env vars from the server at page load. Also with this approach and by using containers we are guaranteed to always run the same front end libraries everywhere even if the front end build tool is misconfigured or buggy in terms of not always using the exact same dependency on every run.

In your case you do have a release workflow where you can use all the same built-in and marketplace tasks of a build workflow. This means its straight forward to add a front-end compile task to the release workflows.

On the downside a frontend build of a large react app can take a lot of time and a lot of memory. Memory probably won't be an issue for you. The time taken will be a little annoying. Yet the approach wouldn't be invasive so I would recommend it may be the least effort to set up and the easiest for colleagues to troubleshoot and maintain in the future as long as you have a stable frontend build. Historically npm was less reliable than other frontend builds at giving the exact same lib versions every run for the same package.json (e.g., npm vs. yarn). So you need to study that aspect and ensure that you are certain that every time you run the frontend build it is guaranteed to use the same exact dependency versions.
> 1 votes

---
Tags: azure, azure-devops, tfs, azure-pipelines, environments
---


# Question
Title: If we are using containers do we still need to worry about configuration management at an infrastructure level?
We are about to begin our journey with containers and I am wondering if we still need to utilize something like Chef, Ansible, Terraform, etc. to ensure that our infrastructure is properly maintained?

Since containers simply sit on that environment my assumption is that we will still need to properly maintain our configuration idempotence but how is this going to differ with us using containers?

# Answer
> 3 votes
It depends on what you are willing to rent and allow to be managed by someone else.

If we look at this diagram:

It shows a full range of options. At one end is Do It Yourself with on-prem hardware and setting up networks etc. Next to that is ”co-lo” which implies you buy servers that are installed in someones rack/network/data centre. Next to that is IaaS where you usually rent VMs on purely software defined networks and pay someone to manage the physical servers and physical network.

So where do containers fit in? Well I have seen big organizations treat cloud IaaS a bit like on-perm by setting up networks and firewalls in cloud to exactly match on-prem with private network connections into the cloud datacenter. So getting very little advantages of cloud. What they are mostly managing are dozens of legacy systems that they have ”lifted and shifted” from on-prem into a cloud provider as part of a multiple million dollar cloud contract. Obviously, they still need all the configuration management like they were using on-prem to manage the VMs. Then they install Kubernetes to manage containers and have to manage that. The cost of all the engineers they need to hire to keep it all running is huge. In such a setup its clear they need configuration management at every level and clusters of containers are just one more new thing they need to configure and manage.

At the other end of the spectrum is SaaS and you don't know if they use VMs or Containers. You then have no traditional configuration management and usually no devops. From the devops perspective an interesting choice is CaaS+PaaS where you rent capacity on a managed Kubernetes service that has devops extensions such as OpenShift. As a start up you can simply rent memory on a shared cluster where you are isolated from other tenants via software defined networks within OpenShift Kubernetes. Then you have outsourced both management and configuration of the VMs in the cloud and also outsourced patching and expanding the Kubernetes cluster running on the VMs. You can then focus on configuration management of your containers on Kubernetes. (**Note**: You still need to security patch the container runtimes. We have a script that checks for new security patches to RHOAR containers and rebuilds our containers with the latest versions.)

So it depends on how much you can and are willing to relinquish control of. Its easy to start out renting a fraction of a shared managed cluster then upgrade to a dedicated cluster still managed by someone else. Once you start out doing that on public cloud its hard to imagine any additional advantage of going back to managing VMs in the cloud. Unless you are forced to do so to run legacy apps or software you had to buy that doesn't support being run in containers. If you must configure VMs you should apply best practices of configuration management.

# Answer
> 3 votes
Welcome to containers, a very exciting direction indeed.

Software configuration management tools such as the ones you mention do cater for containers in many ways and if you're already familiar with Terraform then you'll probably already know this.

Kubernetes for instance can manage deployments and replication sets (of containers; 'pods') and nodes much like Puppet might manage virtual machines and micro-services.

I would say I personally use Puppet less in the container world but believe configuration management is still necessary in a container infrastructure though you may change the way you approach it with regard to the native tools that come along with containers as opposed to Puppet perhaps.

A Dockerfile is fairly self explanatory to satisfy the infrastructure as code perspective that Puppet delivers; and if you then get into Kubernetes clusters and nodes the same could be said about YAML files too.

Containers are bundles of repeatable infrastructure. I find this stack overflow answer supports my answer too; it certainly helped me in the past.

https://stackoverflow.com/a/41472271/1524645

---
Tags: docker, kubernetes, containers, configuration-management, infrastructure-as-code
---


title: Is jenkins-x the successor of jenkins?

text:
# Question
Title: Is jenkins-x the successor of jenkins?
I strongly doubt that jenkins-x is the successor of Jenkins. To me it looks like a tool that is able to create a ci/cd pipeline for new apps.

# Answer
> 0 votes
After evaluating and trying to build our current monolith, my conclusion is that jenkins-x should be used if one starts from scratch and wants to deploy a spring boot app in the cloud automatically. If that is not the case and one would like to run an existing monolith in the cloud then I suggest to use the stable/chart jenkins.

# Answer
> 1 votes
Jenkins X is a CI / CD platform for Kubernetes. This project is a sub-project of Jenkins. see below here

---
Tags: jenkins-x
---


# Question
Title: Apply resilience-testing in a company (to promote DevOps)? If true, why, how and what effect does it have on the Engineers? If false, why not?
Back in the day this Q&A was created.

ChaosMonkey is a tool that is able to stop random services on a platform in a certain time period. The goal of this is to get a stable platform that is fault tolerant. If this happens during the day when most of the Engineers are working, they get proactive and the infrastructure will be hardened.

At the moment, more and more services are created in AWS, GCP and Azure. The services on these platforms are getting more and more critical. Now it is time to onboard more and more engineers. Especially most of the developers are still silo minded, i.e. some of them only want to develop, but this is shifting drop by drop.

According to Wikipedia, ChaosMonkey was invented at Netflix and additional resilience tools were created as well:

```
| Purpose                                        | Tool              |
|:----------------------------------------------:|:-----------------:|
| Stops random services in production randomly   | Chaos Monkey      |
| Drop an availability zone, e.g. in AWS         | Chaos Gorilla     |
| Communication delays                           | Latency Monkey    |
| Fix unhealthy services                         | Doctor Monkey     |
| Dispose unused resources                       | Janitor Monkey    |
| Detect nonconforming instance and notify owner | Conformity Monkey |
| Disable instances that are vulnerable          | Security Monkey   |
| Detect problems between 10-18                  | 10-18 Monkey      |

```

Based on this table one could also achieve also the same purpose by start to write some shell scripts that stop random services for example.

Based on the information that was read, the definition of resilience-testing is basically testing if an app and underlying infrastructure is error prone in production.

This is also confirmed by this blog post that refers to how resilience testing is done at AWS, but also describes IBMs view on this type of testing. According to IBM there are two parts that have to be taken into account, i.e. problem impact and service level. In an ideal world the customer will not notice any downtime at all, but if a machine dies (minutes) or a complete data center (hours) then the customer could experience downtime. The most important thing to be transparently as possible to the customer and to minimize impact.

Currently, resilience-testing is not used in the company, but the more is read about it, it becomes interesting. Especially as it is also aligned with DevOps and the company tries to make Engineers DevOps minded. It looks like that resilience-testing promotes a DevOps mindset.

**Resilience testing is DevOps related**

Some people could argue that testing is not part of DevOps, but according to Wikipedia, tools like ChaosMonkey are aligned with the DevOps toolchain.

# Answer
> 0 votes
I think that chaos testing will promote DevOps unless people have been informed in advance and proper alert management has been enabled. If people do not get triggered then this will not work.

---
Tags: resilience-testing
---


# Question
Title: If we are using Kubernetes do we need to implement HA on both the infrastructure and application level?
Is this something that's going to be covered at the application level with pod autoscaling? Or should we implement both horizontal pod autoscaling and infrastructure high availability within our cloud provider? The goal is for us to see minimal downtime.

# Answer
> 2 votes
Your application requirements really do influence your options here, maybe add more detail about the app because this subject is very much about your application architecture too.

The application state is fixed, pod's are rebuilt as per the spec or source container that defines them and the replication sets take care of the deployment so I'd say the application from a Kubernetes perspective is already highly available by design and the reason why the documentation contains little coverage if any with regard to application level HA.

It's worth noting too that Kubernetes High Availability with kubeadm is still in its experimental stage and your clusters need to be running Kubernetes version 1.12 or later.

Google's Kubernetes Engine for instance; offers 'Regional Clusters' and these are spread over many zones in a region. This certainly increases your cluster availability. More here in GKE regions

The official word on the current state of Kubernetes HA cluster options are well documented here. https://kubernetes.io/docs/setup/independent/ha-topology/

---
Tags: kubernetes, infrastructure, high-availability, gcp, minikube
---


# Question
Title: Is there a good standard to follow when choosing a service mesh for your microservice application?
The industry appears to have chosen the de facto standards for containers (docker) and orchestration (kubernetes) but this doesn't appear to be the case for the options for implementing a service mesh.

Are there some good standards to follow when choosing from the available options for implementing a service mesh?

# Answer
> 1 votes
I would say that Istio is the de facto standard for Service Mesh. It is launched by both Google and IBM.

# Answer
> 0 votes
In general you are going to find the best support for “an X” on Kubernetes by looking at what “X” projects are members of the same software foundation so governed by the same organisation. Kubernetes is part of the Cloud Native Computing Foundation so you should study what “Xs” it offers. In the case of service mesh CNCF has Linkerd.

At this point in time though as other answers point out there is a lot of momentum around itsio. This uses Envoy which is a top level CNCF project to actually shape the traffic between your containers. That is reassuring as we would not expect k8s to suddenly break Envoy at a minor release so we might expect itiso to be able to keep up to date with k8s. That’s a guess though.

You should also consider whether there is a “service mesh like” tech for that stack you use already. If for example you are a Java shop doing microservice like a lot of big firms then you are probably using spring boot and spring cloud. In which case spring cloud already had a bunch of components that offer the same benefits as a mesh but do it in a more traditional way (libraries embeds in your app). By way of example here is an architecture demo of how to port a java spring cloud airline booking microservices demo app onto openshift Kubernetes. In that doc they mention itsio but actually they stick with spring cloud ribbon, eureka and zuul. Spring cloud also has distributed tracing etc.

This then leads to three choices if you’re a java shop. In order of more conservative first:

1. If your writing Spring Boot microservices stick with Spring Cloud. Developers can run it all as “just java” in their laptop and will take responsibility for it all working well on k8s as it’s “obviously” something they should master.
2. Deploy Linkerd as it’s an established CNCF project so the most mature option
3. Deploy itsio which went to v1.0 relatively recently but is likely to mature fast.

The smart money if you have the time is to try out all three.

---
Tags: containers, microservices, consul, orchestration
---


# Question
Title: How to kill all Docker containers based on a given image?
How to kill all Docker containers based on a given image?

As an example:

```
sudo docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
58251f04892a        mariadb:10.4        "docker-entrypoint.s…"   53 seconds ago      Up 52 seconds       3306/tcp                 optimistic_ramanujan
2a08ee7d9afe        mariadb:10.3        "docker-entrypoint.s…"   2 hours ago         Up 2 hours          3306/tcp                 thirsty_taussig
65abd7a3b3e1        mariadb:10.3        "docker-entrypoint.s…"   2 hours ago         Up 2 hours          3306/tcp                 angry_lederberg
32076aa4745e        mariadb:10.3        "docker-entrypoint.s…"   2 hours ago         Up 2 hours          3306/tcp                 distracted_kapitsa

```

I would like to kill all containers based *directly* on the `mariadb` image (ideally, both 10.4 and 10.3).

The following solution does not work since it will kill all containers having `mariadb:10.3` as an ancestor--and not only those directly based on that image:

```
sudo docker kill $( sudo docker ps --filter ancestor=mariadb:10.3 )

```

# Answer
> 4 votes
This will do want I want: killing all containers *directly* based on `mariadb` regardless of their tag--and without killing containers having `mariadb` somewhere higher in their ancestor list.

```
sudo docker kill $(
  sudo docker ps --format '{{.ID}} {{.Image}}' | awk '$2 ~ /^mariadb:/ { print $1}'
)

```

---
Tags: docker
---


# Question
Title: Deploy to GKE from CI server outside of cluster
I'm setting up our CI (Jenkins pipeline) to deploy apps to a GKE cluster. Our Jenkins sever is running external to GKE or any Google Cloud infrastructure. I'm thinking of using a docker image with helm and kubectl to deploy, but need to set up credentials. On my local dev machine, I authenticate with:

```
gcloud container clusters get-credentials [CLUSTER_NAME]

```

What would be the best way of setting up credentials for a Jenkins job? Do I have to install and use the `gcloud` tools on Jenkins, or is there another approach?

# Answer
> 1 votes
The easiest approach does appear to be to use `gcloud`, as trying to get a kubeconfig without it proved tricky.

Fortunately, `gcloud`, `kubectl` and `helm` are available as a single docker image `kiwigrid/gcloud-kubectl-helm`. For example:

```
docker run -it --rm --volume ./gcp-key-file.json:/data/gcp-key-file.json:ro kiwigrid/gcloud-kubectl-helm:2.11.0-224.0.0 bash
gcloud auth activate-service-account --key-file=/data/gcp-key-file.json
gcloud container clusters get-credentials dev-cluster --project <project name> --zone <zone>

```

---
Tags: jenkins, kubernetes, continuous-integration, google-cloud-platform
---


# Question
Title: What is the best way to deploy a compiled language project with docker and jenkins?
I have a java project, and I'm generating `.jar` file. My question is what is the best practice to use docker and jenkins for build and deploy the project? Is it build the jar with one docker image, then put the Jar file in another docker image and run a container from the last image in the production env or is there a better way to do that?

# Answer
I do not know how how about Jenkins, but GitLab CI has feature called artifacts which allow you to

* explitely share some files/directories between builds
* download them from GitLab Pipeline view

Additional pro is that allows you to make deployments manual actions.

Also if you are building fat JARs you do not need Docker for your deployments at all.

So the best solution for you is to have separate task for building release and separate for deployment (which can be manual job or not).
> 2 votes

# Answer
So, assuming Docker and Jenkins are set as stated, I would say your approach covers it: Build Jar in one container, grab the Jar and build an image with it, then build containers from that image for your environments.

For some thoughts on how to set it up:

Have a Docker Host (or swarm) running and start a Jenkins service on it. This is a compose file that I have running:

```
version: '3.7'

services:
  jenkins-local:
    user: root
    image: jenkinsci/blueocean
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ../../..:/var/projects
      - ./jenkins_home:/var/jenkins_home
    ports:
      - "9000:8080"

volumes:
  jenkins-home:

```

The important part is the volume where /var/run/docker.sock of the host is mounted to /var/run/docker.sock of the container. This allows Jenkins to access the host's Docker daemon via the Docker socket (make sure "user" is "root"). Find some reading about this here: https://getintodevops.com/blog/the-simple-way-to-run-docker-in-docker-for-ci

The reason why I'd choose the jenkinsci/"blueocean" image is because it allows for usage of blueocean pipelines. https://jenkins.io/projects/blueocean/

Within pipelines you write Jenkinsfiles to define your build pipelines:

```
node {
    stage('checkout scm') {
        checkout scm
    }
    stage('Build artifact') {
        // Connect to our Nexus Docker repository with the nexus_jenkins_user user credentials managed in Jenkins
        docker.withRegistry('https://registry.swarm', 'nexus_jenkins_user') {
            // Run a Maven builder image and attach it to the nexus network
            // Add a volume to cache the maven-repo in for consecutive runs
            docker.image('registry.swarm/builder/maven:maven-3.3.9').inside('-v maven-repo:/root/.m2 --network=nexus_nexus') {
                sh 'mvn clean package -Dmaven.test.failure.ignore=true'
            }
        }
    }
    stage('archive artifacts and save test results') {
        archiveArtifacts artifacts: "target/*.jar", fingerprint: true
        junit 'target/surefire-reports/*.xml'
    }
    stage('Build run container image, push Docker image to nexus') {
        docker.withRegistry('https://registry.swarm', 'nexus_jenkins_user') {
            docker.build("app/my-service:current").push()
        }
    }
}

```

Since everything happens in Jenkins Workspace, the only important line for the Dockerfile is this one:

```
COPY ./target/my-service-*.jar ./my-service.jar

CMD [ "java", "-Duser.timezone=UTC", "-jar", "my-service.jar"]

```

This will grab the jar, copy it over and use it for the run command in the image.
> 0 votes

---
Tags: jenkins, docker
---


# Question
Title: How to create a MariaDB/MySQL image using utf8 instead of the default latin1 charset?
I noticed when running a stock MariaDB Docker image in a container, the default character set is `latin1`:

```
CID=$(sudo docker run -d --rm -e MYSQL_ROOT_PASSWORD=p mariadb:10.3)
echo "show variables like 'char%';" | sudo docker exec -i "$CID" mysql -t -u root -pp
+--------------------------+----------------------------+
| Variable_name            | Value                      |
+--------------------------+----------------------------+
| character_set_client     | latin1                     |
| character_set_connection | latin1                     |
| character_set_database   | latin1                     |
| character_set_filesystem | binary                     |
| character_set_results    | latin1                     |
| character_set_server     | latin1                     |
| character_set_system     | utf8                       |
| character_sets_dir       | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+

```

I *suspect*(correct me if I'm wrong) this may be related to the container's system locale :

```
echo "locale" | sudo docker exec -i "$CID" bash
[sudo] password for sylvain:
LANG=
LANGUAGE=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=

```

Is there a way to create a MariaDB image using utf8 everywhere without having to write and mount a custom `my.cnf` configuration file? Ideally, I'm looking for a Dockerfile-based solution.

# Answer
In the official MariaDB image, `my.cnf` is configured to read additional configuration files from the `/etc/mysql/conf.d/` and `/etc/mysql/maria.conf.d/` directories:

```
root@bbd24e22be98:/# tail /etc/mysql/my.cnf

#
# * IMPORTANT: Additional settings that can override those from this file!
#   The files must end with '.cnf', otherwise they'll be ignored.
#
!include /etc/mysql/mariadb.cnf
!includedir /etc/mysql/conf.d/

root@bbd24e22be98:/# tail /etc/mysql/mariadb.cnf
# * Character sets
#
# Default is Latin1, if you need UTF-8 set all this (also in client section)
#
#character_set_server   = utf8
#collation_server       = utf8_general_ci
# Import all .cnf files from configuration directory
!includedir /etc/mysql/mariadb.conf.d/

```

Based on that, it is quite easy to setup utf8 everywhere by writing a configuration file to override the relevant settings, copying it at the right place from your Dockerfile:

```
$ cat db/utf8.cnf
[client]
default-character-set=utf8

[mysql]
default-character-set=utf8

[mysqld]
init_connect='SET collation_connection = utf8_unicode_ci'
init_connect='SET NAMES utf8'
character-set-server=utf8
collation-server=utf8_unicode_ci
skip-character-set-client-handshake

```

```
$ cat db/Dockerfile
# Base image
FROM mariadb:10.4

# utf8 everywhere
COPY utf8.cnf /etc/mysql/mariadb.conf.d/
ENV MYSQL_RANDOM_ROOT_PASSWORD=yes

```

I let you compare the output of `show variables` after those changes with the one provided in the question:

```
MariaDB [(none)]> show variables like 'char%'
    -> ;
+--------------------------+----------------------------+
| Variable_name            | Value                      |
+--------------------------+----------------------------+
| character_set_client     | utf8                       |
| character_set_connection | utf8                       |
| character_set_database   | utf8                       |
| character_set_filesystem | binary                     |
| character_set_results    | utf8                       |
| character_set_server     | utf8                       |
| character_set_system     | utf8                       |
| character_sets_dir       | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+

```
> 2 votes

---
Tags: docker
---


# Question
Title: declaratively mount a service account secret in OpenShift
I am using helm charts to deploy some webhook handlers. The handlers need to connect to the cluster they are running on to deploy the configuration within it. It works but one step is tricky to move into the helm chart. When the chart makes the service account it gives it a randomly named secret:

```
$ oc describe sa sa-build-webhook-realworld
Name:                sa-build-webhook-realworld
Namespace:           your-eng2
Labels:              app=sa-build-webhook-realworld
Annotations:
Image pull secrets:  sa-build-webhook-realworld-dockercfg-4qz9g
Mountable secrets:   **sa-build-webhook-realworld-token-bqtnw**
                     sa-build-webhook-realworld-dockercfg-4qz9g
Tokens:              sa-build-webhook-realworld-token-bqtnw
                     sa-build-webhook-realworld-token-k7lq8
Events:              <none>

```

I can grab that and set it on the deployment config with:

```
#https://codereview.stackexchange.com/a/212095/75693
SECRET_NAME=$(
    oc describe sa sa-tag-realworld |
    awk -F': *' '
        $2  { KEY=$1 ; VALUE=$2;  }
        !$2 {          VALUE=$1; }
        KEY=="Mountable secrets" && VALUE !~ /docker/ { print VALUE }
    '
)
oc set volume dc/webhook-realworld \
  --add --name=sa-secret-volume \
  --mount-path=/sa-secret-volume \
  --secret-name=$SECRET_NAME

```

I am trying to see if there is a way to do this declaratively in the chart yaml that creates the `dc` and `sa` at the same time which is here.

Is there a way to reference the generated service account secret from the deployment config in the chart?

Or should I be trying to get helm/helmfile to do the command-line lookup as part of its work?

# Answer
> 1 votes
Okay the answer to this is that you can set `spec.serviceAccountName` on the `dc` and the secrets are mounted.

Annoyingly the same does not hold true for BuildConfig objects. They have an inconsistently named `spec.serviceAccount`. Setting that doesn’t mount any secrets associated with the `sa`. It also means your build isn’t an “image puller” until you explicitly set that up. So this configuration option appears pretty useless as at openshift 3.11.

---
Tags: kubernetes, helm, openshift
---


title: Saltstack Jinja template pillar merge

text:
# Question
Title: Saltstack Jinja template pillar merge
I have a several pillars and I am trying to merge them in the same jinja variable

```
{%- import_yaml "server_list_de.yaml" as servers %}
{%- import_yaml "server_list_uk.yaml" as servers_uk %}
{%- servers.update(servers_uk) %}

```

but I've got following error:

```
File "/usr/lib/python2.7/dist-packages/salt/utils/context.py", line 211, in __getitem__
return self._dict()[key]
KeyError: '_errors'

```

# Answer
> 1 votes
The "saltey" way of doing this is typically in a map.jinja file. This one for example...

In that, you would:

```
{%- load_yaml as serverlist %}
#Note that a default list could be provided here
{%- endload %}

{%- set serverlist2 = salt['grains.filter_by'](serverlist, merge=salt['pillar.get']('pillarname:server_list_de')) %}
{%- set servers = salt['grains.filter_by'](serverlist2, merge=salt['pillar.get']('pillarname:server_list_uk')) %}

```

Then in your sls file you would then put at the top:

```
{%- from "formulaname/map.jinja" import servers with context %}

```

Servers should then be available to you. I've never tried merging twice though, so give it a try. At a minimum this may give you some direction as the syntax for doing a pillar merge like this may get you what you are looking for.

---
Tags: salt, jinja
---


# Question
Title: Chef - assign a value to variable in execution or converge phase
I am creating a recipe that has two resources. The initial VPN IP address is a empty valued string. Upon execution of the first resource, a IP address within the range 10.12.xx.xx is set. Then a second resource is executed with a guard block that checks for the VPN IP.

```
#
# Cookbook:: test_cookbook
# Recipe:: check-vpn-ip
#
# Copyright:: 2019, The Authors, All Rights Reserved.

#Getting the IP address using the ruby's Socket class.

ip_list = Socket.ip_address_list
vpn_ip_list = ip_list.select{ |ip| ip.ip_address.match(/^10.12/) }
!vpn_ip_list.empty? ? ip_addr = vpn_ip_list.first.ip_address : ip_addr = ""

execute 'manually_start_open_vpn' do
command "sudo openvpn #{node['openvpn-conf-path']}/#{host}.conf &"
action :nothing
only_if {ip_addr.length.eql?(0)}
end

new_ip_list = Socket.ip_address_list
new_vpn_ip_list = new_ip_list.select{ |ip|
                                   ip.ip_address.match(/^10.12/) }
!new_vpn_ip_list.empty? ? newvpn_ip_addr =
                new_vpn_ip_list.first.ip_address : newvpn_ip_addr = ""

ruby_block 'chat-bot' do
   block do
        machine_data = {text: "OpenVPN IP not assigned to #{host}
            software_version: 18.4.4 \n This is a test message please
              ignore @all"}.to_json
        header = {'Content-Type': 'text/json'}
        http = Net::HTTP.new(google_chat_uri.host,
                   google_chat_uri.port)
        http.use_ssl = true
        request = Net::HTTP::Post.new(google_chat_uri.request_uri, header)
        request.body = machine_data
        response = http.request(request)
    end

   end
only_if {newvpn_ip_addr.length.eql?(0)}
action :nothing
end

```

Since there is no vpn ip address of the range 10.12.xx.xx during compile phase, the value of `ip_addr.length.eql?(0)` is true and the guard block condition in first resource is satisfied and it's executed.

As the vpn ip has already been assigned, the `newvpn_ip_addr.length.eql?(0)` should be false and the 2nd ruby block should not be executed. But this returns a false value as the code `newvpn_ip_addr` is executed during compile time itself.

Is there any way to make sure that the value for `newvpn_ip_addr` in the below code is assigned and checked during the execution phase post the execution of first resource?

```
    new_ip_list = Socket.ip_address_list
    new_vpn_ip_list = new_ip_list.select{ |ip|
                                   ip.ip_address.match(/^10.12/) }
    !new_vpn_ip_list.empty? ? newvpn_ip_addr =
                new_vpn_ip_list.first.ip_address : newvpn_ip_addr = ""

```

So that the guard block in 2nd resource is not executed if the expected IP address is not assigned. `only_if {newvpn_ip_addr.length.eql?(0)}`

# Answer
> 1 votes
You have a problem of compile vs converge times as you guessed.

You code between the execute resource and the ruby\_block resource is executed before the execute resource is converged.

The easiest way to circumvent that is to put that code within a ruby\_block, and to avoid scoping problem use node.run\_state hash to store the value to be checked.

A rewrite of your recipe which should work would be:

```
#
# Cookbook:: test_cookbook
# Recipe:: check-vpn-ip
#
# Copyright:: 2019, The Authors, All Rights Reserved.

#Getting the IP address using the ruby's Socket class.

ip_list = Socket.ip_address_list
vpn_ip_list = ip_list.select{ |ip| ip.ip_address.match(/^10.12/) }
!vpn_ip_list.empty? ? ip_addr = vpn_ip_list.first.ip_address : ip_addr = ""

execute 'manually_start_open_vpn' do
  command "sudo openvpn #{node['openvpn-conf-path']}/#{host}.conf &"
  action :nothing
  only_if {ip_addr.length.eql?(0)}
end

ruby_block 'check_vpn_ip_list' do
  block do
    new_ip_list = Socket.ip_address_list
    new_vpn_ip_list = new_ip_list.select{ |ip| ip.ip_address.match(/^10.12/) }
    node.run_state['newvpn_ip_addr'] = !new_vpn_ip_list.empty? ?  new_vpn_ip_list.first.ip_address : ""
  end
end

ruby_block 'chat-bot' do
  block do
    machine_data = {text: "OpenVPN IP not assigned to #{host}
        software_version: 18.4.4 \n This is a test message please
        ignore @all"}.to_json
    header = {'Content-Type': 'text/json'}
    http = Net::HTTP.new(google_chat_uri.host,                       google_chat_uri.port)
    http.use_ssl = true
    request = Net::HTTP::Post.new(google_chat_uri.request_uri, header)
    request.body = machine_data
    response = http.request(request)
  end
  only_if {node.run_state['newvpn_ip_addr'].length.eql?(0)}
  action :nothing
end

```

There's no real use of response = in the last line as you don't use it later, but I let it as is in case you wish to log in the ruby block the response.

---
Tags: chef
---


title: How to filter ELB logs by today's date?

text:
# Question
Title: How to filter ELB logs by today's date?
I've tried this query:

```
SELECT request_timestamp
FROM elb_logs
WHERE date_trunc('day', request_timestamp) = date_trunc('day', current_timestamp)
LIMIT 10

```

but it fails with:

> SYNTAX\_ERROR: line 3:7: Unexpected parameters (varchar(3), varchar) for function date\_trunc. Expected: date\_trunc(varchar(x), date) , date\_trunc(varchar(x), time) , date\_trunc(varchar(x), time with time zone) , date\_trunc(varchar(x), timestamp) , date\_trunc(varchar(x), timestamp with time zone)

I've checked the Date and Time Functions and Operators docs page, but I can't see anything wrong.

I've created the `elb_logs` table in Amazon Athena according to this page.

What I am missing?

# Answer
Either `request_timestamp` needs to be converted from the ISO 8601 formatted string into a timestamp with time zone, or `current_timestamp` needs to be converted into the string.

One method is to convert `request_timestamp` into a string in format `%Y-%m-%d`, e.g.

* `split_part(request_timestamp, 'T', 1)`
* `substr(request_timestamp, 1, 10)`

then compare it with `current_timestamp` which can be converted to the ISO 8601 formatted string.

Here is the final query:

```
SELECT *
FROM elb_logs
WHERE split_part(request_timestamp, 'T', 1) = split_part(to_iso8601(current_timestamp), 'T', 1)
LIMIT 10

```
> 1 votes

---
Tags: amazon-web-services, sql
---


# Question
Title: How to list the containers attached to a given user-defined network?
I'm reworking a bunch of scripts using the deprecated `docker run --link` option in favor of user-defined bridges.

By searching in the doc, I didn't find any command to list all containers attached to a given network. For now, I am parsing the JSON output of `docker network inspect` through the `jq` tool; I've posted that as an answer below since it may help someone. But, do you know another option to achieve that? Maybe using a dedicated Docker command, option or a custom Go format?

# Answer
> 1 votes
The `docker network inspect` command produces a JSON output that can be parsed using `jq` to extract the relevant piece of information:

```
sudo docker network inspect app-bridge | jq '.[].Containers[].Name'

"app2"
"app3"
"app1"
"laughing_easley"
"database"
"thirsty_wing"

```

If you are already familiar with JSON processing with `jq`, the advantage is you can reinvest your knowledge without having to bother with Go templates. On the other hand, it relies on an external tool which may or may be not available at your site.

---
Tags: docker
---


# Question
Title: Jenkins: mercurial scm: clean=false is not working as expected
**EDIT:** It seems as if the MercurialSCM plugin hardcodes the `update` to use the `--clean` option. See this line. Anyone have any ideas what's the idea?

I have the following checkout step in my Jenkinsfile:

```
checkout([
        scm: [
            $class: "MercurialSCM",
            source: "ssh://hg@bitbucket.org/xxxxx/${repo}",
            credentialsId: "jenkins",
            revision: params[branch] ? params[branch] : "default",
            subdir: (repo == "f8_root") ? "" : "aarch64/${repo}",
            clean: false,
        ],
        poll: true,
])

```

The argument for `clean` is set to `false`. But, the plugin still seems to issue an `update` with the `--clean` option. From the logs:

```
[Pipeline] checkout
[f8_fw] $ hg showconfig paths.default
[f8_fw] $ hg pull --rev feature/test-framework
pulling from ssh://hg@bitbucket.org/doblesc/f8_fw
no changes found
[f8_fw] $ hg update --clean --rev feature/test-framework

```

How can I fix this?

# Answer
Seems like `--clean` is hardcoded into the MercurialSCM Jenkins plugin.

See OP for the link to the culprit code.

I have opened up a ticket on Jenkins jira.
> 0 votes

---
Tags: jenkins, jenkins-plugins
---


# Question
Title: Chef14 Load a provider in a recipe Chef::Provider::DeployKeyGithub
I have a problem during migration of chef-client 12 to chef-client 14 and seems that the old way to load Providers is different in chef14

`Chef::Provider::DeployKeyGithub` cannot be load

```
    deploy_key deploy_key_name do
        provider Chef::Provider::DeployKeyGithub
        path "/home/#{current_user}/.ssh"
        credentials({
            :token => File.read('/root/someid').strip
        })
        repo 'test/test_config'
        owner 'root'
        group 'root'
        mode '0400'
        action :add
    end

```

Error:

<pre><code>================================================================================

NameError
---------
uninitialized constant Chef::Provider::DeployKeyGithub

Cookbook Trace:
---------------
  /var/chef/cache/cookbooks/flume_wrapper/recipes/default.rb:89:in `block in from_file'
  /var/chef/cache/cookbooks/flume_wrapper/recipes/default.rb:88:in `from_file'```

Does anyone knows how is the new way to load a Provider in a recipe for chef14 client?
</code></pre>

# Answer
> 1 votes
This compatibility issue was solved for chef14 with the new load format of a provider in a recipe.

```
deploy_key deploy_key_name do
      provider Chef::ProviderResolver.new(node, find_resource!("deploy_key[github]"), :nothing).resolve
      path "/home/#{current_user}/.ssh"
      credentials({
          :token => File.read('/root/someid').strip
      })
      repo 'test/test_config'
      owner 'root'
      group 'root'
      mode '0400'
      action :add
  end

```

# Answer
> 0 votes
Unfortunately, I wasn't able to get the cookbook working using the ProviderResolver syntax for loading a provider.

However, I did manage to find an updated fork of the cookbook on GitHub that resolves the Chef 13 deprecations here: https://github.com/alvlapo/cookbook-deploy\_key

I believe that should circumvent the issue you're seeing.

---
Tags: chef
---


# Question
Title: How does Docker container orchestration work?
I am familiarizing with the architecture and practices to package, build and deploy software or unless, small pieces of software.

If suddenly I am mixing concepts with specific tools (sometimes is unavoidable), let me know if I am wrong, please.

On the road, I have been reading and learning about the images and containers terms and their respective relationships in order to start to build the workflow software systems of a better possible way.

And I have a question about the services orchestration in the context of the docker :

* The containers are lightweight and portable encapsulations of an environment in which we have all the binary and dependencies we need to run our application. OK
* I can set up communication between containers using **container links** `--link` flag.

I can replace the use of container links, with docker-compose in order to automate my services workflow and running multi-containers using `.yaml` file configurations.

And I am reading about of the Container orchestration term, which defines the relationship between containers when we have distinct "software pieces" separate from each other, and how these containers interact as a system.

Well, I suppose that I've read good the documentation :P

My question is:

A docker level, are **container links** and `docker-compose` a way of container orchestration?

Or with docker, if I want to do container orchestration ... should I use `docker-swarm`?

# Answer
> 4 votes
welcome to DevOps SE, @bgarcial!

Well I think there are the following options depending on the use case:

* Docker Container *links* are a according to the vendor site a legacy feature and you shouldn't use it
* Docker *compose* should work well but you can't scale horizontally: the stack is runnable only one a single machine
* With *Docker Swarm* using mostly same YML you can distribute your containers across many machines
* If you are going to deploy large scale production environments you might want to choose *Kubernetes* over Swarm, maybe have a look for example on *Rancher/Kubernetes*.

# Answer
> 3 votes
Welcome to the wonderful world of container orchestrators.

Consider microservices for a moment. Each service should be independently deployed and independently scaled across a cluster of VMs. Where does basic Docker and Docker Compose fit in? They don’t compete with distributed system orchestrators like Openshift Kubernetes, CloudFoundry or Docker Swarm.

A microservices platform that scales out and has fault tolerances needs a cluster of servers with internal load balancing where traffic to each service is load balanced to many replicas. The idea that you would link two containers doesn’t make sense in a microservices world as each container should connect to a logical service that balances traffic to many pods that are scaled independently of other services. So study the templates for how large scale apps run across a whole cluster of VMs with something like OKD Kubernetes.

So where does Docker Compose fit in if you are using a full orchestrator? I don’t think it does if you are using something like Kubernetes. Developers can run Minikube or Minishift on their laptop. Better yet one “not production” cluster can run a different namespace for every developer to have their own personal or shared sandpits to develop in. You can also setup one namespace per test environment on that one cluster simplifying you management. (Production can be in a dedicated cluster.)

Note that Docker isn’t the only container runtime these days and IMHO it’s unlikely to not be the default one for orchestrators in other than Swarm in the future. For example k8s now uses a standard API that lets you plug in alternatives and runtimes that focus on security and footprint are likely to become the default.

**Update**: There are some special cases where “running two image in the same process space” makes sense. One is the “sidecar pattern”. Rather than than mixing business services together this is where you run a technical service image with every business service image. An example of that is a service mesh that uses an smart proxy in every container. Every business service only connects to the smart proxy in the same process space that forwards traffic to the next business service. The smart proxy can then do things like service discovery, load balancing, circuit breaking, load shedding, protocol upgrades, mutual TLS between all business services.

---
Tags: docker, containers, orchestration
---


# Question
Title: Impact on policy after deleting IAM user
I have created one IAM User named "test" with S3 and Glacier Permission.

Now with user "test" i created one life cycle policy to move data from S3 to Glacier at every 5 days.

Now, i delete IAM user "test" because i don't need it anymore.

So my question here is, After deleting user "test", S3 Life cycle policy will work or not? OR will there be any impact on life cycle policy?

# Answer
> 4 votes
> S3 Life cycle policy will remain and working as it is.

When you use the AWS Management Console to delete an IAM user, IAM automatically deletes the following information for user:

* The user Any group memberships.
* The user is removed from any IAM groups.
* That the user was a member of Any password associated with the user
* Any access keys belonging to the user.
* All inline policies embedded in the user (policies that are applied to a user via group permissions are not affected)

# Answer
> 4 votes
The policy will still exist.

The documentation lists what is deleted when a user is deleted, any resources that the user has created will persist.

---
Tags: amazon-web-services, amazon-s3, aws-iam
---


# Question
Title: What are the benefits of spreading the various services of a multi-tiers application over several user-define bridges?
I have a simple system made of a database backend, several instances of a web application and a load balancer. Each service is running in its own Docker container, all of them being hosted on the same physical server. The load balancer listens on the port 80 which is mapped to the same port on the host. All ingress/egress traffic is supposed to pass through that load balancer.

Currently, all those containers are attached to a single user-defined bridge.

I read it would be better to connect the load balancer and the app instances through one bridge, and the app instances and the database through a second one. *Instinctively*, it makes sense to avoid putting the DB on the public-facing bridge. But *objectively*, in a dockerized environment, **what are the benefits of using two bridge networks instead of only one here?** Which kind of threats is mitigated by doing so?

# Answer
Your updated question completely changed my understanding :)

Since the db is on a user-defined bridge in both cases, the advantages of using user-defined vs legacy/default bridges discussed in the doc section you referenced (which are applicable in both cases) aren't what differentiates the 2 cases. From that perspective they're equivalent.

Original Answer:

I think most/all of the differences between the legacy/default bridge and the user-defined ones listed in the doc section you referenced are pretty objective advantages :)

For your particular question, in `User-defined bridges provide better isolation and interoperability between containerized applications` we see:

> Using a user-defined bridge, only the web port needs to be opened, and the database application doesn’t need any ports open, since the web front-end can reach it over the user-defined bridge

Since in such case the db doesn't have open ports it can't be accessed from the outside, so all threats associated with having the db reachable from the outside are avoided.
> 1 votes

---
Tags: docker, deployment, networking
---


# Question
Title: Why `docker stack` needs Docker to run in swarm mode? What are the implications when working on a single node?
I am experimenting with `docker stack` as a replacement to `docker-compose`. But I had the following message on my first deploy attempt:

```
sudo docker stack deploy --compose-file stack/app-stack.yml app-stack

this node is not a swarm manager. Use "docker swarm init" or "docker swarm join" to connect this node to swarm and try again

```

Once issued the `docker swarm init` command, `docker stack deploy` works as expected.

Why it is mandatory to switch to *swarm mode* to run `docker stack`? And most important, what are the implications of switching to *swarm mode* when working on a single node?

# Answer
> 3 votes
> Why it is mandatory to switch to swarm mode to run docker stack?

A stack and service are swarm mode objects by their very definition. The stack is a collection of services, and each service is a definition of a target state. Swarm mode takes those target states and manages containers to achieve that target state.

> And most important, what are the implications of switching to swarm mode when working on a single node?

Pretty minimal from the user perspective. There's another few ports opened and listening. If you try to directly manage containers (stop, rm, restart) that are managed by a service, you'll find that swarm mode will take steps to get you back to the service definition, which confuses some that keep finding new containers recreated when they delete all running containers directly. You just need to run the `docker service` commands to manage them instead.

# Answer
> -1 votes
docker stack api is introduced from docker version 1.12.0 and above. the api is enabled in swarm mode.

to enable the docker stack api, run below command docker swarm init

when you run the above command, your host would become single master docker swarm cluster. you should be able to use docker stack and docker service and other new api's.

you can also add new hosts to the swarm cluster as a worker or manager

---
Tags: docker, docker-swarm
---


# Question
Title: Has `docker-compose` to be phased out in favor of `docker stack`?
According to the August 2017 article the core functionalities of `docker-compose` were integrated into the new `docker stack` command. At the end of the article, the author also assumes `docker-compose` will soon to be deprecated. The `docker stack deploy` documentation is currently pretty sparse.

The article is six months old now. So, is `docker stack` the way to go, or is there still some advantage in using `docker-compose` today? Especially, I will soon introduce Docker to students. Is it worth spending time studying with them details of `docker-compose` or should I just jump to `docker stack`?

# Answer
There are a few features of `docker-compose` that are not implemented in `docker stack deploy` and vice versa.

With `docker-compose`, you can run privileged containers and change capabilities which is not supported with swarm services yet. You can also run a container that is designed to exit, e.g. a batch job, more intuitively in docker-compose than in swarm mode. If your containers are designed for sysadmin tasks, this may make more sense.

With `docker stack deploy`, you get access to secrets and configs, along with replication settings, configuration of rolling updates, ingress networking with the routing mesh, and various other swarm mode features. If your containers are designed for highly available microservices, this makes more sense.

Both of these use the same docker-compose.yml definition file (with version 3 required for swarm mode). That makes moving between the two fairly easy. There are also projects like compose-on-kubernetes and kompose that make the transition to kubernetes with the same stack definitions easy.
> 5 votes

---
Tags: docker, docker-compose
---


# Question
Title: Github rest api post label gives error for array input
We are hosting GitHub enterprise for our development. I am able to access the reset api and create jira, create PR etc.

When I try to add label to my PR, it gives error for array.

```
curl -X POST -u githuser:gittoken https://api.github.mycompany.com/repos/team/repo/issues/560/labels -H "Content-type: application/json" -k -d '{"labels": ["bug"]}' -H "Accept: application/json"
{
  "message": "Invalid request.\n\nFor 'links/2/schema', {\"labels\"=>[\"bug\"]} is not an array.",
  "documentation_url": "https://developer.github.com/enterprise/2.13/v3/issues/labels/#add-labels-to-an-issue"
}

```

I check, `bug` is valid label.

```
curl -u githuser:gittoken -X GET \
  https://api.github.mycompany.com/repos/team/repo/labels
[
    {
        "id": 163461,
        "url": "https://github.mycompany.com/api/v3/repos/team/repo/labels/+1",
        "name": "+1",
        "color": "c2e0c6",
        "default": false
    },
    {
        "id": 382069,
        "url": "https://github.mycompany.com/api/v3/repos/team/repo/labels/Blocked",
        "name": "Blocked",
        "color": "fbca04",
        "default": false
    },
    {
        "id": 163462,
        "url": "https://github.mycompany.com/api/v3/repos/team/repo/labels/Changes%20Requested",
        "name": "Changes Requested",
        "color": "cc317c",
        "default": false
    },
    {
        "id": 404926,
        "url": "https://github.mycompany.com/api/v3/repos/team/repo/labels/Release%20Review",
        "name": "Release Review",
        "color": "5319e7",
        "default": false
    },
    {
        "id": 228780,
        "url": "https://github.mycompany.com/api/v3/repos/team/repo/labels/Review%20Pass",
        "name": "Review Pass",
        "color": "009800",
        "default": false
    },
    {
        "id": 228781,
        "url": "https://github.mycompany.com/api/v3/repos/team/repo/labels/Review%20Requested",
        "name": "Review Requested",
        "color": "eb6420",
        "default": false
    },
    {
        "id": 426113,
        "url": "https://github.mycompany.com/api/v3/repos/team/repo/labels/Staging%20Bug",
        "name": "Staging Bug",
        "color": "d6021a",
        "default": false
    },
    {
        "id": 163457,
        "url": "https://github.mycompany.com/api/v3/repos/team/repo/labels/bug",
        "name": "bug",
        "color": "fc2929",
        "default": true
    },
    {
        "id": 163458,
        "url": "https://github.mycompany.com/api/v3/repos/team/repo/labels/duplicate",
        "name": "duplicate",
        "color": "cccccc",
        "default": true
    },
    {
        "id": 163459,
        "url": "https://github.mycompany.com/api/v3/repos/team/repo/labels/enhancement",
        "name": "enhancement",
        "color": "84b6eb",
        "default": true
    },
    {
        "id": 163460,
        "url": "https://github.mycompany.com/api/v3/repos/team/repo/labels/help%20wanted",
        "name": "help wanted",
        "color": "159818",
        "default": true
    },
    {
        "id": 163463,
        "url": "https://github.mycompany.com/api/v3/repos/team/repo/labels/wontfix",
        "name": "wontfix",
        "color": "ffffff",
        "default": true
    }
]

```

I tried with other combinations, like string only, string with coma etc.. but its same error.

Actual command:

# Answer
> 2 votes
From my understanding of the docs, you should directly send an array (which is valid JSON), without wrapping it in an object. That is `["bug"]` instead of `{"labels": ["bug"]}`.

So, the proper request should be:

```
curl -X POST -u githuser:gittoken \
    https://api.github.mycompany.com/repos/team/repo/issues/560/labels \
    -H "Content-type: application/json" -k \
    -d '["bug"]' \
    -H "Accept: application/json"

```

---
Tags: git, github, api
---


# Question
Title: What are the differences between application virtualization, emulation layer, and container virtualization?
What are the differences and relations between application virtualization (such as Flatpak), emulation layer (such as wine, cygwin), and container virtualization (such as Docker)?

I have been trying to figure them out, but always end up not catching the main points. Especially the fundamental differences between their virutalization levels.

# Answer
> 4 votes
Container virtualization with Linux is a combination of kernel features, namespaces and cgroups, used to run an application in a sandbox. Each application has it's own filesystem, network stack, pid namespace, etc, but runs on the same OS kernel as the host. From the host you can see the processes running, but from inside the sandbox, all you can see is your application. It is designed for managing server side applications.

Application virtualization is very similar to container virtualization, and the lines are a bit fuzzy to me. In the context of something like Flatpak, it is using the same namespaces and cgroups you see with container tools like docker. The key difference is they have designed the tool for desktop use cases, meaning tools that are designed to be accessed with a local GUI, rather than running remotely as something like a web server.

Emulation is a way to run programs designed for another OS. You typically cannot run a Windows binary on a Linux system and vice versa, the OS kernels are very different, the binary formats are different, they're effectively speaking different languages. The emulation layer provides a translation to run non-native programs on the local machine without creating an entire virtual machine and running a separate OS. This is very different from the container and application virtualization where you run native OS applications on a shared OS kernel.

# Answer
> 1 votes
As often, the vocabulary is overloaded, and the limits between technologies are somewhat fuzzy.

As a complement to the BMitch's answer, in my understanding, *emulation* is a way to run an application designed for another instruction set architecture. For example, it allows to run an ARM software on an x86 host. QEMU virtual machines are designed for ISA emulation.

Running x86 Windows application on x86 Linux using Wine1, or x86 Linux app on x86 Solaris using an lx branded zone does not require ISA emulation since that application and the host are using is the same instruction set. Only the system calls needs to be "emulated", not the processor. Sometimes we call that system call translation. It is the role of the **compatibility layer** to perform the needed translations as transparently as possible for the guest application.

---

1 Actually the *Wine* name was dubbed as a backronym for "Wine Is Not an Emulator"

---
Tags: docker, virtualization
---


# Question
Title: How to attach two (or more) networks to a container so they are available when the container starts?
I noticed if you try to attach a container to two networks, the following syntax does not cause an error, but effectively attach the container only on the network specified by the last `--network`:

```
sudo docker run \
        --network front-end \
        --network back-end \
        --name app1 ${IMAGE}

sudo docker network inspect front-end | jq '.[].Containers[].Name'
# <-- nothing here
sudo docker network inspect back-end | jq '.[].Containers[].Name'
"app1"

```

How to attach two (or more) networks to a container so they are available when the container starts?

# Answer
> 1 votes
## Do that:

You cannot attach several networks using the `--network` option. You have to rely on the `docker network connect` command. So if you want the multiple network interfaces to be available at container startup, you have to do it in three steps:

1. *create* the container,
2. *connect* the extra network needed,
3. and finally *run* the container.

```
# create a container, attaching it to one user-defined bridge:
docker create --name app3 \
              --network back-end \
              ${IMAGE}

# connect the other bridge networks:
docker network connect front-end app3

# start the container
docker start app3

```

```
docker exec -it app3 ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
465: eth1@if466: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:ac:13:00:03 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 172.19.0.3/16 brd 172.19.255.255 scope global eth1
       valid_lft forever preferred_lft forever
467: eth0@if468: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:ac:14:00:03 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 172.20.0.3/16 brd 172.20.255.255 scope global eth0
       valid_lft forever preferred_lft forever

```

## Dont' do that:

It is tempting to do that instead:

```
# You probably DON'T want this:
# create a container with the default network settings
docker create --name app3 \
              ${IMAGE}

# connect the user-defined bridge networks:
docker network connect back-end app3
docker network connect front-end app3

# start the container
docker start app3

```

However, there is a pitfall here: when creating the container without an explicit `--network` option, Docker attaches it to the default bridge. Then the two `network connect` commands will add two *more* networks. And the container end-up bein attached to the *front\_end* and *back\_end* networks as expected. But also to the *default bridge*, something you probably don't want when using user-defined networks.

---
Tags: docker, networking
---


# Question
Title: Issue with custom domain SSL certification
We are working on saas application which is hosted on app.example.com. Now we are trying to give the option to the users where their own domain or sub domain like dev.stack.com point to my application by adding A record to 1.2.3.4 IP address (ip address where app.example.com is hosted).

With the above setup, we are facing the issues with ssl certificate of the vendor domains. Is there any way we can create global certificate and add it to our servers which can work for any domain pointed to our servers.

# Answer
Can you obtain a secure certificate for someone else's domain or subdomain without their help - hopefully not.

One of the core security pillars of a secure certificate is that it ensures that you are actually connected (through an encrypted connection) to the the site you wanted to connect to. For this purpose the minimum check every certificate issuer makes is to check for ownership of the domain by the entity requesting it.

For "manual" issuers, this often involves uploading a file to a web server on this domain, creating a DNS record or clicking on a link in an email to the registrant, administrator or technical contact in the whois record of the domain.

You can replicate this process for your needs for your SaaS application by using LetsEncrypt: It allows you to either

1. upload a file to a folder on a web server for the domain: In this case you would advise the client to point an A record (or probably better CNAME) to your server and once resolved, you can issue the certificate with letsencrypt by copying the file yourself
2. create a specific DNS record, which again allows you to create the certificate.

Lets encrypt certificates are valid for 90 days, but as you will need the A record in place in anycase to run your SaaS, you will be able to renew the certificate yourself regularly.

P.S.: Not too long ago, subsidiaries of Symantec were found (twice) to not always have checked domain ownership and then did not appear to take remedial action consistently (twice). This led to the "distrusting" of all Symantec certificates in Chrome and Firefox after giving owners of these certs some time to create new certificates.
> 1 votes

# Answer
You can provide subdomain entry for client use. eg. Client\_x.yourapplication\_domain.com. Otherwise, you need to have workaround like putting request forwarder(proxy LB) in front of your application and configure client domain with SSL there. Forward clientdomain.com to Client\_x.yourapplication\_domain.com.
> 0 votes

---
Tags: nginx, ssl, dns
---


# Question
Title: CIS Benchmark for Docker and Docker Bench for Security: why, how and when?
I just learned about Docker Bench for Security and CIS Benchmark for Docker Community Edition.

Why, how and when am I supposed to use these tools?

# Answer
Larger organizations are concerned that containers introduce more security risks than traditional VMs as there isn't any hypervisor segregation. Developers running arbitrary images, that run as root, that they don't bother to security patch, looks worse for security team's than automatically patched VMs controlled by ops teams without containers.

Misconfiguration of docker demon itself can also introduce security risks. It's fairly common for complex software to have security tuning tools and to run them as part of ”operational readiness checks” of the live environment before going live as a security audit. Then also run them on some periodic cycle or after any major upgrades of the software that is being audited. The tools you name will be very useful for that purpose.

Software vendors who build container orchestrators are actively working in removing any dependency on needing to run docker demon see cri-o.io. Security, footprint and fork time are areas that vendors are competing on. CoreOS was a big innovator in this space before they were acquired you can look at some of their architecture presentations to get a flavor of what containers at scale workout docker looks like.

From a devops perspective we should empower devs to run security tools all the time to fix issues early. In practice devs might be using Docker for Windows locally but the live env would be a container orchestrator possibly running rkt. In reality devs in large organizations are not going to security scan infrastructure they didn't create and dont have root on. Even at a startup not every dev can be familiar with all of the tooling and software running in production. An external security audit is a smart investment.

IMHO if your teaching containers from a developer perspective it's worth pointing out such tools. Certainly devs in my experience are woefully unaware that running containers they find on docker hub as root isn't the smartest way to be secure. I would only expect to see such tools to be covered in depth by courses teaching security analysts how to audit systems that use Docker.
> 1 votes

# Answer
Docker Bench is a scripted report of many of the CIS recommendations (at least those that can be scripted. Various organizations use the CIS recommendations as a starting point for their security policy, the goal is to have a recognized organization provide the best practices. This is useful when dealing with auditors in regulated environments since a company can often show what they've done towards the CIS recommendations to quickly get through most of the audit (the auditors will be familiar with the CIS recommendations).

That said, I personally feel the advice from CIS is a mixed bag when dealing with Docker. Some of the suggestions look like a copy and paste from advice given to security physical bare metal systems, rather than a sandboxed application. And some of the recommended practices, weird things like avoiding the use of `ADD` in a Dockerfile, have little security reasoning (anything you can include with an `ADD` of a tar you could include with a `COPY` of a full directory) and will flag just about every image you are running since `ADD` is used as the first step of nearly every base image. There's a list of other advice they give that has little to nothing to do with security and is just a recommended configuration setting that cannot be applied in all environments.

What's lacking with CIS recommendations for Docker is actually securing the applications running inside of the container. This gets out of scope since you can run anything in a container, and they often have other recommendations that cover this. However, I think it's important since many of the security exploits using containers have nothing to do with docker but rather running vulnerable applications in docker.

Because of this, my advice is to start with CIS but then adapt it for your own environment, remove the advice has little security benefit, and add in recommendations that can help secure the environment for your own use case.
> 1 votes

---
Tags: docker, devsecops
---


# Question
Title: Service backend/frontend in one or two containers?
Imagine in your stack you have RESTful services which also provide some rudimentary frontends, mostly for admin/other tech user use.

Do you include the UI inside the container or are these two containers? Why?

# Answer
Are they separate projects (from a logical or technical perspective)? Would one ever need to be updated/recreated independently of the other one? Would they need to scale independently of each other? I think if any of those answers is a yes, then they should be separated. If not, you should be fine bundling them together. Bundling them together will reduce discrepancy or compatibility issues between the two containers.
> 9 votes

# Answer
We always use different container for each service (Like Frontend ,Backend and Database). Because a container can run only a single service inside it at a time by using docker-compose.yml file. But we can define multiple services inside one docker-compose.yml file but one for each container.
> 2 votes

---
Tags: docker, architecture
---


# Question
Title: Dockerfile parse error line 5: unknown instruction: -ESSENTIAL
this is my dockerfile ;

```
FROM ubuntu:16.04
MAINTAINER James Turnbull "james@example.com"
ENV REFRESHED_AT 2016-06-01
RUN apt-get update -yqq; apt-get -yqq install ruby ruby-dev build
  -essential redis-tools

RUN gem install --no-rdoc --no-ri sinatra json redis
RUN mkdir -p /opt/webapp
EXPOSE 4567

CMD [ "/opt/webapp/bin/webapp" ]

```

and this is how i ran it :

> docker build -t jamtur01/sinatra .

however i get this error in regard to -essential

> Dockerfile parse error line 5: unknown instruction: -ESSENTIAL

i am not clear what i have done wrong. is there an equivalent for -essential

# Answer
> 3 votes
I assume you have a copy-paste issue. The right package name is `build-essential`. And if you want to split a RUN statement over several lines, you will have to use backslash for line continuation.

All that leading to something along the lines of:

```
RUN apt-get update -yqq; apt-get -yqq install ruby ruby-dev \
          build-essential redis-tools

```

As a personal suggestion, when you have several commands to run, it is usually better to join them using `&&` rather than `;` since you want the build to fail if *either* command is failing.

```
RUN apt-get update -yqq && apt-get -yqq install ruby ruby-dev \
          build-essential redis-tools

```

---
Tags: dockerfile
---


# Question
Title: What is the duration for the App Engine task queue tombstones?
Recently I got TombstonedTaskError and I would like to know what is the duration for the TombstonedTaskError?

# Answer
> 1 votes
`TombstonedTaskError` is raised if you're attempting to enqueue a named task using a name that has been used recently, see What is TombstonedTaskError from App Engine's Task Queue?.

The duration of the de-duplication logic has been documented since that answer was posted. From Naming a task (emphasis mine):

> When you create a new task, App Engine assigns the task a unique name by default. However, you can assign your own name to a task by using the name parameter. An advantage of assigning your own task names is that named tasks are de-duplicated, which means you can use task names to guarantee that a task is only added once. **De-duplication continues for 9 days after the task is completed or deleted**.

---
Tags: google-cloud-platform
---


# Question
Title: How do I use docker command's "--format" option?
I can't seem to find any good documentation on how to use the docker command's "--format" option in real life. I know how to use Go templates. But specific to docker, how do I know what placeholders are available to use?

I tried `docker inspect` on a container and I see JSON output, but I can't use the names from that as Go template placeholders. I did find out that the placeholders are case-sensitive, which seems unfortunate. The default output isn't much help either, since (I assume) it isn't showing me everything I might use, the columns that are there are all uppercase, and they sometimes have spaces in them. `docker container ls`, for example, has a column called "CONTAINER ID". No combination of underscores, quoting schemes, or so on will make that name work. I figured out that it's "{{.ID}}".

How am I supposed to be using "--format"? How do I discover placeholders for any given docker object?

# Answer
I take the risk of being downvoted here, but I find Go templates a nightmare to use beyond the most basic use cases. Don't take me wrong: they are extremely powerful as you can guess it by reading the docs, but I find the syntax unfriendly, and for the most complex cases, I prefer parsing the JSON output with `jq`. Anyway:

# Formatting `expect` commands

```
sho$ CID=$(sudo docker run --rm -d busybox sleep 3000)
sh$ sudo docker inspect $CID
[
    {
        "Id": "0470a3549d2f41efd0477cf8fc4eb3a27aca592f089c018ad24704031bbef967",
        "Created": "2019-02-05T16:41:40.040349047Z",
        "Path": "sleep",
        "Args": [
            "3000"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 26467,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2019-02-05T16:41:40.659056612Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:3a093384ac306cbac30b67f1585e12b30ab1a899374dabc3170b9bca246f1444",
        "ResolvConfPath": "/var/lib/docker/containers/0470a3549d2f41efd0477cf8fc4eb3a27aca592f089c018ad24704031bbef967/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/0470a3549d2f41efd0477cf8fc4eb3a27aca592f089c018ad24704031bbef967/hostname",
        "HostsPath": "/var/lib/docker/containers/0470a3549d2f41efd0477cf8fc4eb3a27aca592f089c018ad24704031bbef967/hosts",
        "LogPath": "/var/lib/docker/containers/0470a3549d2f41efd0477cf8fc4eb3a27aca592f089c018ad24704031bbef967/0470a3549d2f41efd0477cf8fc4eb3a27aca592f089c018ad24704031bbef967-json.log",
        "Name": "/vibrant_shaw",
        "RestartCount": 0,
        "Driver": "aufs",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "docker-default",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": null,
            "ContainerIDFile": "",
           "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "default",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "no",
                "MaximumRetryCount": 0
            },
            "AutoRemove": true,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "shareable",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": [],
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DeviceCgroupRules": null,
            "DiskQuota": 0,
            "KernelMemory": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": false,
            "PidsLimit": 0,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/asound",
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": null,
            "Name": "aufs"
        },
        "Mounts": [],
        "Config": {
            "Hostname": "0470a3549d2f",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "sleep",
                "3000"
            ],
            "Image": "busybox",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {}
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "654da638965d9bb26e394ba61036ea713bdccb6304eb09bee86f3c0611a26ca5",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/654da638965d",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "c69ef9cb870a3c477105c7bc19e370683886e4e3c62fac8444d3e1e097551250",
            "Gateway": "172.17.0.1",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "172.17.0.3",
            "IPPrefixLen": 16,
            "IPv6Gateway": "",
            "MacAddress": "02:42:ac:11:00:03",
            "Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "5ec8728b05efd5ce6bfde5cbe6e42f247348345e0fdfb305fda76566327fc151",
                    "EndpointID": "c69ef9cb870a3c477105c7bc19e370683886e4e3c62fac8444d3e1e097551250",
                    "Gateway": "172.17.0.1",
                    "IPAddress": "172.17.0.3",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:11:00:03",
                    "DriverOpts": null
                }
            }
        }
    }
]

```

## Returning scalar properties

You can query any scalar attribute by its name, prefixed by a dot:

```
sh$ sudo docker inspect $CID --format '{{.Id}}
0470a3549d2f41efd0477cf8fc4eb3a27aca592f089c018ad24704031bbef967
sh$ sudo docker inspect $CID --format '{{.Name}} {{.Driver}}'
/vibrant_shaw aufs

# jq notation
sh$ sudo docker inspect $CID | jq '.[].Id'
"0470a3549d2f41efd0477cf8fc4eb3a27aca592f089c018ad24704031bbef967"

sh$ sudo docker inspect $CID | jq '.[]|(.Name,.Driver)'
"/vibrant_shaw"
"aufs"

```

You can also query nested fields:

```
sh$ sudo docker inspect $CID --format '{{.State.Running}}'
true

# jq notation
sudo docker inspect $CID | jq '.[].State.Running'
true

```

## Returning entire objects and arrays

But it does not work as expected for nested structures:

```
sh$ sudo docker inspect $CID --format '{{.State}}'
{running true false false false false 26467 0  2019-02-05T16:41:40.659056612Z 0001-01-01T00:00:00Z <nil>}

```

You end up having the data, but without the field names, it is pretty useless--unless you are assuming some predefined field order (something that is probably not safe).

When accessing entire objects, there are chances you want an explicit formatter:

```
sh$ sudo docker inspect $CID --format '{{json .State}}'
{"Status":"running","Running":true,"Paused":false,"Restarting":false,"OOMKilled":false,"Dead":false,"Pid":26467,"ExitCode":0,"Error":"","StartedAt":"2019-02-05T16:41:40.659056612Z","FinishedAt":"0001-01-01T00:00:00Z"}

# jq notation
sh$ sudo docker inspect $CID | jq '.[].State'
{
  "Status": "running",
  "Running": true,
  "Paused": false,
  "Restarting": false,
  "OOMKilled": false,
  "Dead": false,
  "Pid": 26467,
  "ExitCode": 0,
  "Error": "",
  "StartedAt": "2019-02-05T16:41:40.659056612Z",
  "FinishedAt": "0001-01-01T00:00:00Z"
}

```

Same thing for arrays:

```
sh$ sudo docker inspect $CID --format '{{.HostConfig.ReadonlyPaths}}'
[/proc/asound /proc/bus /proc/fs /proc/irq /proc/sys /proc/sysrq-trigger]

sh$ sudo docker inspect $CID --format '{{json .HostConfig.ReadonlyPaths}}'
["/proc/asound","/proc/bus","/proc/fs","/proc/irq","/proc/sys","/proc/sysrq-trigger"]

sh$ sudo docker inspect $CID --format '{{join .HostConfig.ReadonlyPaths ":"}}'
/proc/asound:/proc/bus:/proc/fs:/proc/irq:/proc/sys:/proc/sysrq-trigger

# jq notation
sh$ sudo docker inspect $CID | jq '.[].HostConfig.ReadonlyPaths'
[
  "/proc/asound",
  "/proc/bus",
  "/proc/fs",
  "/proc/irq",
  "/proc/sys",
  "/proc/sysrq-trigger"
]
sh$ sudo docker inspect $CID | jq '.[].HostConfig.ReadonlyPaths | join(":")'
"/proc/asound:/proc/bus:/proc/fs:/proc/irq:/proc/sys:/proc/sysrq-trigger"

```

## Arrays of objects

More complicated (according to my limited Go Template skills) is accessing fields nested in an array of objects:

```
# This works as expected
sh$ sudo docker network inspect bridge --format '{{json .IPAM.Config}}'
[{"Subnet":"172.17.0.0/16","Gateway":"172.17.0.1"}]

# But not that
sudo docker network inspect bridge --format '{{.IPAM.Config.Gateway}}'

Template parsing error: template: :1:7: executing "" at <.IPAM.Config.Gateway>: can't evaluate field Gateway in type interface {}

```

When I want to extract such nested value, I resort on an explicit range loop:

```
sh$ sudo docker network inspect bridge --format '{{range .IPAM.Config}}{{.Gateway}}{{end}}'
172.17.0.1

# jq notation
sh$ sudo docker network inspect bridge | jq '.[].IPAM.Config[].Gateway'
"172.17.0.1"

sh$ sudo docker network inspect bridge | jq -r '.[].IPAM.Config[].Gateway'
172.17.0.1

```

# Formatting `ls` commands

With the `json` formatter, we can obtain (some? all?) the possible selectors you can later use with `--format`:

```
sh$ sudo docker network ls --format '{{json .}}'
{"CreatedAt":"2019-01-07 14:28:55.63238018 +0100 CET","Driver":"bridge","ID":"5ec8728b05ef","IPv6":"false","Internal":"false","Labels":"","Name":"bridge","Scope":"local"}
{"CreatedAt":"2019-02-05 15:36:57.701603791 +0100 CET","Driver":"bridge","ID":"45d6f09ca923","IPv6":"false","Internal":"false","Labels":"com.docker.compose.network=default,com.docker.compose.project=compose-demo,com.docker.compose.version=1.23.2","Name":"compose-demo_default","Scope":"local"}
{"CreatedAt":"2019-02-05 17:30:47.98883918 +0100 CET","Driver":"bridge","ID":"aaff418d1c56","IPv6":"false","Internal":"false","Labels":"com.docker.compose.network=net,com.docker.compose.project=compose-demo,com.docker.compose.version=1.23.2","Name":"compose-demo_net","Scope":"local"}
{"CreatedAt":"2018-05-26 10:31:48.149819089 +0200 CEST","Driver":"host","ID":"988954899b53","IPv6":"false","Internal":"false","Labels":"","Name":"host","Scope":"local"}
{"CreatedAt":"2018-05-26 10:31:48.118573602 +0200 CEST","Driver":"null","ID":"ce1680ed377f","IPv6":"false","Internal":"false","Labels":"","Name":"none","Scope":"local"}
{"CreatedAt":"2019-02-05 19:44:21.286955573 +0100 CET","Driver":"bridge","ID":"41c62c023d16","IPv6":"false","Internal":"false","Labels":"com.docker.compose.version=1.23.2,com.docker.compose.network=default,com.docker.compose.project=test","Name":"test_default","Scope":"local"}

```

Here, you can see `docker network ls` exposes, among others, the "ID", "Driver" and "Scope" fields. You can use them in your format:

```
sh$ $ sudo docker network ls --format '{{.ID}}: {{.Driver}} ({{.Scope}})'
5ec8728b05ef: bridge (local)
45d6f09ca923: bridge (local)
aaff418d1c56: bridge (local)
988954899b53: host (local)
ce1680ed377f: null (local)
41c62c023d16: bridge (local)

```

# The best of both worlds

The `docker ... ls` commands only exports a small subset of the docker object's properties. Depending your needs, you may prefer something along the lines of `docker ... inspect $(docker ... ls -q)`.

For example, `docker image ls` does not export the image architecture. but you can obtain it with `docker image inspect`. If you want that information, you could use something like that:

```
sh$ sudo docker image inspect \
         --format '{{.Id}}: {{.RepoTags}} ({{.Architecture}})' \
         $(sudo docker image ls -q)
sha256:fc37804350a8adea0b41dba0c413a6c967140ca2ed6dfeefdbe88d7868902406: [compose-demo_monitor:latest] (amd64)
sha256:a6f30c0890bca94e3cf5930ad5c3358b06cb38450e92d4ddabeb208ee34383f8: [compose-demo_app:latest] (amd64)
sha256:a79892b33e0bb51598e9311cc1d23ca8d0e3013a5a15fd67d202d22ffd08a6af: [] (amd64)
sha256:394c82635f220db389b8532e316001674965b51679dbc8dff747d5f224e6d568: [] (amd64)
sha256:95f0c755feb65af441c0cb8727bb13b35517977a3bed06e4a3456c95d6a69dd4: [redis:5.0-alpine] (amd64)
sha256:1f6c34f7921c7b50ef4452737399b5cff0eb834fc183d188d82886fd23b7fc34: [node:8] (amd64)
sha256:3a093384ac306cbac30b67f1585e12b30ab1a899374dabc3170b9bca246f1444: [busybox:latest] (amd64)

```
> 30 votes

---
Tags: docker, go
---


# Question
Title: Should development tools be installed on the host or in an environment standardized at project level?
On a development host, do you consider it still worth installing the development tools (compiler, interpreter, package manager, testing tools, ...) on the host? Or are the developers supposed to interact with those tools through an environment1 standardized at project level to ensure a consistent development stack across the team and with the production environment?

1local virtual environment, chroot, container, remote login on a compile farm, ...
Note: somewhat related to a previous question

# Answer
At many companies (eg banks) the provenance of the code that gets out to live needs to be known and traceable. This is typically done by having a controlled build farm that builds from source. Release builds upload the binary artefacts to a location where they cannot be overwritten. The artefacts are then deployed by any number of mechanisms that are traceable to an individual and an approved change request. This model is generally just referred to as a ”pipeline”.

Under such a model a ”dev host” or ”dev environment” can have anything and everything that developers want on it. It can use newer versions of software than are available in production. It's entirely down to the context whether that is going to cause problems with code failing testing in a live-like environment. It's typical to put release build code through at least one test environment before pushing it to live. That environment is kept as similar to the live environment as possible.
> 3 votes

# Answer
I'd say whatever works for your context.

It's not always possible to have a standardized environment (as in container/VM solutions) at the project level which can be used effectively regardless of the host type/state. For example we had some massive builds for which we used finely tuned bare-metal machines for performance reasons - ~40% faster builds than with the most performant environment the ops teams could offer). So we used hosts in this case.

But nothing stops you from having host configurations standardized at the project level (which is what we had). Or, better yet, standardized environments running on standard-configured hosts - which *might* allow you to achieve higher optimisation levels by using existing host resources instead of building them from scratch in each environment - if that's what you're really after.
> 1 votes

---
Tags: virtualenv
---


# Question
Title: Very slow network when serve container content via Nginx after published Docker port
What I have created is:

I have a Docker container that is published the port "3000:3000".

If I'm accessing the container in this way: http://example.com:3000 everything is working great.

The problem appears when I install on example.com server (CentOS7) Nginx. I configured Nginx in this way:

```
server {
    listen 80;

    root /opt/;
    index index.php index.html index.htm;

    server_name example.com;

    location / {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Protocol $scheme;
        proxy_pass http://localhost:3000;
        proxy_set_header X-Host $http_host;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
    }
}

```

Now, if I want to access my container via http://example.com (without to tell the port 3000) the network/connection/speed is incredible slow.

Had someone this issue before? What can I do to solve the issue?

In Nginx I'm seeing errors like this:

```
 2019/02/04 11:53:18 [error] 8667#0: *289 upstream timed out (110: Connection timed out) while reading response header from upstream, client:

```

Regards,

> EDIT: I want to specify that the app is in replicas mode and I have 3 replicas (and 3 servers; one replica/app with port 3000 published for each server).

# Answer
> 1 votes
I would suggest to use Ubuntu form docker hub and try Ubuntu alpine image.

there should be also problem form database if your database is outside the K8s cluster so may possible latency problem should be there

---
Tags: docker, nginx
---


# Question
Title: How to restrict only one parameter in jenkins pipeline
I have the below pipeline script with string parameters. The Target parameter will fail if multiple comma separated inputs are provided in Jenkins. How can I restrict the Jenkins pipeline to accept just one value as parameter and not multiple comma separated values.

```
properties([
parameters([
    string(defaultValue: '', description: '', name: 'ID'),
    string(defaultValue: '', description: '', name: 'Target')
    ])
])

```

# Answer
Extended Choice Parameter plugin is the way to go for such requirement. You need to select Extended Choice Parameter from the drop-down list as shown below:

enter image description here

In Name text-box, assign a name. For example, Environment. This is the name with which you will be accessing all the values (dev1,dev2,...) that you will select while triggering the build. Now, in Simple Parameter Types section, you will see another drop-down with the name Parameter Type. Select Multi Select from that drop-down. Now, in Choose Source for Value section, enter the values (dev1,dev2,qa1,qa2,...) in Value box. Comma (,) is the default delimiter. Refer screenshot below:

enter image description here

Once you are done with the above settings, you will then have to access the selected option in your script (using variable assigned to Name as described above) and decide the course of action.
> 1 votes

---
Tags: jenkins-pipeline
---


# Question
Title: Does AWS Amazon Certificate Manager Generate Client certificates for Customer (3rd Party Clients)
Could you kindly help me out with an answer.

Does AWS Amazon Certificate Manager (ACM CA) (or maybe LetsEncrypt) generate Client certificates for Customer (3rd Party Clients), i.e. the client certificates will be deployed on my third-party customers' laptops? They are staff of third-party organisations; and I need to do mutual client certificate-based TLS/SSL authentication between their laptop/PC browsers and my web-site or service, which will be AWS-hosted.

If so are are there any associated costs or limitations. Next are there any AWS, LetsEncrypt or other tools to support the Certificate deployment, key rotation on Certificate Expiry, etc.

Thanks, JoeBoy

# Answer
> 0 votes
Yes you can do it using let's encrypt while with AWS cert-manager it is also possible. if you have generated certificate locally you can also manually upload it to AWS cert manager.

for more info you can check this out : https://community.letsencrypt.org/t/certificate-for-a-third-party-web-host-using-csr/76805/8

---
Tags: amazon-web-services
---