# Question
Title: Best Practices for a multi environment deployment using Jenkins
I have 3 environments, each on their own Virtual Network with their own configurations. Do I need to have 3 separate instances of Jenkins if I am to do Continuous Deployments on each of the environments? What are some best practices regrading deploying on a multi environment architecture?

# Answer
This is a very good question as it is a common anti-pattern to couple `ci/cd` tools with environments.

Jenkins is a build factory. As such, it should be totally agnostic of the notion of environment, or even delivery.

The best practice is to have some sort of staging process and / or interface (if you can afford it: a dedicated delivery software).

### Staging process

You should try to create some sort of staging jobs for each environment, in which you'll define the configuration of the environment and bundle the final package with it.

Next you'll have delivery jobs for each environment.

### Delivery tools

In order to do create the staging and delivery jobs, you may use very basic scripting tools, like `shell`, but there are more adapted scripting tools, like `Ansible`, `puppet`, `chef`...

If you can afford the expense, you may consider investing in a deployment software (I'll mention some that I worked with in the comments section).

Notice that those softwares manage some kind of environment staging process.

Obviously, it makes a lot of sense to combine deploy softwares and scripting.

### Beyond segregating tools and environment

Since you asked for best practices, it appears to me that it is worth mentioning another commonly found anti-pattern: coupling SCM tools and delivery process.

It is a very **good** practice to store environment configuration (not passwords or confidential information, of course) and go live scripts in SCM tools (like `svn` or `git`...). It is however a very **bad** practice to check-out environment configuration and go live scripts DURING the go live. It might simply be unavailable when you need it.

This check-out phase should be part of the staging process that I mentioned before.

### Idempotence

Another best practice is that your scripts should be `idempotent`: this means that you should be able to play your scripts once to perform the staging and delivery. Then you should be able to play them again and again, and they would not change the state of your system unless something chacked in the configuration.

### Scaling

As a final best practice I shall share from direct experience is scaling Jenkins: different teams have different needs and uses of Jenkins. When too many teams share a common Jenkins, there might be issues with resources. The worst case is when a team requires to restart the Jenkins master while another needs to deliver.

The ideal would be to have one Jenkins per team or per group of teams that share the same goal and delivery planning.
> 10 votes

# Answer
I've just transitioned away from having Jenkins freestyle jobs and a separate job for each environment to using Jenkins pipelines (declarative syntax).

In Jenkins pipelines we have a single file (`Jenkinsfile`) and define different stages for our different environments (Dev/Staging/Prod) and in those stages we just define different behaviour for each environment. For example, our 'Deploy' stage we will deploy to a different kubernetes namespace/cluster depending on if it's the development or staging environment.

I like this approach as you only have one `Jenkinsfile` per project that exists on all branches.
> 3 votes

# Answer
The short answer to your question is no. Jenkins has no need to talk to your production environments but it can if you need it to. In other words, if Jenkins can reach these environments, it can likely interact with them.

From a best practices approach, typically I recommend using software that will make your deploy as simple and repeatable as possible (e.g. make, Ansible, Terraform, Cloud Formation or Kubernetes).

Pipelines are capable of handling parallel targets meaning you can build and deploy to all three environments at the same time if you choose to.

You might want to make your question more precise or detailed in order to garner more feedback.
> 2 votes

# Answer
We can use single Jenkinsfile for different environment. But we need something to differentiate between this environment, may be an environment variable or a parameter in a parameterised job to control what we need to do in each environment. May be we want to do build, test, perform e2e and push image into a repository in one environment. Then use the same image and deploy in the next ( staging, may be) and perform smoke test. Then use the same image to deploy into production.
> 0 votes

---
Tags: jenkins
---


# Question
Title: How to introduce(join) new worker node to an existing k8s cluster with selective scheduling only
Given a working Kubernetes cluster consisting of a master and some workers.

We need to add a node to run a very specific pod and be part of the cluster for networking reasons. Then being largely ignored for later pod creation by the master.

Adding selectors to every deployment to avoid this node is out of question.

Thanks!

# Answer
> 2 votes
Possibly you are looking for this feature which is closed with no action since there are ways to work around this problem.

> kubeadm already allows to set taints for the joining node using the configuration file at join time (***check my other answer for details on how to do that***)

One Other way to achieve this is by running join as usual but taking few extra steps as below after join

> 1) After successful join , **Force Drain the new node** to make sure it removes unwanted pod which might have scaled to it post join operation `kubectl drain <node name> --force`
>
> 2) **Drain command will auto cordon** off the node. (Note : also **consider special handling to remove any daemonset** which might have scaled to this node after join)
>
> 3) Now **Taint the new node** as needed **and then uncordon** it
>
> 4) **Add nodeselector plus tolerance to pods** you want to be scheduled on this new node.

Logs of above steps for reference are listed below.

On below example i have joined a new node-02 and assume that some pod were scaled in an existing deployment so new node got used immediately on join, so before adding we need drain action and then taint the node so going forward no scheduling can be done unless pod has tolerance to the taint added

**Nodes Before Drain or Taint**

```
$ kubectl get all -o wide
NAME                         READY   STATUS    RESTARTS   AGE   IP               NODE                NOMINATED NODE   READINESS GATES
pod/nginx-86c57db685-8cdkx   1/1     Running   0          27s   192.168.58.120   k8s-node02-calico   <none>           <none>
pod/nginx-86c57db685-xnh6q   1/1     Running   0          65s   192.168.182.78   k8s-node01-calico   <none>           <none>

NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE   SELECTOR
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   2d    <none>

NAME                    READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES   SELECTOR
deployment.apps/nginx   2/2     2            2           65s   nginx        nginx    app=nginx

NAME                               DESIRED   CURRENT   READY   AGE   CONTAINERS   IMAGES   SELECTOR
replicaset.apps/nginx-86c57db685   2         2         2       65s   nginx        nginx    app=nginx,pod-template-hash=86c57db685

```

**Drain The node**

```
$ kubectl drain k8s-node02-calico
node/k8s-node02-calico cordoned
evicting pod "nginx-86c57db685-8cdkx"
pod/nginx-86c57db685-8cdkx evicted
node/k8s-node02-calico evicted

$ kubectl get nodes
NAME                STATUS                     ROLES    AGE   VERSION
k8s-master-calico   Ready                      master   2d    v1.17.2
k8s-node01-calico   Ready                      <none>   2d    v1.17.2
k8s-node02-calico   Ready,SchedulingDisabled   <none>   2d    v1.17.2

```

**Taint the new node**

```
$ kubectl taint node k8s-node02-calico newnode=nobodyallowed:NoSchedule
node/k8s-node02-calico tainted

```

**Now we are good to uncordon the node, no more pods should be able to use this noe unless they add toleration to the new taint**.

```
$ kubectl uncordon k8s-node02-calico
node/k8s-node02-calico uncordoned

$ kubectl get nodes
NAME                STATUS   ROLES    AGE   VERSION
k8s-master-calico   Ready    master   2d    v1.17.2
k8s-node01-calico   Ready    <none>   2d    v1.17.2
k8s-node02-calico   Ready    <none>   2d    v1.17.2

```

**We test by scaling the deployment and nothign should be added on node02**

```
$ kubectl scale deployment --replicas=3 nginx
deployment.apps/nginx scaled

$ kubectl get all -o wide
NAME                         READY   STATUS    RESTARTS   AGE     IP               NODE                NOMINATED NODE   READINESS GATES
pod/nginx-86c57db685-vm8ql   1/1     Running   0          3m30s   192.168.182.75   k8s-node01-calico   <none>           <none>
pod/nginx-86c57db685-xnh6q   1/1     Running   0          10m     192.168.182.78   k8s-node01-calico   <none>           <none>
pod/nginx-86c57db685-zh2sj   1/1     Running   0          8m19s   192.168.182.76   k8s-node01-calico   <none>           <none>

NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE   SELECTOR
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   2d    <none>

NAME                    READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES   SELECTOR
deployment.apps/nginx   3/3     3            3           10m   nginx        nginx    app=nginx

NAME                               DESIRED   CURRENT   READY   AGE   CONTAINERS   IMAGES   SELECTOR
replicaset.apps/nginx-86c57db685   3         3         3       10m   nginx        nginx    app=nginx,pod-template-hash=86c57db685

```

# Answer
> 1 votes
> **kubeadm already allows to set taints for the joining node using the configuration file at join time**.

**Steps on how to do this at node join time are as below**

Construct a **Config.yaml file with taint details and bootstrap details** as recived from master at kubeadm init time.

**Example config.yaml** you should update values as per your cluster info on below

```
apiVersion: kubeadm.k8s.io/v1beta1
kind: JoinConfiguration
discovery:
  bootstrapToken:
    apiServerEndpoint: "<control_plain_ip>:6443"
    token: "your_token"
    caCertHashes:
    - sha256:$CERT_HASH
nodeRegistration:
  taints:
  - effect: NoSchedule
    key: specialnode1.kubernetes.io/SpecialNode1

```

**Pass this config from node you want to join** it will join with a taint set at startup.

```
root@k8s-node01:# kubeadm join --config=config.yaml

[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

```

**Node should join the cluster and will have a taint at startup time so pods not having tolerance will not be scheduled on this new node**.

```
ubuntu@k8s-master:~$ kubectl get nodes
NAME         STATUS   ROLES    AGE     VERSION
k8s-master   Ready    master   18m     v1.17.2
k8s-node01   Ready    <none>   8m11s   v1.17.2

```

**Verify the taint as below**

```
$ kubectl describe node k8s-node01 | grep -i Taint

Taints:             specialnode1.kubernetes.io/SpecialNode1:NoSchedule

```

**To further test this we will create a daemon set which will run on node01 and add another node02 with new taint , daemon set should not autoscale to new node with new taint**.

```
ubuntu@k8s-master:~$ kubectl get nodes
NAME         STATUS   ROLES    AGE   VERSION
k8s-master   Ready    master   24m   v1.17.2
k8s-node01   Ready    <none>   14m   v1.17.2

ubuntu@k8s-master:~$ kubectl get all -o wide
NAME              READY   STATUS    RESTARTS   AGE   IP               NODE         NOMINATED NODE   READINESS GATES
pod/nginx-npmz5   1/1     Running   0          50s   192.168.85.193   k8s-node01   <none>           <none>

NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE   SELECTOR
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   24m   <none>

NAME                   DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE   CONTAINERS   IMAGES   SELECTOR
daemonset.apps/nginx   1         1         1       1            1           <none>          50s   nginx        nginx    app=nginx

```

**Add new Node with new config.yaml and verify the node is ready and taint is set on new node added to cluster**

```
apiVersion: kubeadm.k8s.io/v1beta1
    kind: JoinConfiguration
    discovery:
      bootstrapToken:
        apiServerEndpoint: "<control_plain_ip>:6443"
        token: "your_token"
        caCertHashes:
        - sha256:$CERT_HASH
    nodeRegistration:
      taints:
      - effect: NoSchedule
        key: specialnode2.kubernetes.io/SpecialNode2

$ kubectl get nodes
NAME         STATUS   ROLES    AGE   VERSION
k8s-master   Ready    master   25m   v1.17.2
k8s-node01   Ready    <none>   15m   v1.17.2
k8s-node02   Ready    <none>   22s   v1.17.2

```

**Note the new taint on node02**

```
$ kubectl describe node k8s-node02 | grep -i Taint
Taints:             specialnode2.kubernetes.io/SpecialNode2:NoSchedule

```

**Now check if daemon set has scaled to new node or not, it will not unless we remove the taint or add tolerance to daemonset**

```
$ kubectl get all -o wide
NAME              READY   STATUS    RESTARTS   AGE     IP               NODE         NOMINATED NODE   READINESS GATES
pod/nginx-npmz5   1/1     Running   0          3m46s   192.168.85.193   k8s-node01   <none>           <none>

NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE   SELECTOR
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   27m   <none>

NAME                   DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE     CONTAINERS   IMAGES   SELECTOR
daemonset.apps/nginx   1         1         1       1            1           <none>          3m46s   nginx        nginx    app=nginx

```

> **So we can use above steps to add a new node to a running cluster and choose what is scheduled on it using taints**.

---
Tags: kubernetes, kubeadm
---


# Question
Title: How to manage images versions via docker-compose with Ansible
I planned to use *docker-compose* and a few scripts to manage the deployment of my app on staging and production. My idea was to use one `docker-compose.yml` with one `.env` file for each environment. In those `.env` files I would have set some specific values and the version of the different images that compose my application.

But then I discovered *Ansible* and it looks like my few scripts powered by a community of experts. So I started to investigate it and I would like to use it.

Based on the best practices, I can easily imagine having a standard directory layout\[1\] with the one inventory for each environment. And a couple of roles.

However, I have no idea how to manage the versions; how to describe which version is executed in one environment?

Ideally, I would like to find a simple solution where my CI can update this version.

Thanks

\[1\] https://docs.ansible.com/ansible/latest/user\_guide/playbooks\_best\_practices.html#directory-layout

# Answer
> 1 votes
The best way is of course (like Zeitounator comments) to use a inventory var. Either you use the version number in the docker-compose task as a variable or you create a Jinja2 template docker-compose.yml.j2 and use the variable there. That's what I do.

Example:

```
all:
  hosts:
    host1.prod:
      image: "xyz/12"
    host2.dev:
      image: "xyz/latest"

```

And the `docker-compose.yml.j2` contains the variable:

```
version: '2.1'

services:
  myservice:
    image: "{{ image }}"
    ...

```

---
Tags: ansible, docker-compose, environments, versioning
---


# Question
Title: How to measure Lead Time for a Scrum Team
I'm currently revisiting Accelerate by Nicole Forsgren and I'm looking at her metrics for measuring high performance. Specifically Lead Time which she defines as:

> Lead Time is the time it takes from the customer making a request to the request being satisfied.

She also notes that

> there are two parts to the lead time: the time it takes to design and validate a product or feature, and the time to deliver the feature to customers.

However, in a normal scrum team stories are added and prioritised to the backlog all the time. Something may be added and then deprioritized or even cancelled. when exactly does a customer "make a request?"

Is it:

* When it's added to the backlog?
* When it's accepted in Sprint planning?
* When it's "accepted" and estimated?
* Somewhere in between?

I'm less worried about "delivered" that feeds into definition of done.

Any "word of god" statements from Forsgren herself would be preferred.

# Answer
The idea of lead time is that when the customer asks for something, a clock starts ticking in their head and that clock stops when they have it in their hands. Therefor, a feature's lead time would be from when the request is received (this usually corresponds to adding it to the backlog) until it is delivered and usable to the requestor.

Full disclosure, I haven't read Accelerate, but since Forsgren is using an existing term and it seems it's being used correctly, I am referring to the proper definition of the term.

Now, that isn't to say all those other things you mention aren't valuable. They may be. They may even be more valuable to you than lead time. They just aren't lead time.
> 3 votes

---
Tags: agile
---


# Question
Title: Why are valid directives in my Dockerfile apparently being ignored?
Using the following Dockerfile:

```
FROM ubuntu:latest

USER root

RUN apt-get -y update \
    && apt-get -y upgrade \
    && apt-get install -y \
        python-pip \
        nmap \
        vim-tiny

FROM python:2.7

USER root

RUN pip install --no-cache-dir cassandra-driver

FROM cassandra:3.11.4

WORKDIR /cassandra

COPY ./cassandra.yaml /etc/cassandra/cassandra.yaml

EXPOSE 7000 7001 7199 9042 9171

CMD ["cassandra", "-f"]

```

and then running the commands:

```
docker system prune -a -f

docker-compose -f docker-compose.yml -f dev-docker-compose.yml up --build

```

when I then enter the cassandra container via the command:

```
docker exec -ti cassandra_app /bin/bash

```

None of the things I asked for has been installed!

```
root@2f1776a10084:/cassandra# nmap
bash: nmap: command not found

root@2f1776a10084:/cassandra# vi
bash: vi: command not found

root@2f1776a10084:/cassandra# python --version
Python 2.7.13

root@2f1776a10084:/cassandra# python -c "import cassandra"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named cassandra

```

I also tried running the build with the "USER root" lines removed, but no difference.

I need a Cassandra container that includes "vi" and the python cassandra driver, and this wretched Docker is wasting hours and hours of my time trundling through its builds (which don't have any relevant error messages that I can see).

It has gradually become clear that Docker caches things when it can, hence the "-a -f" arguments to the "docker prune" command - I want to be sure I'm doing a clean build each time.

In case it is relevant, following are the Docker YAML files for the above build command:

docker\_compose.yml :

```
version: '3.3'

services:

  application:
#    image: appweb/app:python3
    container_name: django_app
    build:
      context: ./application
    ports:
      - "8000"
     depends_on:
      - postgres
      - cassandra
      - cassandra-node
    networks:
      - appnet
    environment:
      - DOCKERISED=1
      - CASS_HOST1=cassandra:9042
      - CASS_HOST2=cassandra-node:9042
    extra_hosts:
      - "34.253.98.48:34.253.98.47"

  postgres:
    image: postgres:9.6.10
    container_name: postgres_app
    networks:
      - appnet

  cassandra:
    build:
      dockerfile: Dockerfile
      context: ./cassandra
    container_name: cassandra_app
    environment:
      - "CASSANDRA_SEEDS=cassandra"
      - "CASSANDRA_CLUSTER_NAME=App2 Cluster"
    networks:
      - appnet

  cassandra-node:
    build:
      dockerfile: Dockerfile
      context: ./cassandra
    container_name: cassandra_app2
    #command: /bin/bash -c "echo 'Waiting for seed node' && sleep 30 && /docker-entrypoint.sh cassandra -f"
    environment:
      - "CASSANDRA_SEEDS=cassandra"
      - "CASSANDRA_CLUSTER_NAME=APp2 Cluster"
    depends_on:
      - "cassandra"
    networks:
      - appnet

networks:   appnet:

```

dev-docker-compose.yml :

```
version: '3.3'

services:

  application:
    environment:
      - CASS_USER
      - CASS_PASS
      - POSTGRES_USER
      - POSTGRES_PASSWORD
      - TESTMODE=1
      - CQLENG_ALLOW_SCHEMA_MANAGEMENT="1"
    volumes:
      - ./application/source:/application/source
      - ./application/source/private_settings.py:/application/source/app2-config/app2/config/private_settings.py

  postgres:
    volumes:
      - postgres_data:/var/lib/postgresql

  cassandra:
    volumes:
       - cassandra_data:/var/lib/cassandra

  cassandra-node:
    volumes:
       - cassandra_data2:/var/lib/cassandra

volumes:   postgres_data:   cassandra_data:   cassandra_data2:

```

Any ideas? Thanks in anticipation

# Answer
It looks like your Dockerfile is attempting a multistage build:

```
FROM ubuntu:latest
...
FROM python:2.7
...

```

This is a feature that enables you to build parts of your image in a *separate* build container and then import what you need. Unless you import stuff into your end container, it will be lost in the build process; this is exactly what is happening with your `RUN apt-get...` statement. To fix it, try removing the extra stages of the build, (`FROM ubuntu:latest` and `FROM python:2.7`) and moving your commands to after `FROM cassandra:3.11.4` like so:

```
FROM cassandra:3.11.4

USER root

RUN apt-get -y update \
    && apt-get -y upgrade \
    && apt-get install -y \
        python-pip \
        nmap \
        vim-tiny

RUN pip install --no-cache-dir cassandra-driver
...

```

I'm sorry you're having a hard time with Docker. Hopefully this helps and it starts making more sense. Good luck, friend!
> 1 votes

---
Tags: cassandra
---


# Question
Title: Terraform 0.12 - For each subnet-id in array, get cidr_block
I have a list of subnets IDs and I want to get an array of the cidr\_blocks associated to each subnet.

```
variable "aws_subnet_ids" {
  type = list(string)
  default = ["subnet-aaaaaaaa","subnet-bbbbbbbb"]
}

desired_output = ['10.0.1.0/24', '10.0.2.0/24']

```

I've been fiddling around with for loops and count data for a quite a while for what should be a really simple problem. But I'm having a hard time finding an appropriate way to actually do this.

This seemed like a decent example to work off of, but you can't actually use a for\_each in a data source off a list of strings. https://www.terraform.io/docs/providers/aws/d/subnet\_ids.html

```
data "aws_subnet" "example" {
  for_each = var.aws_subnet_ids
  cidr_blocks       = each.cidr_block
}

output "subnet_cidr_blocks" {
  value = [for s in data.aws_subnet.example : s.cidr_block]
}

```

# Answer
```
data "aws_subnet" "subnets" {
  for_each = toset(var.aws_subnet_ids)
  id       = each.value
}

locals {
  cidr_blocks = [
    for subnet in data.aws_subnet.subnets :
      subnet.cidr_block
    ]

  cidr_blocks_rendered = [
    for cidr_block in local.cidr_blocks :
     {
       from_port   = 6443,
       to_port     = 6443,
       protocol    = "tcp",
       cidr_blocks = cidr_block
     }
  ]
}

output "output1-1" {
  value = local.cidr_blocks_rendered
}

```

This is what ultimately I needed.
> 3 votes

---
Tags: terraform
---


# Question
Title: Using a ubuntu base image vs an application specific image (node, python, etc)
What are the benefits of using an application specific Docker image vs a generic OS? I'll use an angular app here as a contextual example, but this should apply to other apps as well.

Example 1 | Use application specific images:

```
FROM node as build
RUN npm install
COPY package-lock.json
RUN ng build --prod

FROM nginx
# ... move files from build to nginx/html
EXPOSE port

```

Example 2 | Use a linux base image:

```
FROM ubuntu
RUN apt-get install npm
RUN apt-get install nginx
COPY . .
RUN npm install
# ... build Angular app and mv files to nginx/html

```

Example 3 | Build angular app outside of docker first and:

```
FROM nginx
COPY . .
# mv files to nginx/html
EXPOSE port

```

What are the advantages / disadvantages to doing it one of these 3 ways? Are there image size differences? Does the orchestration environment (kubernetes) need access to the base images or is everything needed included when it's pushed? Which way is considered best practice?

# Answer
> 2 votes
They're all functionally equivalent. When you `docker push` or `docker pull` an image (or mention it in a Kubernetes manifest), all of its layers are pushed or pulled, so you always get a complete image with all of its dependencies at the point where it was built. (It turns out the versioned `ubuntu` images are re-released pretty routinely to include current security patches.)

Building your application outside of Docker was pretty common before there were multi-stage builds. There's nothing wrong with doing it, if it makes sense. It makes more sense if your built application artifact is portable (in your example, Webpack JavaScript/HTML/CSS artifacts; Java `.jar` files work well here too). This is also pretty easy to plug into an existing CI system that produces the artifacts, and then you just need to wrap them up in an image.

Using prebuilt images like `nginx` or `node`, or building your own on top of `ubuntu` or `alpine`, is mostly a matter of taste and organizational requirements. Building an image `FROM node` will be easier than building your own Node image and can get you running faster; if you start `FROM ubuntu` then you know exactly who built it and what security updates and packages are in it.

If you're pinning to specific versions of software, the other important thing to be aware of is that the Docker Hub packages eventually stop getting updates. If you're using `node:8` that image isn't being automatically built any more; if you pin to a more specific version, `node:12.13` or `node:12.14.0` aren't getting updates either. (As of this writing the Docker Hub `node` page lists `12.14.1` as the current LTS build.) That could be an issue if you're forced to take a language runtime update to get an OS-level security fix.

---
Tags: docker, kubernetes
---


# Question
Title: Jenkins-Deployment failure, Build successful
I have been trying to work on Jenkins and I'm able to Build the code but unable to deploy on to the Tomcat server. Have checked the users.xml file and

```
Started by user nakul
Running as SYSTEM
Building in workspace /home/ubuntu/.jenkins/workspace/Development
using credential 1da4e373-a6ab-4c02-8231-6c3040786f0b
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/nakul-crypto/maven.git # timeout=10
Fetching upstream changes from https://github.com/nakul-crypto/maven.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials
 > git fetch --tags --progress -- https://github.com/nakul-crypto/maven.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 7599ae0e083b03eb84d05b6606b3dc0df36b45f8 (refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 7599ae0e083b03eb84d05b6606b3dc0df36b45f8 # timeout=10
Commit message: "maven"
 > git rev-list --no-walk 7599ae0e083b03eb84d05b6606b3dc0df36b45f8 # timeout=10
[Development] $ mvn package
[[1;34mINFO[m] Scanning for projects...
[[1;33mWARNING[m]
[[1;33mWARNING[m] Some problems were encountered while building the effective model for com.example.maven-project:server:jar:1.0-SNAPSHOT
[[1;33mWARNING[m] Reporting configuration should be done in <reporting> section, not in maven-site-plugin <configuration> as reportPlugins parameter.
[[1;33mWARNING[m]
[[1;33mWARNING[m] Some problems were encountered while building the effective model for com.example.maven-project:webapp:war:1.0-SNAPSHOT
[[1;33mWARNING[m] Reporting configuration should be done in <reporting> section, not in maven-site-plugin <configuration> as reportPlugins parameter.
[[1;33mWARNING[m]
[[1;33mWARNING[m] Some problems were encountered while building the effective model for com.example.maven-project:maven-project:pom:1.0-SNAPSHOT
[[1;33mWARNING[m] Reporting configuration should be done in <reporting> section, not in maven-site-plugin <configuration> as reportPlugins parameter. @ line 56, column 24
[[1;33mWARNING[m]
[[1;33mWARNING[m] It is highly recommended to fix these problems because they threaten the stability of your build.
[[1;33mWARNING[m]
[[1;33mWARNING[m] For this reason, future Maven versions might no longer support building such malformed projects.
[[1;33mWARNING[m]
[[1;33mWARNING[m] The project com.example.maven-project:maven-project:pom:1.0-SNAPSHOT uses prerequisites which is only intended for maven-plugin projects but not for non maven-plugin projects. For such purposes you should use the maven-enforcer-plugin. See https://maven.apache.org/enforcer/enforcer-rules/requireMavenVersion.html
[[1;34mINFO[m] [1m------------------------------------------------------------------------[m
[[1;34mINFO[m] [1mReactor Build Order:[m
[[1;34mINFO[m]
[[1;34mINFO[m] Maven Project                                                      [pom]
[[1;34mINFO[m] Server                                                             [jar]
[[1;34mINFO[m] Webapp                                                             [war]
[[1;34mINFO[m]
[[1;34mINFO[m] [1m--------------< [0;36mcom.example.maven-project:maven-project[0;1m >---------------[m
[[1;34mINFO[m] [1mBuilding Maven Project 1.0-SNAPSHOT                                [1/3][m
[[1;34mINFO[m] [1m--------------------------------[ pom ]---------------------------------[m
[[1;34mINFO[m]
[[1;34mINFO[m] [1m------------------< [0;36mcom.example.maven-project:server[0;1m >------------------[m
[[1;34mINFO[m] [1mBuilding Server 1.0-SNAPSHOT                                       [2/3][m
[[1;34mINFO[m] [1m--------------------------------[ jar ]---------------------------------[m
[[1;34mINFO[m]
[[1;34mINFO[m] [1m--- [0;32mmaven-resources-plugin:2.5:resources[m [1m(default-resources)[m @ [36mserver[0;1m ---[m
[debug] execute contextualize
[[1;34mINFO[m] Using 'utf-8' encoding to copy filtered resources.
[[1;34mINFO[m] skip non existing resourceDirectory /home/ubuntu/.jenkins/workspace/Development/server/src/main/resources
[[1;34mINFO[m]
[[1;34mINFO[m] [1m--- [0;32mmaven-compiler-plugin:2.3.2:compile[m [1m(default-compile)[m @ [36mserver[0;1m ---[m
[[1;34mINFO[m] Nothing to compile - all classes are up to date
[[1;34mINFO[m]
[[1;34mINFO[m] [1m--- [0;32mmaven-resources-plugin:2.5:testResources[m [1m(default-testResources)[m @ [36mserver[0;1m ---[m
[debug] execute contextualize
[[1;34mINFO[m] Using 'utf-8' encoding to copy filtered resources.
[[1;34mINFO[m] skip non existing resourceDirectory /home/ubuntu/.jenkins/workspace/Development/server/src/test/resources
[[1;34mINFO[m]
[[1;34mINFO[m] [1m--- [0;32mmaven-compiler-plugin:2.3.2:testCompile[m [1m(default-testCompile)[m @ [36mserver[0;1m ---[m
[[1;34mINFO[m] Nothing to compile - all classes are up to date
[[1;34mINFO[m]
[[1;34mINFO[m] [1m--- [0;32mmaven-surefire-plugin:2.11:test[m [1m(default-test)[m @ [36mserver[0;1m ---[m
[[1;34mINFO[m] Surefire report directory: /home/ubuntu/.jenkins/workspace/Development/server/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------

Running com.example.TestGreeter
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.116 sec

Results :

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0

[[1;34mINFO[m]
[[1;34mINFO[m] [1m--- [0;32mmaven-jar-plugin:2.4:jar[m [1m(default-jar)[m @ [36mserver[0;1m ---[m
[[1;34mINFO[m]
[[1;34mINFO[m] [1m------------------< [0;36mcom.example.maven-project:webapp[0;1m >------------------[m
[[1;34mINFO[m] [1mBuilding Webapp 1.0-SNAPSHOT                                       [3/3][m
[[1;34mINFO[m] [1m--------------------------------[ war ]---------------------------------[m
[[1;34mINFO[m]
[[1;34mINFO[m] [1m--- [0;32mmaven-resources-plugin:2.5:resources[m [1m(default-resources)[m @ [36mwebapp[0;1m ---[m
[debug] execute contextualize
[[1;34mINFO[m] Using 'utf-8' encoding to copy filtered resources.
[[1;34mINFO[m] skip non existing resourceDirectory /home/ubuntu/.jenkins/workspace/Development/webapp/src/main/resources
[[1;34mINFO[m]
[[1;34mINFO[m] [1m--- [0;32mmaven-compiler-plugin:2.3.2:compile[m [1m(default-compile)[m @ [36mwebapp[0;1m ---[m
[[1;34mINFO[m] No sources to compile
[[1;34mINFO[m]
[[1;34mINFO[m] [1m--- [0;32mmaven-resources-plugin:2.5:testResources[m [1m(default-testResources)[m @ [36mwebapp[0;1m ---[m
[debug] execute contextualize
[[1;34mINFO[m] Using 'utf-8' encoding to copy filtered resources.
[[1;34mINFO[m] skip non existing resourceDirectory /home/ubuntu/.jenkins/workspace/Development/webapp/src/test/resources
[[1;34mINFO[m]
[[1;34mINFO[m] [1m--- [0;32mmaven-compiler-plugin:2.3.2:testCompile[m [1m(default-testCompile)[m @ [36mwebapp[0;1m ---[m
[[1;34mINFO[m] No sources to compile
[[1;34mINFO[m]
[[1;34mINFO[m] [1m--- [0;32mmaven-surefire-plugin:2.11:test[m [1m(default-test)[m @ [36mwebapp[0;1m ---[m
[[1;34mINFO[m] No tests to run.
[[1;34mINFO[m] Surefire report directory: /home/ubuntu/.jenkins/workspace/Development/webapp/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[[1;34mINFO[m]
[[1;34mINFO[m] [1m--- [0;32mmaven-war-plugin:2.2:war[m [1m(default-war)[m @ [36mwebapp[0;1m ---[m
[[1;34mINFO[m] Packaging webapp
[[1;34mINFO[m] Assembling webapp [webapp] in [/home/ubuntu/.jenkins/workspace/Development/webapp/target/webapp]
[[1;34mINFO[m] Processing war project
[[1;34mINFO[m] Copying webapp resources [/home/ubuntu/.jenkins/workspace/Development/webapp/src/main/webapp]
[[1;34mINFO[m] Webapp assembled in [33 msecs]
[[1;34mINFO[m] Building war: /home/ubuntu/.jenkins/workspace/Development/webapp/target/webapp.war
[[1;34mINFO[m] WEB-INF/web.xml already added, skipping
[[1;34mINFO[m] [1m------------------------------------------------------------------------[m
[[1;34mINFO[m] [1mReactor Summary for Maven Project 1.0-SNAPSHOT:[m
[[1;34mINFO[m]
[[1;34mINFO[m] Maven Project ...................................... [1;32mSUCCESS[m [  0.010 s]
[[1;34mINFO[m] Server ............................................. [1;32mSUCCESS[m [  2.344 s]
[[1;34mINFO[m] Webapp ............................................. [1;32mSUCCESS[m [  0.776 s]
[[1;34mINFO[m] [1m------------------------------------------------------------------------[m
[[1;34mINFO[m] [1;32mBUILD SUCCESS[m
[[1;34mINFO[m] [1m------------------------------------------------------------------------[m
[[1;34mINFO[m] Total time:  3.453 s
[[1;34mINFO[m] Finished at: 2020-02-06T19:47:13Z
[[1;34mINFO[m] [1m------------------------------------------------------------------------[m
[DeployPublisher][INFO] Attempting to deploy 1 war file(s)
[DeployPublisher][INFO] Deploying /home/ubuntu/.jenkins/workspace/Development/webapp/target/webapp.war to container Tomcat 8.x Remote with context testapp
ERROR: Build step failed with exception
org.codehaus.cargo.container.ContainerException: Failed to redeploy [/home/ubuntu/.jenkins/workspace/Development/webapp/target/webapp.war]
    at org.codehaus.cargo.container.tomcat.internal.AbstractTomcatManagerDeployer.redeploy(AbstractTomcatManagerDeployer.java:188)
    at hudson.plugins.deploy.CargoContainerAdapter.deploy(CargoContainerAdapter.java:81)
    at hudson.plugins.deploy.CargoContainerAdapter$DeployCallable.invoke(CargoContainerAdapter.java:167)
    at hudson.plugins.deploy.CargoContainerAdapter$DeployCallable.invoke(CargoContainerAdapter.java:136)
    at hudson.FilePath.act(FilePath.java:1075)
    at hudson.FilePath.act(FilePath.java:1058)
    at hudson.plugins.deploy.CargoContainerAdapter.redeployFile(CargoContainerAdapter.java:133)
    at hudson.plugins.deploy.PasswordProtectedAdapterCargo.redeployFile(PasswordProtectedAdapterCargo.java:95)
    at hudson.plugins.deploy.DeployPublisher.perform(DeployPublisher.java:113)
    at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78)
    at hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:45)
    at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:741)
    at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
    at hudson.model.Build$BuildExecution.post2(Build.java:186)
    at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
    at hudson.model.Run.execute(Run.java:1878)
    at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
    at hudson.model.ResourceController.execute(ResourceController.java:97)
    at hudson.model.Executor.run(Executor.java:428)
Caused by: javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
    at sun.security.ssl.InputRecord.handleUnknownRecord(InputRecord.java:710)
    at sun.security.ssl.InputRecord.read(InputRecord.java:527)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
    at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:167)
    at org.codehaus.cargo.container.tomcat.internal.TomcatManager.invoke(TomcatManager.java:567)
    at org.codehaus.cargo.container.tomcat.internal.TomcatManager.list(TomcatManager.java:882)
    at org.codehaus.cargo.container.tomcat.internal.TomcatManager.getStatus(TomcatManager.java:895)
    at org.codehaus.cargo.container.tomcat.internal.AbstractTomcatManagerDeployer.redeploy(AbstractTomcatManagerDeployer.java:169)
    ... 18 more
javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
    at sun.security.ssl.InputRecord.handleUnknownRecord(InputRecord.java:710)
    at sun.security.ssl.InputRecord.read(InputRecord.java:527)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
    at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:167)
    at org.codehaus.cargo.container.tomcat.internal.TomcatManager.invoke(TomcatManager.java:567)
    at org.codehaus.cargo.container.tomcat.internal.TomcatManager.list(TomcatManager.java:882)
    at org.codehaus.cargo.container.tomcat.internal.TomcatManager.getStatus(TomcatManager.java:895)
    at org.codehaus.cargo.container.tomcat.internal.AbstractTomcatManagerDeployer.redeploy(AbstractTomcatManagerDeployer.java:169)
    at hudson.plugins.deploy.CargoContainerAdapter.deploy(CargoContainerAdapter.java:81)
    at hudson.plugins.deploy.CargoContainerAdapter$DeployCallable.invoke(CargoContainerAdapter.java:167)
    at hudson.plugins.deploy.CargoContainerAdapter$DeployCallable.invoke(CargoContainerAdapter.java:136)
    at hudson.FilePath.act(FilePath.java:1075)
    at hudson.FilePath.act(FilePath.java:1058)
    at hudson.plugins.deploy.CargoContainerAdapter.redeployFile(CargoContainerAdapter.java:133)
    at hudson.plugins.deploy.PasswordProtectedAdapterCargo.redeployFile(PasswordProtectedAdapterCargo.java:95)
    at hudson.plugins.deploy.DeployPublisher.perform(DeployPublisher.java:113)
    at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78)
    at hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:45)
    at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:741)
    at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
    at hudson.model.Build$BuildExecution.post2(Build.java:186)
    at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
    at hudson.model.Run.execute(Run.java:1878)
    at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
    at hudson.model.ResourceController.execute(ResourceController.java:97)
    at hudson.model.Executor.run(Executor.java:428)
Build step 'Deploy war/ear to a container' marked build as failure
Finished: FAILURE

```

# Answer
It seems to be an issue with Tomcat URL you are using. Either you are using "HTTP" in deployment when server is expecting HTTPS or vice versa.

> hudson.model.Executor.run(Executor.java:428) Caused by: javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?

Have a look at this answer unrecognized-ssl-message-plaintext-connection-exception
> 3 votes

---
Tags: jenkins, git, jenkins-plugins, java, tomcat
---


# Question
Title: Is there a limitation in AWS to open an IPSec VPN connection?
Consider the following situation (for test purposes):

* there is a public IPSec VPN endpoint (StrongSwan);
* a machine with either Ubuntu (Strong Swan client) or Windows OS (native client) running in the cloud should be the VPN guest.

Result:

* Windows Server 2019 machine establishes VPN connection successfully but can't send any packet
* Ubuntu 18 machine establishes connection to some degree but does not create PPP0 interface.
* Ubuntu 16 machine makes a handshake but then there is a message about protocol mismatch (maybe in this specific case there is an incompatibility between both endpoints).

Are there any AWS limitations to IPSec VPN?

# Answer
For me it looks to be tunnel misconfigured - "there is a message about protocol mismatch"

Regarding the limitations quick google took me here: \[strongSwan\] strongswan limits
> 1 votes

---
Tags: amazon-web-services, devsecops, vpn
---


# Question
Title: docker | database connection refused between two containers
I have a mysql database running with the following `docker-compose.yml`:

```
version: '3.3'
    services:
        db:
            image: mysql:5.7
            restart: always
            environment:
                MYSQL_DATABASE: 'demo'
                # So you don't have to use root, but you can if you like
                MYSQL_USER: 'user'
                # You can use whatever password you like
                MYSQL_PASSWORD: 'password'
                # Password for root access
                MYSQL_ROOT_PASSWORD: 'password'
            ports:
            # <Port exposed> : < MySQL Port running inside container>
                - '3306:3306'
            expose:
                # Opens port 3306 on the container
                - '3306'
            # Where our data will be persisted
            volumes:
                - my-db:/var/lib/mysql
            networks:
                - backend

networks:
    backend:
        driver: bridge
# Names our volume
volumes:
    my-db:

```

`$ docker-compose build`

`$ docker-compose up`

---

I have a basic golang rest api with the following `Dockerfile`:

```
# Start from golang:1.12-alpine base image
FROM golang:1.12-alpine

# Adding git, bash and openssh to the image
RUN apk update && apk upgrade && \
    apk add --no-cache bash git openssh

# Set the Current Working Directory inside the container
WORKDIR /app

# Copy the source from the current directory to the Working Directory inside the container
COPY . .
RUN go get -d github.com/gorilla/mux
RUN go get -d github.com/go-sql-driver/mysql
RUN go get -d github.com/golang-migrate/migrate
RUN go get -d github.com/golang-migrate/migrate/database/mysql
RUN go get -d github.com/golang-migrate/migrate/source/file

# Build the Go app
RUN go build -o main .

# Expose port 8080 to the outside world
EXPOSE 8080

# Run the executable
CMD ["./main"]

```

and the following function is called:

```
func CreateDatabase() (*sql.DB, error) {
    serverName := "localhost:3306"
    user := "user"
    password := "password"
    dbName := "demo"

    connectionString := fmt.Sprintf("%s:%s@tcp(%s)/%s?charset=utf8mb4&collation=utf8mb4_unicode_ci&parseTime=true&multiStatements=true", user, password, serverName, dbName)
    db, err := sql.Open("mysql", connectionString)
    if err != nil {
        return nil, err
    }

    if err := migrateDatabase(db); err != nil {
        return db, err
    }

    return db, nil
}

```

`$ docker run -p 80:8080 --network=<appname>-mysql_backend <imageid>`

`$ Database connection failed: %sdial tcp 127.0.0.1:3306: connect: connection refused`

---

I cannot get the api to establish a database connection with the database container?

# Answer
Your golang api should be included in the compose file and added to the same network.

It also looks like when you run the golang api that you are adding it to a network other than `backend`, which is the network that is created in your compose file.
> 2 votes

# Answer
In addition to connecting app container with same `backend` network, you will also need to fix MySQL host parameter in connection string.

Looking at `main` function you have MySQL host set to `localhost`

`serverName := "localhost:3306"`

However, MySQL is not listening at `lcoalhost` of appc container you will need to change it to MySQL service name in compose file, i.e. `db`
> 1 votes

---
Tags: docker, docker-compose, containers, dockerfile, mysql
---


title: Dockerizing a tool to test web apps?

text:
# Question
Title: Dockerizing a tool to test web apps?
I'm new to docker/Jenkins and would like a hand understanding something I'd like to practice.

I'm trying to set up a tool (like owasp-zap) and using it to run against a container built of an intentionally vulnerable website (like owasp-juice shop). Then saving those results. My only experience in Docker is grabbing an existing image and running the container locally. I'm struggling to find docs about this and could use some guidance.

# Answer
You can dockerize/containerize pretty much anything fairly easily. You basically just have to add a Dockerfile that:

* Inherits from some base image; this provides the basic OS features and whatever else you need. E.g. a Java app may inherit from alpine-jdk-8 (a slim linux type image) that comes pre-bundled with the Java Development Kit.
* Copies in your code and any dependencies.
* Sets any variables you require.
* Exposes any ports your app needs (e.g. ports 80/443 for serving a web page).
* Mounts any volumes to save permanent data. Generally anything in a container is ephemeral; it will be gone when the container dies. So, if you want to save/export results permanently you can mount a volume for, say, your local computer's file system, or for an amazon s3 bucket, or whatever.
* Specifies how to start the app (the entry point).

Here's an example for arbitrary python scripts; but it's the same deal for any other language/tool:

https://runnable.com/docker/python/dockerize-your-python-application

If you're taking an existing executable which may be your case, you just do the same thing but instead of loading in your code, you load in the binary and set that as your entry point.

The container lives until the entry point program ends. So, some containers can be short lived - like just executing a CURL or something. Some are permanent until they die/are killed (e.g. a web service).

Again, to save state/results, unless you're writing them to a database or some other external system, you will want to mount a volume.

You build your Dockerfile into an image with a docker build command; then it will show up in your local docker image registry (`docker image ls -a`). After that, you can run it with a standard docker run command (where you will likely want to expose any ports/etc that are needed). I assume you're good with this one since you already run existing images in docker.

Generally you will need to store your image in a registry so you can use it on/from other systems. But if running it locally is all you need you're done now :).

You can store your own containers in DockerHub or in Amazon AWS ECR or in any other number of options if you require a registry.
> 2 votes

---
Tags: docker
---


# Question
Title: Find most popular images with `zip` and `git` on Docker Hub
To speed up builds, I'd like to avoid install tools that I need on every commit with `apt-get update -y && apt-get install -y git zip`.

Normal way is to create custom build image, register at Docker Hub and upload it there. But I am 100% sure that there is already an up to date image that contains both `git` and `zip` tools that I need. The only question is how to find this image?

**UPDATE 202004**: There is no indexer for DockerHub, and there is no even indexer for Docker image contents. To start with that one could get a list of files from an image by patching `dive` by @wagoodman, or using https://github.com/GoogleContainerTools/container-diff

# Answer
> 2 votes
I can answer half of your question: If you have a set of images you're trying to investigate, the best way would be to use `which` in a new container using the image in question. This way you can quickly see if a particular image has the binaries your are looking for already on `PATH`:

```
docker run --rm ubuntu:latest which zip git
# no result

docker run --rm ubuntu:latest which bash sh
/bin/bash
/bin/sh
# ^-- both bash and sh are in the image

```

I don't know of a way to discover the set of images in question, as there is no manifest of files that a registry will provide. That is, yes, an image is a set of tar files, which you can extract a set of files/directory names... however, you'd still need to pull the entire image to get this information, so it would be easier to run the `which` command above.

There are tools for exploring information that a registry exposes (skopeo, doocker-ls, reg, manifest-tool, etc) but none of them will do exactly what you're looking for, which is "find a repository within a registry which contains an image that has a particular binary in it".

# Answer
> 0 votes
There are several options:

* The first one is like you mentioned, creating an image yourself, push it to dockerhub and run `docker run yourimage git` etc.
* Another option is to use an image that has already been published like this one. One could run it by issuing:

```
docker run dockerinpractice/docker-dev-tools-image git

```
* A third option is to add git to an existing image by using a Dockerfile:

```
FROM alpine/git:1.0.7

RUN apk add --update zip

```

Personally I wonder whether docker is the right tool for the job in this case.

# Answer
> 0 votes
If you have don't want to push your images to docker.com, are on an isolated network, or if you are doing this kind of thing regularly, you could consider setting up your own local docker registry. Doing this would

Setting up a docker registry is not a trivial task, you can find detailed instructions for setting up a docker registry at docker.com, but here is a brief summary:

1. Get/run the docker registry image (yes, the registry will run inside docker):

`docker run -d -p 5000:5000 --restart=always --name registry registry:2`
2. build your image or use an existing one:

`docker tag ubuntu:16.04 localhost:5000/my-ubuntu`
3. Push this image to your logcal registry:

`docker push localhost:5000/my-ubuntu`

Once you do this, you will be able to treat this local registry like Docker Hub; you could delete your "local" copy of this ubuntu image, then pull it down from your local repository again:

```
docker image remove ubuntu:16.04
docker image remove localhost:5000/my-ubuntu
docker pull localhost:5000/my-ubuntu

```

---
Tags: docker, dockerhub
---


# Question
Title: Schedule Jenkins Job with Active choice Reactive parameters and Reactive reference parameters
I have created a Jenkins job which have Choice Parameters, Active Choice Parameters, Active choice reactive params and Reactive reference params.

This job needs to be scheduled with parameters.

As the active choice params and reactive choice params are dynamically executed with groovy scripts.

How can I pass these parameters to schedule a job with cron present in Jenkins.

I'm using Parameterized Scheduler plugin for scheduling.

**EDIT**

Say I have a job which have

1 Active choice parameter PARAM\_A

1 Active choice reactive parameter PARAM\_B

2 active choice reactive reference parameters PARAM\_C and PARAM\_D.

I tried by hard coding active choice parameter in the cron like below

```
15 7 * * * % PARAM_A=somestring

```

But with the provided active choice value, the job is getting failed. Because the **reactive reference parameters** which were written in groovy script are not getting executed when i triggered with scheduler in jenkins.

In a jenkins schedule job using parameterized-scheduler, where if we give input

# Answer
Although I haven't tried using the Parameterized Scheduler in particular, I don't think you will be able to use this plugin to trigger a build with Active Choice parameters - most plugins don't support passing Active Choice parameters to builds. There is currently an open bug report on this issue.

As for a workaround - I have been able to successfully trigger builds with Active Choice parameters using the HTTP REST API `/build` endpoint (a bit counterintuitively, NOT `/buildWithParameters`). So for instance you could have a second job that runs on a schedule and does nothing but hit the API endpoint to trigger the job and pass parameters, or you could have a plain Unix cronjob that hits the endpoint, or something else along those lines.
> 1 votes

---
Tags: jenkins
---


# Question
Title: Jenkins - Running code in Windows agent but script says we are in Linux
I have the following pipeline (Jenkinsfile):

```
@Library('mylib@master')

import com.mylib.*

pipeline {

    agent { label 'WindowsSlaveWS2016' }

    stages {
        stage('Demo') {
            steps {
                echo 'a'
                determineOS()
                echo 'b'
                // sh 'echo "[Version 1.0.0]" | tee -a changelog.txt'
                sayHello()
                //getServiceVersion('.', 'changelog.txt', 'myservice')
                // cat service.properties
                // sendEmail('failed', 'test@test.com')
                script {
                    def utils = new Utils()
                    if (utils.isWindows()) {
                        echo '<-- Is windwos -->'
                    }
                    else {
                        echo '<-- Is linux -->'
                    }
                }
            }
        }
    }

}

```

determineOS() groovy script is a simple script that does the following:

```
#!/usr/bin/env groovy

def call() {
    if (System.properties['os.name'].toLowerCase().contains('windows')) {
        println "it's Windows"
    } else {
        println "it's not Windows"
    }
}

```

and utils.Windows() does the following:

```
#!/usr/bin/groovy

package com.ciplatforms;

class Utils implements Serializable {

    def isWindows() {
        if (System.properties['os.name'].toLowerCase().contains('windows')) {
            println "it's Windows"
            return true
        } else {
            println "it's not Windows"
            return false
        }
    }

}

```

I am running the job in a Windows machine as you can notice in the pipeline but the log displays the following:

```
Running on WindowsSlaveWS2016 in C:\Jenkins\workspace\test-pipeline-shared-library
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Demo)
[Pipeline] echo
a
[Pipeline] echo
it's not Windows
[Pipeline] echo
b
[Pipeline] echo
Hello, human.
[Pipeline] script
[Pipeline] {
[Pipeline] echo
<-- Is linux -->
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

```

Any ideas why this is happening? Thank you.

# Answer
> 2 votes
I got the answer. As Mark Waite answered here:

https://groups.google.com/forum/#!topic/jenkinsci-users/aGYt-OEYds8

Use the isUnix pipeline step rather than writing your own version of that step. The System.properties that you are using will execute on the master, not on the agent. When it executes on the master, it will report the os.name value of the master.

---
Tags: jenkins-pipeline
---


title: kubeadm init fails when using cri-o

text:
# Question
Title: kubeadm init fails when using cri-o
I've been following this doc to deploy kubernetes https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/high-availability/

After launching the "kubeadm init" command and during its excecution, this is the output:

> \[kubelet-check\] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused. Unfortunately, an error has occurred: timed out waiting for the condition This error is likely caused by: \- The kubelet is not running \- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled) If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands: \- 'systemctl status kubelet' \- 'journalctl -xeu kubelet' Additionally, a control plane component may have crashed or exited when started by the container runtime. To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker. Here is one example how you may list all Kubernetes containers running in docker: \- 'docker ps -a | grep kube | grep -v pause' Once you have found the failing container, you can inspect its logs with: \- 'docker logs CONTAINERID' error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster To see the stack trace of this error execute with --v=5 or higher

It runs fine if I install docker and switch the runtime to docker instead of "remote" and "systemd".

I've tried several things, but this is the content of my last try:

> cat etc/sysconfig/kubelet
>
> KUBELET\_ARGS="--container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --runtime-request-timeout=10m --cluster\_domain=topota.madre"
>
> systemctl cat kubelet
>
> \# /usr/lib/systemd/system/kubelet.service \[Unit\] Description=kubelet: The Kubernetes Node Agent Documentation=https://kubernetes.io/docs/
>
> \[Service\] ExecStart=/usr/bin/kubelet Restart=always StartLimitInterval=0 RestartSec=10
>
> \[Install\] WantedBy=multi-user.target
>
> \# /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf \# Note: This dropin only works with kubeadm and kubelet v1.11+ \[Service\] Environment="KUBELET\_KUBECONFIG\_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/ Environment="KUBELET\_CONFIG\_ARGS=--config=/var/lib/kubelet/config.yaml" \# This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET\_KUBEADM\_ARGS var EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env \# This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user shoul \# the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET\_EXTRA\_ARGS should be sour EnvironmentFile=-/etc/sysconfig/kubelet ExecStart= ExecStart=/usr/bin/kubelet $KUBELET\_KUBECONFIG\_ARGS $KUBELET\_CONFIG\_ARGS $KUBELET\_KUBEADM\_ARGS $KUBELET\_EXTRA\_ARGS
>
> cat /var/lib/kubelet/kubeadm-flags.env
>
> is empty right now

Cri-O service ir running ok and Docker is not installed.

* kubelet version --short -\> Client Version: v1.17.2
* kubeadm version -o short -\> v1.17.2
* cat /etc/redhat-release -\> CentOS Linux release 7.7.1908 (Core)
* crictl version -\> \["Version: 0.1.0","RuntimeName: cri-o","RuntimeVersion: 1.15.1-2.el7","RuntimeApiVersion: v1alpha1"\]
* crictl info -\> {"status": {"conditions": \[{"type": "RuntimeReady","status": true,"reason": "","message": ""},{"type": NetworkReady","status": true,"reason": "","message": ""}\]}}
* grep -i cgroup\_manager /etc/crio/crio.conf -\> cgroup\_manager = "systemd"
* lsmod | grep -i "overlay\\|br\_netfilter" -\> \["br\_netfilter","overlay"\] (both ok).
* Other "extra args tested" (/etc/sysconfig/kubelet)

> KUBELET\_EXTRA\_ARGS=--cgroup-driver=systemd --container-runtime-endpoint=unix:///var/run/crio/crio.sock

* Full output of some commands: -\> "kube admin init --upload-certs" -\> https://pastebin.com/isbAynEg -\> "while true ; do systemctl status kubelet ; sleep 1 ; done" -\> https://pastebin.com/9pgtDMNu -\> "cat /etc/sysctl.d/99-kubernetes-cri.conf" -\> \["net.bridge.bridge-nf-call-iptables = 1","net.ipv4.ip\_forward = 1","net.bridge.bridge-nf-call-ip6tables = 1"\] -\> "journalctl -xeu kubelet" -\> https://pastebin.com/qreJyhsu (this one has some things like "aws credentials" which i dont understand and not sure if are the reason or not for the errors).

Any help please?

# Answer
I'm running crio 1.17.0-rc1 with kubernetes 1.17.1 on RHEL7/CENTOS7. One thing I had to do was build crio 1.17 myself because there was no package available and I read the version should match (crio 1.17 and k8 1.17).

I am using systemd as the cgroup driver. I'm not sure what your issue might be, but my setup goes like this:

```
cat /etc/sysconfig/kubelet
KUBELET_EXTRA_ARGS=--cgroup-driver=systemd --container-runtime-endpoint=unix:///var/run/crio/crio.sock --cloud-provider=external

```

My kubeadm init line is (I'm using an env variable to set the control plain endpoint):

```
kubeadm init --pod-network-cidr=10.244.0.0/16 --control-plane-endpoint {{ lookup('env','master_lb_ip_address') }}:6443 --upload-certs >> $HOME/log/init-cluster.log

```

I also edited /etc/crio/crio.conf to set systemd as the cgroup manager.

```
cgroup_manager = "systemd"

```

Also make sure you are enabling the modules / sysconfig options specified on this page.

```
modprobe overlay
modprobe br_netfilter

# Setup required sysctl params, these persist across reboots.
cat > /etc/sysctl.d/99-kubernetes-cri.conf <<EOF
net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward                 = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

sysctl --system

```
> 2 votes

---
Tags: kubernetes, kubeadm
---


# Question
Title: Docker won't create a bridge network without being given an explicit subnet
# The problem

The problem is that this happens:

```
$ docker network create  alpine4
Error response from daemon: could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network

```

Notwithstanding that I have *no* non-builtin docker networks created.

The logs suggest that these network ranges are being attempted:

```
DEBU[2018-08-08T17:21:00.877120275Z] ReleasePool(LocalDefault/192.168.240.0/20)
DEBU[2018-08-08T17:21:00.877135316Z] ReleasePool(LocalDefault/192.168.224.0/20)
DEBU[2018-08-08T17:21:00.877162150Z] ReleasePool(LocalDefault/192.168.208.0/20)
DEBU[2018-08-08T17:21:00.877170059Z] ReleasePool(LocalDefault/192.168.192.0/20)
DEBU[2018-08-08T17:21:00.877176154Z] ReleasePool(LocalDefault/192.168.176.0/20)
DEBU[2018-08-08T17:21:00.877198642Z] ReleasePool(LocalDefault/192.168.160.0/20)
DEBU[2018-08-08T17:21:00.877204723Z] ReleasePool(LocalDefault/192.168.144.0/20)
DEBU[2018-08-08T17:21:00.877212452Z] ReleasePool(LocalDefault/192.168.128.0/20)
DEBU[2018-08-08T17:21:00.877220350Z] ReleasePool(LocalDefault/192.168.112.0/20)
DEBU[2018-08-08T17:21:00.877232829Z] ReleasePool(LocalDefault/192.168.96.0/20)
DEBU[2018-08-08T17:21:00.877243072Z] ReleasePool(LocalDefault/192.168.80.0/20)
DEBU[2018-08-08T17:21:00.877253643Z] ReleasePool(LocalDefault/192.168.64.0/20)
DEBU[2018-08-08T17:21:00.877264171Z] ReleasePool(LocalDefault/192.168.48.0/20)
DEBU[2018-08-08T17:21:00.877274553Z] ReleasePool(LocalDefault/192.168.32.0/20)
DEBU[2018-08-08T17:21:00.877284675Z] ReleasePool(LocalDefault/192.168.16.0/20)
DEBU[2018-08-08T17:21:00.877296023Z] ReleasePool(LocalDefault/192.168.0.0/20)
DEBU[2018-08-08T17:21:00.877307062Z] ReleasePool(LocalDefault/172.31.0.0/16)
DEBU[2018-08-08T17:21:00.877322208Z] ReleasePool(LocalDefault/172.30.0.0/16)
DEBU[2018-08-08T17:21:00.877333550Z] ReleasePool(LocalDefault/172.29.0.0/16)
DEBU[2018-08-08T17:21:00.877339489Z] ReleasePool(LocalDefault/172.28.0.0/16)
DEBU[2018-08-08T17:21:00.877350224Z] ReleasePool(LocalDefault/172.27.0.0/16)
DEBU[2018-08-08T17:21:00.877361031Z] ReleasePool(LocalDefault/172.26.0.0/16)
DEBU[2018-08-08T17:21:00.877367371Z] ReleasePool(LocalDefault/172.25.0.0/16)
DEBU[2018-08-08T17:21:00.877374978Z] ReleasePool(LocalDefault/172.24.0.0/16)
DEBU[2018-08-08T17:21:00.877382471Z] ReleasePool(LocalDefault/172.23.0.0/16)
DEBU[2018-08-08T17:21:00.877395020Z] ReleasePool(LocalDefault/172.22.0.0/16)
DEBU[2018-08-08T17:21:00.877405540Z] ReleasePool(LocalDefault/172.21.0.0/16)
DEBU[2018-08-08T17:21:00.877415851Z] ReleasePool(LocalDefault/172.20.0.0/16)
DEBU[2018-08-08T17:21:00.877426869Z] ReleasePool(LocalDefault/172.19.0.0/16)
DEBU[2018-08-08T17:21:00.877437210Z] ReleasePool(LocalDefault/172.18.0.0/16)

```

Even stranger, if I pick any of those subnet ranges to use as an explicit subnet range, creation works *just fine*:

```
$ docker network create --subnet 172.31.0.0/16 alpine3
189ec86443c0799a298f672497aa24fb17dc725d810cd6c3917555ae6c185012

```

# The question

Given the above, why can't I just create a bridge network and have docker pick a subnet range for me?

# Setup

```
$ dockerd --version
Docker version 18.03.1-ce, build 9ee9f40

```

Running on ubuntu 16.04 in aws. Installed using the official .deb, no special configs in place (indeed, no `daemon.json` at all)

# Answer
Do you need to run it beside a VPN "OpenVPN service" ?

I think using below would solve the issue for clearing any unused networks by at least 1 container.

```
$ docker network prune

```
> 2 votes

# Answer
You can explore the sources to investigate concrete case. Also I explained in this stackoverflow answer why it probably happened.
> 0 votes

---
Tags: docker, linux, networking
---


title: DevOps approach to creating demo data?

text:
# Question
Title: DevOps approach to creating demo data?
Is there a good DevOps method to construct a workflow that creates demo data for a product? This situation is difficult when the app constantly changes. There are a few scenarios to consider.

1. Either old demo data becomes obsolete in the new build or the old data can be considered corrupted in the new build.
2. New business logic is introduced which requires interaction with the app for the new data to be formed. You cannot look at the new schema, static rules or old data to derivative/fill in such data.
3. Program managers have the insider info on what the demo data needs to look like, not the engineers. So the workflow needs to match up during the process, from (build -\> test -\> deploy) somehow.

One approach is to run tests that have demo data in it. So after the tests were ran, we can extract the data for our demos. But I can see that the tests can be very long and not as instant as importing a csv file. Also the coordination between PMs and engineers seems to be tightly coupled.

Another approach is to start with the new scheme from the repository and generate data from there. But this requires overhead of creating rules for generation which may not be in sync with the logic of app itself.

It seems a top down approach is better, so you generate dynamic data sets per release. Instead of working with a backward compatibility mindset with older static data.

The problem is to keep the app and the data in sync and valid for each iteration, while allowing the PMs and engineers to work in parallel from one another without relying on each other.

# Answer
> 1 votes
Welcome to DevOps SE!

Is it possible to make data management and operation of data sets to be an asset on its own? It seems like data has become more important so a more *data-centric* approach could help.

> Program managers have the insider info on what the demo data needs to look like

This proves also organizational silo and lack of communication in the process: even before DevOps, in an agile team e.g. scrum you could have not only engineers but also a product owner and testers who take responsibility to provide not only useful tests but also useful testdata to the team.

That is, the fourth possible scenario could be that you take a data set from production and scramble/anonymize it. If it is too large, you need to reduce it to a representative mix. A DevOps way to do it would be to automate all these steps after they are well understood. =\> Working real life scenario.

There are some legal workaround methods though which require though considerable additional effort:

* data scrambling
* data anonymization
* synthetic data

Depending on your domain, there could be also applicable and more or less flexible enterprise tools for master data management where product owners could manage and version various data profiles.

Since DevOps approach is more than just finding a tool to use, but also doing experiments to find out more, here are examples of recent academic research on these topics.

An example study: *Going Beyond Obscurity: Organizational Approaches to Data Anonymization* (Hargitai et al. 2018)

> Rather than being a purely technical question of applying the right algorithms, **anonymization** in practice is a complex socio-technical process that relies on multi-stakeholder collaborations.

Or: *Machine learning using synthetic and real data: Similarity of evaluation metrics for different healthcare datasets and for different algorithms*. (Heyburn et al., 2018)

> In this paper, we carry out an experiment to study the validity of conducting machine learning on **synthetic data**.

# Answer
> 0 votes
Would it be possible to segregate data by level of maturity/stability of its functionality? The same approach which is used to version the API might work here quite well.

Conceptually it would mean that data for *stable* functionality could be copied from build to build via snapshots of the underlying data store (maybe with few alterations depending on your context).

Data for *preview* (unstable, in development) functionality is best to generate - this way no matter how functionality is altered, it is always possible to get the fresh view of how the product is going to look like. Generation can happen (for example) as a pre-demo step, or straight in the app (auto-fill in the UI, it can even be made to look neat).

As soon as *preview* functionality graduates to *stable*, its data can be included into the snapshot (as an additional release process step)

It would be best to align with engineering on common way to generate data for new functionality and then ship generators for it’s data along with code itself.

This hybrid approach allows to achieve following benefits:

* Set up and align expectations between devops, engineering and product about how test data is obtained
* No need to deal with exponential growth of time to generate *entire* data set
* Enable quick iteration of features under development
* Educate developers and make them more DevOps aware (they’d have to take last stable data snapshot and roll it out in their dev env when they start working on a feature)
* Enable fast feedback loop between all the parties involved which leads to better product quality (if snapshot doesn’t rollout or app crashes, then the release process is broken)

There aren’t many cons of this approach, but I’d highlight a few most important:

* It requires work. Main point of this approach is to fix (or actually define) the process, so it would require taking an extra mile to weigh, explain, prototype and build the alignment
* It would not solve problems with depencies in preview functionality, eg if feature B depends on A and they *both* are under development, data for both have to be generated and A has to be stabilized before B.

Overall, it’s not an easy problem you’re dealing with, and I would be appreciste if you could share your experience on how did it go, what path have your project had taken and reasoning behind it. Cheers!

---
Tags: azure-devops, workflow, data
---


# Question
Title: Why will my container run in Docker, but will not run in Kubernetes?
**Background**

I have a container that I have defined with an `CMD` and an `ENTRYPOINT` in Docker. When I run the following command my container runs with no issues whatsoever.

```
docker run --rm -ti custom-container:latest

```

Here is the definition for my container:

```
FROM mcr.microsoft.com/powershell:6.2.3-ubuntu-bionic
COPY . /app
WORKDIR /app

CMD ["/app/Start-Test.ps1"]
ENTRYPOINT ["pwsh", "-f", "/app/Start-Test.ps1"]

```

The script `Start-Test.ps1` is just an infinite loop that writes `Hello World!` to host.

**Problem**

When I go to run my container in Kubernetes, my container refuses to run.

Kubernetes is able to pull my image and it attempts to spin up the container, but I keep getting a status of BackedOff from Kubernetes.

I've gone through and tried to figure out why this is happening because the container runs perfectly fine in Docker.

I only have issues when I try to run the container in Kubernetes.

Here is the definition for my Kubernetes deployment.

```
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: my-custom-container
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-custom-container
  template:
    metadata:
      labels:
        app: my-custom-container
    spec:
      containers:
      - name: my-custom-container
        image: repository-url.io/custom-container:latest
      imagePullSecrets:
      - name: regcred

```

**Question**

What is the issue preventing my container from running when it runs with no problems in Docker?

Additionally, yes I would like to use PowerShell as my entry point. It should be possible to use any scripting langauge (bash, Ruby, Python, Perl, etc) as an entry point.

# Answer
> 2 votes
To debug this, check the container logs and the container events.

* kubectl get pods
* Find your pod name.
* kubectl logs
* If logs don't make it obvious...
* kubectl get events

If you deployed into a namespace, make sure to add -n to all commands.

This should reveal basically everything. If you're stuck, you can also browse around the k8s dashboard if you haven't yet. But that probably wouldn't reveal much more than the CLI is with these commands.

# Answer
> 1 votes
K8s works with concept of Health Check, in your application you have some port or command/trigger that K8s can monitoring? Maybe because of that your container is not works.

# Answer
> 1 votes
1. It looks like your dockerfile has both `CMD` and `ENTRYPOINT`, so it’s highly likely that what gets invoked as a commsnd is `pwsh -f /app/Start-Test.ps1 /app/Start-Test.ps1`. It could be that this command gives an error, `kubectl describe <your-pod>` might shed some light onto what’s going on. Check this answer for details on difference between `CMD` and `ENTRYPOINT`
2. If you’re getting `ImagePullBackOff`, then image might be missing from the registry or secrets are invalid.

Hope this helps

# Answer
> 0 votes
When running PowerShell as the `ENTRYPOINT` for the container PowerShell should be executed as:

```
ENTRYPOINT ["pwsh", "-NoExit", "-File", "/app/Start-Test.ps1"]

```

The short form of each of those parameters can be used. Additionally, the `-NoExit` flag will help with debugging the issue in for your deployment. The reason the `-NoExit` flag helps is the deployment stays alive long enough for you to confirm the logs and the container health.

This answer along with answers from Alexey Skolyarov and John Humphreys to debug and solve my issue.

---
Tags: docker, kubernetes, containers, powershell, azure-kubernetes-service
---


# Question
Title: How do i connect docker pipeline on jenkins to kubernetes cluster
I am trying to setup a pipeline on jenkins that uses docker containers to execute tasks like for example deploy an helm charts using a docker image with helm

here is what my **Jenkinsfile** looks like

```
pipeline {
    agent none
    stages {
        stage('Helm') {
            agent {
                docker { image 'dtzar/helm-kubectl' }
            }
            steps {
                sh '''
                helm version
                kubectl version
                helm ls
                kubectl get all -n test
                '''

            }
        }
    }
}

```

but it fails at the `helm ls` line because it can not connect to the kubernetes cluster. How do i connect this pipeline to use the **Kubernetes configuration(kubeconfig)** i have setup

here is error am getting

```
+ helm ls
Error: Kubernetes cluster unreachable

```

I could not find any documentation on how to connect docker pipeline to connect to the kubernetes cluster What do i add to the Jenkinsfile to be able to connect?

> **UPDATE**

i also tried this

```
pipeline {
    agent none
    stages {
        stage('Helm') {
            agent {
                docker { image 'dtzar/helm-kubectl' }
            }
            steps {
              withCredentials([kubeconfigContent(credentialsId: 'k8s-cluster', variable: 'KUBECONFIG_CONTENT')]) {
                sh '''
                helm version
                helm ls
                kubectl get all -n test
                '''
              }

            }
        }
    }
}

```

and got this error

```
+ helm ls
Error: Kubernetes cluster unreachable

```

I could not find any documentation on how to connect docker pipeline to connect to the kubernetes cluster

What do i add to the Jenkinsfile to be able to connect?

so seems the docker container is unable to resolve the dns of the kubernetes cluster

how do i resolve this?

# Answer
> 1 votes
Maybe it is strange solution, but I would like to provide it to you:

You can get actual kubeconfig file from your Kubernetes cluster or locally and change it to base64 format like in example below:

```
- cat /root/.kube/config | base 64

```

Then use your base64 configuration file in pipeline:

```
- echo $your_variable | base64 -d | tee /root/.kube/config

```

In such case you will be having actual kubeconfig file on runner machine.

In case of DNS problem - just need to specify IP address of your runner machine in /etc/resolv.conf file.

---
Tags: docker, jenkins, kubernetes, cicd
---


# Question
Title: Let docker containers work with each other
I’m new to docker and I want to dockerize a aleardy existing application.

It consists of a react application (I already created a container for that one successfully), a REST-api based on node-express (that one kinda works) and a mysql-database (not workin so far).

While developing I put a proxy into the package.json file of my React-Application. This enabled me to forward all the api-requests to a different port. And I want to also have this feature between my docker containers. So that my react application on port 3000 for example forwards all the unknown requests to the rest api on port 5000 via a proxy. Port 3000 and 5000 are in different containers.

Furhtermore I need the REST-container to have access to port 3306 of another mysql-container. This other mysql container still needs to be created though. It should use hypriot/rpi-mysql or something similar and within the dockerfile it should import a given .sql file into the database and then run in the background.

So my questions are: What would be the best way to share ports among the containers? How can I set up a Dockerfile for a MySql container with a .sql file as setup?

# Answer
> What would be the best way to share ports among the containers?

You don't really "share" ports between containers. Instead, you want to create network and attach each container to that network. Each container gets a **network alias** (essentially a hostname) that you can use to hit the service.

```
$ docker network create foo
$ docker run --network=foo --network-alias=mysql -p 3306:3306 my-mysql:latest
$ docker run --network=foo --network-alias=restapi -p 5000:5000 --link mysql my-restapi:latest
$ docker run --network=foo --network-alias=react -p 3000:3000 --link restapi my-reactapp:latest

```

So the *myrestapi* container can access the mysql container using the hostname `mysql` and port 3306 and your react app can hit the *myrestapi* container using the hostname `restapi`. The containers will also be accessible from your localhost so you can hit your react app with a browser without additional setup

In docker-compose.yml (which I don't use a ton so someone may have a better solution), it would look something like this:

```
networks:
  foo:
services:
  mariadb:
    container_name: mysql # this works for your alias so long as you don't have another container called mysql
    image: mariadb:latest
    networks: ["foo"]
    ports:
    - 3306:3306

```

The other services would follow suit.

As for "priming" the database, that's a little trickier. If the DB has to be running, you would need to replace the docker container's entrypoint to start the server, run the mysql command to import the data, then your script would just spin forever.

Alternately, you could script your container start-up and after it starts, import the data that way.
> 2 votes

---
Tags: docker, docker-compose, node.js, orchestration, mysql
---


title: Kubernetes: CPU Limits at (200%)?

text:
# Question
Title: Kubernetes: CPU Limits at (200%)?
So, let's say there's a cluster node, with 4 CPU cores.

It's running two pods. Each pod sets the CPU requirement at: `500m`, but the CPU limit at `4000m`.

running `kubectl describe node <noode>` tells me CPU limit is at `8 (200%)`

I understand it claims `200%` because I have two pods, each setting its CPU limit at 4 CPU cores.

Is that just semantics or is there a potential issue? I mean, if both pods need to burst to 3 CPU cores (below each one's limit) it would not be physically possible, and they'd have to settle with less.

Is there a potential issue with the sum of all pod limits, exceeding their host node CPU capability? or is just an indication of what's been asked as CPU limit, knowing it may not be able to serve that -- after all, pods may not be aware of each other being in the same node.

# Answer
This works exactly in the way you're describing. You have two pods which each have a limit of 4 CPUs scheduled on the same node; so the total of the limits on that node is 8 CPUs; which is 200% of the node's physical capacity; and if the two nodes combined go over the node's physical abilities then the host kernel will time-slice available resources. For CPU there's nothing especially wrong with this.

The exact same presentation in the `kubectl describe node` output applies to memory, and there is the potential for trouble with that: if your pod limits add up to 200% of a node's physical memory, and together they exceed the physical capacity, then the host kernel will respond by killing one of them off. Lower requests will allow more things to be potentially scheduled on the same node, but keeping requests closer to limits gives you a better chance of them staying running long-term.
> 3 votes

---
Tags: kubernetes
---


title: exponential backoff for git pushes?

text:
# Question
Title: exponential backoff for git pushes?
We've got a ton of jobs in our Jenkins that all do git pushes, sometimes simultaneously. Unsurprisingly, if two jobs try to push at the same exact time, one will succeed while the other fails. My knee-jerk reaction is to build some kind of exponential backoff and retry for these git pushes, but I'm pretty unfamiliar with git. Does something like that exist already?

Our commit/push code generally looks like this:

```
git config --global user.name "username"
git config --global user.email email@example.com
git add --ignore-errors -- dir/
git commit -m "some message"
git pull --rebase origin
git push origin master

```

As you can see, the bandaid originally applied to the problem was to put a pull before the push to fix other jobs that pushed while the current one was running. That works for ~80 of cases, but if the two pushes fall within the same \<1sec window, there are still failures.

Here's an example of a failure for the last two lines above: (Anything in \<\> is edited for privacy)

```
+ git pull --rebase origin
Warning: Permanently added '<bitbucket.org ip>' (RSA) to the list of known hosts.
From <bitbucket.org:dir>
   <commit#>..<commit#2>  master     -> origin/master
First, rewinding head to replay your work on top of it...
Applying: <commit name>
+ git push origin master
Warning: Permanently added '<bitbucket.org, ip>' (RSA) to the list of known hosts.
remote: error: cannot lock ref 'refs/heads/master': is at <commit sha> but expected <different commit sha>
To <bitbucket.org:dir>
 ! [remote rejected] master -> master (failed to update ref)
error: failed to push some refs to 'git@bitbucket.org:<team/dir>'

```

So my question boils down to:

1. Is there some form of backoff-and-retry built into git?
2. If not, how can I build that (assume a shell block in a jenkins freestyle job)?

# Answer
Instead of trying to find a solution in git, I would recommending using the Lockable Resources Plugin, which is an official plugin developed by the Jenkins developers. This plugin implements locks in Jenkins, which are basically the same idea as a semaphore or mutex. Using this plugin, each of your jobs that needs to push to the same branch of the same repo will request a lock when performing the push. If the lock is available, the job will acquire the lock, push right away, and then release the lock when it is finished pushing. If the lock is not available, the job will wait until the lock is available (i.e., all other jobs are done pushing) before grabbing it, pushing, and releasing the lock.
> 2 votes

---
Tags: jenkins, git
---


# Question
Title: How to send local files using Terraform Cloud as remote backend?
I am creating AWS EC2 instance and I am using Terraform Cloud as backend.

in **./main.tf**:

```
terraform {

    required_version = "~> 0.12"
    backend "remote" {
    hostname     = "app.terraform.io"
    organization = "organization"
    workspaces { prefix = "test-dev-" }

  }

```

in **./modules/instances/function.tf**

```

resource "aws_instance" "test" {
    ami = "${var.ami_id}"
    instance_type = "${var.instance_type}"
    subnet_id = "${var.private_subnet_id}"
    vpc_security_group_ids = ["${aws_security_group.test_sg.id}"]
    key_name      = "${var.test_key}"

    tags = {
        Name = "name"
        Function = "function"
    }

  provisioner "remote-exec" {
    inline = [
      "sudo useradd someuser"
    ]

    connection {
      host = "${self.public_ip}"
      type        = "ssh"
      user        = "ubuntu"
      private_key = "${file("~/.ssh/mykey.pem")}"
    }
  }
}

```

and as a result, I got the following error:

```
Call to function "file" failed: no file exists at /home/terraform/.ssh/...

```

so what is happening here, is that terraform trying to find the file in Terraform Cloud instead of my local machine. How can I transfer file from my local machine and still using Terraform Cloud?

# Answer
> 0 votes
As @Vasiliy Shakhunov and @SheldonH have mentioned, there is no straight way to do what I asked in the question. In the end I ended up uploading the keys into AWS with its CLI like this:

```
aws ec2 import-key-pair --key-name "name_for_the_key" --public-key-material file:///home/user/.ssh/name_for_the_key.pub

```

and then reference it like that:

```
resource "aws_instance" "test" {

    ami = "${var.ami_id}"

    ...

    key_name      = "name_for_the_key"

    ...

}

```

**Note** Yes `file://` looks like the "Windowsest" syntax ever but you have to use it on linux too.

# Answer
> 1 votes
May be a better way to manage private key is using protected variables into the Terraform Cloud. Otherwise you have to save it in repo that is not secure.

# Answer
> 1 votes
Currently when you run terraform against a remote workspace in terraform cloud the repository that you are working in gets uploaded. If your key is contained in that repo and is a relative path from your working directory then it should find it. However, as mentioned in another answer, leaving your key in your repo is not a good security practice.

I'd recommend considering something like SSM parameter store, vault, or other secure method of storing such information.

However there's one additional way you could do this. If you create the key as a resource, then you could refer to this created key as a data source and terraform and everything would be stored in the state file. If you are okay with that then you could abstract the need to have any file locally, and instead store the key value as an output from a related plan or the same plan.

Be aware this still has some security implications if you don't ensure your remote workspace and terraform cloud has security applied to different levels of users in the team plan. I'm a pretty big fan of using the state this way though when possible, Is it simplifies and removes one more extra complexity from my deployment.

The last option is the other user mentioned, is secure variables. this means you would set up a environment variable that is marked as sensitive in the settings. This would prevent it from being in plain text, and might be a viable option if you can just pass it in part of your job. There's a Python library that supports API calls to simplify updating them across the board if you don't want to use the terraform cloud provider for terraform cloud.

---
Tags: terraform, provisioning
---


# Question
Title: Does Kubernetes internal networking keep traffic to co-located Pods where possible?
In Kubernetes, if I run a Service *Foo* on every Node, and also Service *Bar* on every Node, i.e. with `DaemonSets`. If a Pod from Service *Foo* needs to make a request to Service *Bar*, does Kubernetes networking magic optimise things so communication will be done within the same Node, i.e. without a external network call? Or will a random Pod *Bar* be picked meaning usually *Bar* will be on a different Node, so there will be a network hop?

*Context to question*: I'm thinking to run `CoreDNS` as a `Daemonset`, because with the *cache* plugin, I'm hoping most DNS queries will not need to use the network as there will always be a CoreDNS Pod co-located within the same Node and most requests will be cached. Currently we only run 3 CoreDNS pods and we have 12 Nodes and we see some DNS failures at times of network traffic surges - particularly during a deployment when lot of logging is done, which have to be shipped across the network to ElasticSearch.

---

In case specifics matter:

* The CoreDNS service runs as `clusterIP: 172.20.0.10`
* The services that use CoreDNS run as `type: NodePort`
* The Kubernetes cluster is AWS EKS, v1.14.9. Networking is done with the AWS VPC CNI.

# Answer
> 3 votes
No, Kubernetes will load-balance requests to a `ClusterIP` across all Pods with matching labels (and passing readiness checks). It won't keep traffic within the same Node.

---

Q: So how does the NodeLocal DNS Cache achieve to keep DNS requests local to the Node?

A: When using the NodeLocal DNS Cache, it is necessary to pass a extra arg to the Kubelet of each Node (e.g. `--kubelet-extra-args "--cluster-dns=169.254.20.10 ..."`). This changes the `resolv.conf` of every container in every Pod to use DNS resolver on the Node, instead of a `clusterIP`. The NodeLocal Cache runs on every Node (as a DaemonSet) and uses a arg (e.g. `"-localip", "169.254.20.10"`) to run on the specific IP set in the `resolv.conf` of all Pods. The NodeLocal DNS Cache Pod also needs to be configured with `hostNetwork: true`.

# Answer
> 2 votes
This looks to be answered on offcail docs here

> NodeLocal DNSCache improves Cluster DNS performance by running a dns caching agent on cluster nodes as a DaemonSet.
>
> With this architecture, Pods will reach out to the dns caching agent running on the same node, thereby avoiding iptables DNAT rules and connection tracking.
>
> **Having a local cache will help improve the latency.**

Image Reference : https://kubernetes.io/docs/tasks/administer-cluster/nodelocaldns/#architecture-diagram

---
Tags: kubernetes, networking, dns
---


title: mariadb docker on raspberry

text:
# Question
Title: mariadb docker on raspberry
I've tried countless docker images, but all of them have the same behaviour.

When I open up a console and try to import my database structure via the mysql/mariadb command. But I always get the same error:

```
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)

```

This is my dockerfile:

```
version: '3'

networks:
    mynet:
        driver: bridge

services:
  mariadb:
    container_name: mariadb
    image: webhippie/mariadb # this one doesn't work (at least for me)
    networks:
        mynet:
            aliases:
                - database
    environment:
        - MARIADB_ROOT_PASSWORD=topsecret
        - MARADB_DATABASE=mydb
    ports:
    - 3306:3306

```

Do you have any suggestions on how I can login via the console?

# Answer
> 1 votes
Does the following work?

```
docker-compose exec mariadb /usr/bin/mysql -u <your_db_username> --password=<your_db_password> < your_db_backup.sql

```

Note: `mariadb` is the name of your container.

Edit: I realize I'm using docker-compose. So I'm not entirely sure if this would work with just a Dockerfile.

---
Tags: docker, docker-compose, mysql
---


# Question
Title: Do I need header files once I install the php extention on Dockerfile?
Usually when I build docker images for php I use this approach:

```
FROM  php:7.2-apache

RUN apt-get update && \
    apt-get install -y libghc-postgresql-simple-dev &&\
    docker-php-ext-configure pgsql -with-pgsql=/usr/local/pgsql &&\
    docker-php-ext-install pdo pgsql pdo_pgsql &&\
    // Install some other extentions there
    apt-get autoremove -y &&\
    apt-get autoclean &&\
    rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* cache/* /var/lib/log/*

```

But this can result an image of 1.2GB and I need to shave some space. AFAIK header files such as the `libghc-postgresql-simple-dev` take a lot of space in my image so after building and installing the module I though of removing it and use the non-dev variant `libghc-postgresql-simple` or removing it completely.

So I want to ask: 1. Instead of dynamically linking it how I can statically link it on my build once i run `docker-php-ext-install` in order to remove the `libghc-postgresql-simple-dev` package? 2. Would once I build the php module removing the `libghc-postgresql-simple-dev` and using the `libghc-postgresql-simple` would make the php psql extention to work as supposed to?

The idea behind it is to optimize the build in order to make my image as small as possible.

# Answer
> 1 votes
You shouldn't need to statically link things to remove the `*-dev` packages and have everything still work. If you do, then that's a bug in the base package and the `-dev` package and should be reported upstream. So yes, you should in theory be able to, after installing your extensions and such, do the following and shrink the image significantly:

```
apt-get install -y libghc-postgresql-simple && apt-get remove libghc-postgresql-simple-dev

```

---

There are two other things you can do to improve your image sizes:

* Use the `--no-install-recommends` switch for all of your `apt-get install` invocations, and explicitly list every package you depend on. Recommended dependencies are a type of optional dependency in Debian packages that are installed by default. They can often take up a *lot* of space, but you may not actually need most (or any) of them, so you can save space by not auto-installing them and only pulling in what you actually need. This can easily cut down image sizes multiple hundreds of MB.
* Use the `--purge` option whenever removing packages. This will ensure that they leave nothing behind, instead of potentially leaving some config files and other similar stuff, which can also help cut down on your space usage.

---
Tags: docker, dockerfile, php
---


# Question
Title: Jenkins file that uses docker-compose gives ambigous error
I'm new to Jenkins and I want to automatize my current workflow a little bit. Right now I push it to the raspberry. Then via SSH I pull the changes, run `docker-compose build` and `docker-compose up` afterwards.

My jenkins file is:

```
pipeline{
    agent any

    stages{
        stage('build'){
            steps{
                script{
                    docker.withTool("default") {
                        sh 'docker-compose build'
                    }
                }

            }

        }

        stage('deploy'){
            steps{
                script{
                    docker.withTool("default") {
                        sh 'docker-compose up'
                    }
                }

            }
        }
    }
}

```

And when clicking 'build now' I get the following console-output:

```
[Pipeline] }
[Pipeline] // stage
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (build)
[Pipeline] script
[Pipeline] {
[Pipeline] tool
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (deploy)
Stage "deploy" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: No org.jenkinsci.plugins.docker.commons.tools.DockerTool named default found
Finished: FAILURE

```

I also tried running it with docker.withTool("docker") but that gave me the same results.

# Answer
> 1 votes
You need to install the Docker Commons plugin? https://plugins.jenkins.io/docker-commons/

Edit: According to this link for I would like to use Docker CLI inside of a Pipeline job

> You need to make sure that “default” points to a docker tool inside of Manage Jenkins\> Configure System which will also run on the os of the slave.

---
Tags: docker, jenkins, jenkins-pipeline, docker-compose
---