title: Cross-CI on Windows and Mac

text:
# Question
Title: Cross-CI on Windows and Mac
I am deploying CI with several runners - however they are all based on the Docker executor on Linux. Is it possible to use Linux runners to test applications on Windows or Mac instead?

I understand I could use the VirtualBox executor and have a MacOS/Windows image, however I fail to understand if it is possible to do this without acquiring a license for MacOS or Windows (maybe there are free images for CI available?)

My actual initial question was geared towards running Windows/MacOS images inside a docker container on Windows - is it possible? I assume it isn't because we are still using the Linux kernel but would like confirmation.

# Answer
> 1 votes
There are some fundamental misunderstandings in your bullets which makes your question unclear:

1. Apple will not allow you to develop/build using a virtual system. I believe this is a legal constraint. That said, you can run pretty much anything in VirtualBox as long as you have a legally provided image (e.g. a Linux image).
2. You can only run the Windows base images if you are running Windows Containers which will be incompatible with MacOS systems. If you want to build on MacOS using Docker you will need to use Linux containers.

However, you can run your continuous-integration runners across Windows and Mac. This is one of the major benefits of implementing runners in Docker for your CI tool. Now if you have agents on Windows, MacOS, or even Linux, you can get a consistent build experience by using Docker. This official GitLab article has more about setting this up.

---
Tags: docker, continuous-integration, windows, gitlab-ci-runner, macos
---


# Question
Title: Can Terraform pull down the newest state file by itself?
We started using Terraform to create VMs in some Cloud Providers.

But we are a team with several people.

Terraform has a state file.

Whenever someone changes something via Terraform, ex.: creating, decomming VMs, that state file need to be again sent to the others too, or else, when the others open their terraform, it will re-create the decommed VMs or destroy existing ones since they had old state file.

**My question:** Can Terraform pull down the newest state file by itself? Or manually, before doing anything, humans needs to be trusted to "always download the newest state file"? Humans will make mistakes..

# Answer
Did you try to use remote state storage like S3?

Also you can use Terraform Cloud to organzie workflow and have the shared state.
> 1 votes

---
Tags: terraform
---


# Question
Title: Docker container and their IPs on overlay (swarm) networks
I have a Docker overlay network which includes 2 containers and the loadbalancer for this network:

```
    "Containers": {
        "0ece6b958aa6b451478b5004ba1b1681ec91b718b99a0c2c88b79d5e8e135a66": {
            "Name": "flvw_nginx.1.v9ycohipbvizlagm840o7jd0d",
            "EndpointID": "0fba09c56d5098efb3a89645a5dce575a12b5f3496d3839ce4e34bb29bc60091",
            "MacAddress": "02:42:0a:00:16:06",
            "IPv4Address": "10.0.22.6/24",
            "IPv6Address": ""
        },
        "25b2988afe5d0cf02f718831bf12858b3dccb875a3609804273da02bd4b422ac": {
            "Name": "flvw_pimcore.1.j3m0i52sksg0fmhdawzhwwhei",
            "EndpointID": "a2ea9ddcfba20b05017dda7a79af542ab09b0172a74f61be9fbb861fad39d008",
            "MacAddress": "02:42:0a:00:16:03",
            "IPv4Address": "10.0.22.3/24",
            "IPv6Address": ""
        },
        "lb-flvw_app": {
            "Name": "flvw_app-endpoint",
            "EndpointID": "c4cf9798cd3d2d8ee11057176c9f7395fd81546860f97c2623cd7f5043104294",
            "MacAddress": "02:42:0a:00:16:04",
            "IPv4Address": "10.0.22.4/24",
            "IPv6Address": ""
        }
    },

```

This states:

```
pimcore = 10.0.22.3
nginx = 10.0.22.6

```

So I exec into the containers and the following commands.

On the pimcore container:

```
[root@25b2988afe5d pimcore]# ip a | grep -i "10.0.22"
    inet 10.0.22.3/24 brd 10.0.22.255 scope global eth0

[root@25b2988afe5d pimcore]# ping nginx
PING nginx (10.0.22.5) 56(84) bytes of data.
64 bytes from 10.0.22.5 (10.0.22.5): icmp_seq=1 ttl=64 time=0.062 ms
64 bytes from 10.0.22.5 (10.0.22.5): icmp_seq=2 ttl=64 time=0.052 ms
^C

```

On the nginx container:

```
[root@0ece6b958aa6 /]# ip a | grep -i "10.0.22"
    inet 10.0.22.6/24 brd 10.0.22.255 scope global eth0
[root@0ece6b958aa6 /]# ping pimcore
PING pimcore (10.0.22.2) 56(84) bytes of data.
64 bytes from 10.0.22.2 (10.0.22.2): icmp_seq=1 ttl=64 time=0.036 ms
64 bytes from 10.0.22.2 (10.0.22.2): icmp_seq=2 ttl=64 time=0.065 ms

```

Result:

Pimcore says its own IP is 10.0.22.3 (which corresponds to the network inspect output) and that nginx' IP is 10.0.22.5 (which does not correspond).

Nginx says its own IP is 10.0.22.6 (which corresponds to network inspect) and that pimcore's IP is 10.0.22.2 (which does not correspond).

Containers seem to see other containers "1 number off", in a way of speaking. Why?

My TCP/IP knowledge is definitely not high level, but this feels weird. Maybe someone can englighten me!

# Answer
When you `ping nginx` the docker swarm returns Virual IP of swarm service named "nginx" not the container ip.

This is is how internal load balancer works - you can have multiple replicas of "nginx" containers behind the "nginx" service.
> 3 votes

---
Tags: docker, networking
---


title: Route53 DNS records stopped resolving

text:
# Question
Title: Route53 DNS records stopped resolving
Last week, my TXT and other DNS records on Route53 stopped resolving:

https://dnschecker.org/#TXT/thuijls.net

The entries are set like:

What else could be missing here? As far as I'm aware, nothing has changed in the last week yet most of the entries are not being picked up. The domain was automatically extended earlier september if that makes a difference.

# Answer
When I lookup your domain thuijls.net on MXToolBox I get the following messageL

**Dns record not found**

They explain this to mean:

> More Information About Dns Record Published
> We did not find a SPF (TXT) record for the domain. Either you are using the deprecated type "SPF" record (in which case you will see a warning message for this alongside this message) or no record exists.
>
> For a SPF record to be valid, it must at least contain v=spf1

A TXT lookup for thuijls.net yeilds:

`Type Domain Name TTL Record`
`TXT thuijls.net 5 min v=spf1 include:spf.messagingengine.com ?all`

The problem is the ?all - you probably copy-pasted this from somewhere and the (non-standard) hyphen became a question mark.

Edit your TX record to read `v=spf1 include:spf.messagingengine.com -all` and all will be good.

Then you can add a 2nd TXT entry for your `"protonmail-verification-da....."` setting.
> 1 votes

---
Tags: amazon-web-services, dns
---


title: Azure DevOps extract unique visitors

text:
# Question
Title: Azure DevOps extract unique visitors
I am running a version of Azure DevOps and would like to track adoption.

Is it possible to get the number of unique visitors who have logged in / used Azure DevOps in the last 30 days, for instance?

# Answer
> 1 votes
I don't know of a built-in way but here are two hacks:

You can look at `https://dev.azure.com/{organization}/_settings/audit` , then set the date range to what you want and manually count.

An automated way of doing that could be to use the ADO REST API and send a Query to get the Audit log, then use a bit of code to tally up the unique users from the response.

---
Tags: azure-devops
---


title: Dev-Ops with bitbucket

text:
# Question
Title: Dev-Ops with bitbucket
I want to improve my current dev CI workflow, I would like to create a server with ISPConfig and some kind of CI tool like Jenkins

The catch is It has to all sit on 1 server, I see folks saying jenkins needs its own server but im just wondering is that a hard set rule or what.

Or has anyone got any tips on other tools to use which would work better with bitbucket or bitbucket pipelines?

My current setup has been a maze of bash scripts which get triggered when a dev pushes to the master branch but it's not very convenient for updates or adding new sites etc so id love to hear what other people are using to simplify the process of pushing to staging/dev .

# Answer
> 1 votes
I had this problem when I needed to build something at home.

What I did:

* installed docker in a Ubuntu server VM
* installed docker in a windows server 2016 VM (if you need windows containers like i did)

THEN: In the Ubuntu machine:

* I created a local registry running in a container (1 command :) )
* I spin up a container with a git server - I "linked" the configuration folder to a folder in my machine for persistence.
* I spinn a container with jenkins server (same trick with the folder)
* I spin up containers with slaves
* I have a database container attached to a nexus container for artifacts and build dependencies

In windows VM :

* I spin a jenkins slave container to build windows apps
* I spin IIS machines to test the built app

My home computer is 32 GB ram / core i7 and it works great , even with windows container databases (these are monstrous)

Tricky for me - work in progress - build a container in a slave running in a container . It can be done but I am on googling phase.

Advantages:

* if you use docker compose and you save your containers in the local registry you get to move your work easily . I always version my docker compose files in a github repo.
* you can pause the VM host and move everything in some other host.
* you can branch your things for having a test env by cloning the machine and copying the host folders

# Answer
> 0 votes
There is no hard rule to have jenkins to be running of a single server. However usually it may be better to have separate servers which fulfilling one purpose at a time as this gives some advantages

* you can focus only on dependencies for the particular service (e.g. Bitbucket and jenkins may require two different Java versions, whereas one installation could interfere with other)
* if you have all services on one server, for whatever reason the servers is not available - **none** of your services are available neither

There are a bunch of tools you can use i.e. not only Jenkins. I recommend to write down your needs/requirements and evaluate different options.

---
Tags: jenkins, build-pipeline, bitbucket
---


# Question
Title: Is there a Radiators for Jenkins and Bamboo?
I need to bring our companies rather old build radiators upto standard so it reflects what our current toolstack looks like. However, we're currently inside a weird limbo between using both Jenkins and Bamboo.

Is there an open source, off the shelf build radiator that can interact with both Bamboo and Jenkins? I've already had a crack at making my own system and decided that my JS skills are not up to the task.

Not essential but bonus points for pulling metrics from other sources like SonarQube and Jira...

# Answer
It really depends on what information you are trying to "Radiate":

For an off-the-shelf solution that integrates with everything you mentioned you could check out Prometheus. It has plugins for Jira, Bamboo, Jenkins, and SonarQube.

You could also opt for something quite a bit simpler and use the built in dashboards for the tools you are trying to display metrics for. Just put them on different tabs and use an extension such as Revolver.

If you want to get a little more customized you can check out Elastic Stack's Kibana. The book Agile Metrics in Action has several examples of how to capture and display Atlassian data inside of Kibana.
> 2 votes

---
Tags: jenkins, bitbucket, bamboo
---


# Question
Title: Azure DevOps Server: Why does a Visual Studio Test step take so long to run unit tests?
In an Azure DevOps Server build system installed on premises, I have a Build Pipeline with a Visual Studio Test step. The step is configured to discover and run XUnit unit tests in a DLL.

The step is defined via the GUI. Here are its properties:

```
Version: 2.*
Select tests using: Test assemblies
Test files:
    **\*test*.dll
    !**\*TestAdapter.dll
    !**\obj\**
Search folder: $(System.DefaultWorkingDirectory)
Select test platform using: Version
Test platform version: Latest

```

The step runs and passes successfully, identifying 18 tests. However, it's taking an unreasonable time to do so: ~51 seconds.

The run logs show that it's spending about 40 seconds right before launching `vstest.console.exe`, seemingly doing nothing. Please see this line in the full log, below:

```
2019-09-23T07:11:35.7961563Z ========================================================

```

**My Question** \- Why does the Visual Studio Test step spend so much time idling? What can I do to make it waste less time?

---

```
2019-09-23T07:11:32.4121152Z ##[section]Starting: run unit tests
2019-09-23T07:11:32.4135871Z ==============================================================================
2019-09-23T07:11:32.4136477Z Task         : Visual Studio Test
2019-09-23T07:11:32.4136755Z Description  : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
2019-09-23T07:11:32.4137066Z Version      : 2.143.8
2019-09-23T07:11:32.4137295Z Author       : Microsoft Corporation
2019-09-23T07:11:32.4137526Z Help         : [More information](https://go.microsoft.com/fwlink/?LinkId=835764)
2019-09-23T07:11:32.4138042Z ==============================================================================
2019-09-23T07:11:35.0894195Z SystemVssConnection exists true
2019-09-23T07:11:35.0937440Z SystemVssConnection exists true
2019-09-23T07:11:35.2406351Z Running tests using vstest.console.exe runner.
2019-09-23T07:11:35.2407337Z ======================================================
2019-09-23T07:11:35.2408408Z Test selector : Test assemblies
2019-09-23T07:11:35.2409125Z Test filter criteria : null
2019-09-23T07:11:35.2409694Z Search folder : C:\pool\_work\238\s
2019-09-23T07:11:35.2412135Z VisualStudio version selected for test execution : latest
2019-09-23T07:11:35.2758339Z Run in parallel : false
2019-09-23T07:11:35.2810059Z Run in isolation : false
2019-09-23T07:11:35.2813902Z Path to custom adapters : null
2019-09-23T07:11:35.2831140Z Other console options : null
2019-09-23T07:11:35.2883526Z Code coverage enabled : false
2019-09-23T07:11:35.2884892Z Diagnostics enabled : false
2019-09-23T07:11:35.2937817Z SystemVssConnection exists true
2019-09-23T07:11:35.3039161Z Run the tests locally using vstest.console.exe
2019-09-23T07:11:35.3040102Z ========================================================
2019-09-23T07:11:35.3075806Z Test selector : Test assemblies
2019-09-23T07:11:35.3094635Z Test assemblies : **\*test*.dll,!**\*TestAdapter.dll,!**\obj\**
2019-09-23T07:11:35.3095999Z Test filter criteria : null
2019-09-23T07:11:35.3114780Z Search folder : C:\pool\_work\238\s
2019-09-23T07:11:35.3125782Z Run settings file : C:\pool\_work\238\s
2019-09-23T07:11:35.3144368Z Run in parallel : false
2019-09-23T07:11:35.3158394Z Run in isolation : false
2019-09-23T07:11:35.3243577Z Path to custom adapters : null
2019-09-23T07:11:35.3244967Z Other console options : null
2019-09-23T07:11:35.3269743Z Code coverage enabled : false
2019-09-23T07:11:35.3273300Z Diagnostics enabled : false
2019-09-23T07:11:35.3286973Z Rerun failed tests: false
2019-09-23T07:11:35.3308152Z VisualStudio version selected for test execution : latest
2019-09-23T07:11:35.7961563Z ========================================================
2019-09-23T07:11:50.3641983Z ======================================================
2019-09-23T07:12:16.2956956Z [command]"C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" @C:\pool\_work\_temp\60ff2cb1-ddd1-11e9-a3ed-db0a48eb3ef5.txt
2019-09-23T07:12:16.9940929Z Microsoft (R) Test Execution Command Line Tool Version 15.9.0
2019-09-23T07:12:16.9941829Z Copyright (c) Microsoft Corporation.  All rights reserved.
2019-09-23T07:12:16.9942101Z
2019-09-23T07:12:16.9942399Z vstest.console.exe
2019-09-23T07:12:16.9942679Z "C:\pool\_work\238\s\REDACTED\bin\Release\REDACTED.Tests.dll"
2019-09-23T07:12:16.9942954Z /logger:"trx"
2019-09-23T07:12:16.9943238Z /TestAdapterPath:"C:\pool\_work\238\s"
2019-09-23T07:12:17.6998070Z Starting test execution, please wait...
2019-09-23T07:12:20.5147395Z [xUnit.net 00:00:00.6296658]   Discovering: REDACTED.Tests
2019-09-23T07:12:20.6880418Z [xUnit.net 00:00:00.8113825]   Discovered:  REDACTED.Tests
2019-09-23T07:12:20.6955571Z [xUnit.net 00:00:00.8192449]   Starting:    REDACTED.Tests
2019-09-23T07:12:21.9022641Z Passed   REDACTED 1
2019-09-23T07:12:21.9024375Z Passed   REDACTED 2
2019-09-23T07:12:21.9024811Z Passed   REDACTED 3
2019-09-23T07:12:21.9025086Z Passed   REDACTED 4
2019-09-23T07:12:21.9025460Z Passed   REDACTED 5
2019-09-23T07:12:21.9026429Z Passed   REDACTED 6
2019-09-23T07:12:21.9027211Z Passed   REDACTED 7
2019-09-23T07:12:21.9027581Z Passed   REDACTED 8
2019-09-23T07:12:21.9027938Z Passed   REDACTED 9
2019-09-23T07:12:21.9028266Z [xUnit.net 00:00:01.9910390]   Finished:    REDACTED.Tests
2019-09-23T07:12:22.3918748Z Passed   REDACTED 10
2019-09-23T07:12:22.3920336Z Passed   REDACTED 11
2019-09-23T07:12:22.3920912Z Passed   REDACTED 12
2019-09-23T07:12:22.3922185Z Passed   REDACTED 13
2019-09-23T07:12:22.3923090Z Passed   REDACTED 14
2019-09-23T07:12:22.3925098Z Passed   REDACTED 15
2019-09-23T07:12:22.3926473Z Passed   REDACTED 16
2019-09-23T07:12:22.3928334Z Passed   REDACTED 17
2019-09-23T07:12:22.3930288Z Passed   REDACTED 18
2019-09-23T07:12:22.4707999Z Results File: C:\pool\_work\238\s\TestResults\REDACTED_2019-09-23_10_12_21.trx
2019-09-23T07:12:22.4743977Z
2019-09-23T07:12:22.4746692Z Total tests: 18. Passed: 18. Failed: 0. Skipped: 0.
2019-09-23T07:12:22.4748446Z Test Run Successful.
2019-09-23T07:12:22.4764305Z Test execution time: 2.7962 Seconds
2019-09-23T07:12:23.0702219Z ##[section]Async Command Start: Publish test results
2019-09-23T07:12:23.3336706Z Publishing test results to test run '432'
2019-09-23T07:12:23.3338983Z Test results remaining: 18. Test run id: 432
2019-09-23T07:12:24.1461381Z Published Test Run : http://REDACTED/Runs#runId=432&_a=runCharts
2019-09-23T07:12:24.1474239Z ##[section]Async Command End: Publish test results
2019-09-23T07:12:24.1476508Z ##[section]Finishing: run unit tests

```

# Answer
The root cause turned out to be that there were over 100,000 files for the Visual Studio Step to scan for tests, in the `Search folder`.

To resolve, I pointed the search folder to a more specific bin folder down the folder tree. This reduced the number of files scanned and dramatically shortened the overall run time for the step.
> 1 votes

---
Tags: azure-devops, testing, unit-testing
---


# Question
Title: How to install Certbot via Ansible playbook?
This is how I install Certbot in a Bash script:

```
apt-get update -y && add-apt-repository ppa:certbot/certbot -y && apt-get update -y
apt-get upgrade nginx python-certbot-nginx -y

```

---

That's the playbook syntax I found to install Nginx in default conf:

```
- hosts: localhost
    connection: local
    become: true

tasks:
    - name: install nginx
    apt: pkg=nginx state=installed update_cache=true

```

Is there no Ansible task to install Certbot as well, instead the above "heavier" Bash code?

# Answer
> 5 votes
Here is the direct Ansible translation of your bash install script:

```
- apt_repository:
    repo: 'ppa:certbot/certbot'

- apt:
    name: "{{ item }}"
    update_cache: yes
  with_items:
    - nginx
    - python-certbox-nginx

```

# Answer
> 4 votes
For Ubuntu 16.04 should be something like this

```
tasks
- name: Install Certbot.
  package: "name=letsencrypt state=present"

```

Or you can use a role from Github: https://github.com/geerlingguy/ansible-role-certbot

# Answer
> 0 votes
This is how I do it:

```
- name: Add certbot repository
  apt_repository:
    repo: 'ppa:certbot/certbot'

- name: Install Certbot's Apache package
  apt:
    name: python-certbot-apache
    state: present

```

# Answer
> -1 votes
chipping in with ansible task to install letsencrypt certbot on debian stretch

```
---
- name: Add "stretch-backports" to APT sources (required for Certbot)
  become: yes
  apt_repository:
    repo: deb http://httpredir.debian.org/debian stretch-backports main
    state: present
    update_cache: yes

- name: "Check LetsEncrypt/Certbot is installed"
  become: yes
  apt:
    name:
      - certbot
      - python-certbot-apache # https://github.com/certbot/certbot/issues/3854
    state: present
    default_release: stretch-backports
    cache_valid_time: 3600
    update_cache: yes

```

---
Tags: ansible
---


# Question
Title: How to access hostvars from goss test with molecule
I have a molecule scenario that creates 3 docker instances, each having a different configuration from the same role. The three communicates with each other as I put the instances IP addresses in their respective configuration files.

I need to test that the correct IP address is set the config files of each instance.

How can I access ansible group and host variables from goss?

# Answer
Since goss can use variables from a yaml or json file, I just modified the `verify.yml` example playbook to export this variable and use it in the goss command.

```
- name: Verify
  hosts: all
  become: true
  vars:
    [...]
    goss_vars: "{{ goss_test_directory }}/vars"
  tasks:
    [...]
    - name: Register hostvars
      copy:
        content: "{'hostvars': {{ hostvars }}}"
        dest: "{{ goss_vars }}"

    - name: Execute Goss tests
      command: "{{ goss_bin }} --vars {{ goss_vars }} -g {{ item }} validate --format {{ goss_format }}"
      register: test_results
      with_items: "{{ test_files.stdout_lines }}"
      failed_when: false

    [...]

```

Then the variable is available in goss test files as `{{ .Vars.hostvars }}`.
> 0 votes

---
Tags: molecule
---


# Question
Title: How can I assign an IP address to a docker container different from the host's IP?
**Objective**: Assign fixed IP address to Docker container (Unifi Controller instance).

**History**: I fetched the unifi controller image from Docker Hub to my Synology Docker host. The container of the unifi controller runs properly if I attach it to the network of the host (not the default bridge). That blocks though multiple ports I need for other services (in the future). Thus it would be much easier to assign that unifi container with it's own designated IP --\> 192.168.2.2 (than changing all the ports).

I guess that is a cinch to deal with for most in here.

**Indications of my misconception**:

* I created my own network in the Synology Docker GUI (with the subnet 192.168.2.0/23 and gateway 192.168.2.1)
* I assigned it to a new unifi-controller container instance.
* The GUI suggests to map the host ports with "auto" to the container ports, **which indicates me that I do NOT understand completly the network concept** of Docker. Because if I'm able to set an individual IP, why should it allow to map host ports (to container ports)?

**Questions**:

* Why can I not assign a fixed IP to the container in the GUI?
* What am I conceptually missing?
* If not in the GUI, how can I accomplish it over SSH/Bash?

# Answer
**SOLUTION APPROACH**

The approach is to create a macvlan. This will create a virtual adapter, that is allowed to lease an IP address from the subnet defined.

**1) Activate Promiscous Mode**

For the virtual adapter one must enable promiscuous mode in the network. For Unifi controllers, do a SSH to your gateway and set:

```
ifconfig [interface] promisc

```

Some network gateways/controllers have set promiscuous mode by default. Continue with step 2 before you're wasting time to find that out.

**2) Create macvlan**

Then one can create the macvlan.

```
$ docker network create -d macvlan \
  --subnet=192.168.2.0/23 \
  --ip-range=192.168.2.5/25 \
  --gateway=192.168.2.1 \
  --aux-address="my-router=192.168.2.10" \
  -o parent=eth0 unifinet

```
* Limiting the IP range: use `--ip-range` to scope the possible IPs to lease.
* Avoiding certain IPs: The `--aux-address` marks my Synology host, which should never ever battle for that IP address (It's marked in the unifi controller's DHCP service as a fixed IP anyways).
* the `-o parent` is your network interface you want to attach your macvlan. In my case `unifinet`.

Check with `docker network ls` if the macvlan has been properly created.

```
NETWORK ID          NAME                DRIVER              SCOPE
c49094a4c914        bridge              bridge              local
b2315de1aa7e        host                host                local
c124eda0f9d2        none                null                local
a60da50f0d12        unifinet            macvlan             local

```

**3) Start Docker Container**

While you define the image (--name UnifiController jacobalberty/unifi) assign the freshly created macvlan (--network unifinet) to the container (UnifiController). It will grab the latest image respectively the latest Unifi Controller version available from Docker Hub.

```
docker run -dit --network unifinet --name UnifiController jacobalberty/unifi

```

Verify if the container has been assigned to the network in the container section of the JSON config.

```
sudo docker network inspect unifinet.

[
    {
        "Name": "unifinet",
        "Id": "a60da50f0d1229d1a3c76210141e0c81567c17daf6c2b49d4f1c83d5ec9f02b3",
        "Created": "2019-10-04T14:39:37.377311991+02:00",
        "Scope": "local",
        "Driver": "macvlan",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "192.168.2.0/24",
                    "Gateway": "192.168.2.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "b819bf1d6f3af459825a1a7b58f9a44e15e6e09489e7bf50653ed8e1e176fd73": {
                "Name": "UnifiController",
                "EndpointID": "41a048034ad63e48b46b58aed65661c9eaa2bf6937d3eebacefde4478ad26cce",
                "MacAddress": "02:42:c0:a8:02:02",
                "IPv4Address": "192.168.2.2/24",
                "IPv6Address": ""
            }
        },
        "Options": {
            "parent": "eth0"
        },
        "Labels": {}
    }
]

```

The IP 192.168.2.2 is assigned. The container workes properly and stable.

**4) Optional/Reminder**

If the IP was not assigned respectively one cannot connect to it for any service, the follow up on the promiscious mode setting as described in Step 1.
> 2 votes

---
Tags: docker
---


# Question
Title: How to set a Circle CI environment variable for a project?
I've set the environment variable with a "context" (I have no idea if thats the right place - it's really confusing!)

None the less, it's now doing a build - and I don't see the environment variables set (I'm using the SSH build to inspect the vars with `getenv`).

Why is it not being picked up? Am I meant to set an environment variable by some other means rather than a "context"?

# Answer
> 1 votes
https://circleci.com/docs/2.0/env-vars/#setting-an-environment-variable-in-a-project

Seems "contexts" can be used but must then also be set to "reference" the context in the build yaml config file.

Instead to set a environment variable for a project: Jobs \> Cog / gear icon of a project \> Build settings \> Environment variable

---
Tags: cicd, circleci
---


# Question
Title: How to pass Gitlab env var with sensitive characters to bash script?
I have the following situation: I'm storing sensitive data in Gitlab's environment and then passing these variables to a deployment script in bash.

Gitlab's env var:

```
Key: sensitive_var
Value: aaa$bbb*%

```

Then in the `.gitlab-ci.yml` file the env vars are passed to the deployment script as follows:

```
.deploy:
  script:
    - deploy.sh PASS=${sensitive_var}

```

The deployment script parses the arguments as follows:

```
for arg; do
  TEMP_VAR=$(echo ${arg} | sed -e 's?=? ?')
done

```

The problem is that when we check the output of the `deploy.sh` script the variable's value gets expanded resulting in `aaa*%`.

Solutions tried so far:

* Storing the value in Gitlab's environment with single quotes `'` but then the value was `'aaa*%'`.
* Passing the environment variable with double quotes:

```
.deploy:
  script:
    - deploy.sh PASS="${sensitive_var}"

```

```
.deploy:
  script:
    - deploy.sh PASS="$sensitive_var"

```

But the result was the same: `aaa*%`.

* Combination of the above solutions didn't work either.

Why is this happening, and how I can fix it?

# Answer
> 4 votes
Surprisingly indeed GitLab works weirdly with the env vars.
It evaluates them internally, though it shouldn't in my view.

Value set as (in project Settings \> CI/CD \> Variables): `'aaa$bbb*%'`
becomes `'aaa*%'` (with single quotes left!)

Couple of of proof links:

## Workarounds:

1) You can duplicate dollar ($) sign, i.e.: `aaa$$bbb*%`
(as mentioned on the first link above).

2) Use base64 to encode and decode the value.

Do this first:

```
$ echo 'aaa$bbb*%' | base64
YWFhJGJiYiolCg==

```

Then store this value in project Variables.

And then in your script decode it:

```
PASS=$(echo ${sensitive_var} | base64 -D)

```

# Answer
> 2 votes
This is not answering only the question but also a bit more (I put this here for reference):
I found out a decent way to tackle this and some complications coming afterwards... (after hours and hours of fiddling) - e.g. to execute the script in an SSH session:

```
deploy job:
  variables:
    # the variables could come from anywhere (e.g. GitLab Settings - environment variables)
    SSH_DIR: '/srv/app/'
    SCRIPT_CMD: |
      echo "Double quotes are safe"
      echo 'Single quotes are safe'
      echo "Code executions are safe -" `whoami`@`hostname` $$(date)
      # the '$$' is needed as otherwise GitLab itself tries variable substitution

  # Here's the magic to get the code from the GitLab variable into a bash variable and then even executed on an SSH session
  script:
    # Put gitlab variable into shell variable to improve quote handling
    - CMD=$SCRIPT_CMD
    # Print for debugging
    - echo -e "Executing:\n$CMD"
    - CMD="set -e; cd $SSH_DIR; $CMD" # 'set -e' = stop on error
    # redirect variable to stdin of ssh bash (the -x causes bash to print each command)
    - ssh -p $SSH_PORT $USERNAME@$SSH_HOST "bash -x" <<< "$CMD"

```

More detailed example here.

# Answer
> 0 votes
One should use protected variables in my opinion. One could configure them in the pipeline settings and subsequently call them inside the pipeline. Using base64 formats should be avoided as these could be easily decrypted.

https://docs.gitlab.com/ee/ci/variables/

---
Tags: gitlab-ci-runner, bash, yaml, environment-variables
---


# Question
Title: How to get hostname from node-exporter in Docker Swarm?
I have setup my monitoring with the *cadvisor, node-exporter, prometheus, grafana* stack in my clustered environment using docker swarm.

What is the easiest way to get my actual hostnames picked up by node-exporter, so I can configure my dashboards to filter by node names in grafana?

For my kubernetes environments this works ootb, because all metrics are extended by the `kubernetes_io_hostname`, that contains the actual hostname.

The only solution to this I found, so far, was to introduce my own metric, that creates the combination between node ids and node names (like done by https://github.com/bvis/docker-node-exporter). But I really like to use the vanilla image (for update reasons, for example) and also mounting some file with the configuration on all machines seems to be too much effort to do this.

Is there an easy way to get the `hostname` of the node into my node-exporter metrics? Maybe by just mounting the hosts `/etc/hostname` file to a specific place in the node-exporter container, or something like that?

# Answer
You just have to use the hostname on your docker compose file like this:

```
hostname: whatever-{{.Node.Hostname}}

```

Note that you have to prepend something to the hostname to avoid hostnames duplicates.
> 3 votes

---
Tags: docker-swarm, prometheus
---


title: Kubernetes and Jenkins

text:
# Question
Title: Kubernetes and Jenkins
I started building CI/CD pipeline using two Kubernetes clusters and one Jenkins. Two clusters are totally isolated as one is in the test environment and other in production.

After searching from many blogs, I come up with these approaches:

1. Setup Jenkins master in production and one Jenkins slave/agent in the testing environment.
2. Make Kubernetes api-server of test environment accessible in production environment with proper firewall whitelisting and setup jenkins in production to deploy in both environments.

I like to know which approach is better.

# Answer
For the best practices will be great to install Jenkins master on the lower Environments (best Dev env) and to leave Production in peace with only necessary things (with your end-user application and their dependencies).

The reason why is because production means production and if something will happen with Jenkins or if you will allow a lot of people accessing Jenkins in production you increase your failure rate.

Will be better to install Jenkins separated from production and to do VPC Peering between clusters and after you can create a fully CI/CD pipeline where you can create a Jenkins slave/agent which will deploy your application to PROD.

PROD suppose to be more focused to an end Product result. You didn't specify which cloud provider do you use so, I can't help too much... Hope this will clarify your vision.
> 1 votes

---
Tags: jenkins, kubernetes, deployment
---


# Question
Title: How to setup a permanent storage volume for EC2 instance?
I created an EC2 instance of `r5ad.xlarge` and installed Jenkins on it. However, after a few jobs I configured and ran I saw a warning about no space left. Then realised Jenkins was installed on 8 GB drive and that it's a non-persistent drive.

1st try: After reading on the subject I changed the 2 volumes of the machine with `modify-instance` to be `DeleteOnTerminate` to false and copied Jenkins to the bigger disk and configured auto-mount, following this and this. Then I stopped and started the machine and everything got deleted from the disk.

2nd try: So I defined a new EBS volume (SSD gp2)(with this article) and attached it to the instance. Again with `DeleteOnTerminate=false`, formatted then setup auto mount and copied Jenkins directory. Stopped and Started the machine, again everything gone...

What am I doing wrong? How do you make the attached volume persistent?

# Answer
That *150GB disk* that `r5ad.xlarge` comes with is known as *ephemeral storage* \- its content is wiped when you stop the instance. If you need persistent storage you can use `r5a.xlarge` that doesn't have the ephemeral storage disk and simply give it a bigger EBS disk.

Either you can increase the size of the root disk during instance creation or create another EBS and use that. EBS volumes are persistent, they don't go away when you *stop* the instance (they may be deleted if you *terminate* the instance though).

If you don't see the previous content of EBS volume after a reboot make sure you're properly mounting it during boot (hint: `/etc/fstab`).

Hope that helps :)
> 2 votes

---
Tags: amazon-web-services, amazon-ec2, storage, amazon-ebs
---


# Question
Title: jenkins stuck "Please wait while Jenkins is getting ready to work"
I've been trying to get a simple jenkins master node to start but it keeps getting stuck at the "Please wait while Jenkins is getting ready to work".

My setup is nothing exotic: jenkins is installed inside a LXC hosted by a proxmox 6 instance. 10Go of RAM, 4CPU, 10G of hdd, debian 9

I increased the lxc specs just in case (I started with 4G of RAM and 2 CPU)

I tried a lot of different things such as:

1. Stop / start the jenkins soft
2. Stop / start the container
3. installing using debian package
4. Install using the war (see my service file below, only used for the war install style)
5. Installing on centos 7
6. Installing on qemu

Service file:

```
[Unit]
Description=Jenkins Daemon

[Service]
ExecStart=/usr/bin/java -jar /home/jenkins/jenkins.war --logfile=/home/jenkins/jenkins.log
User=jenkins

[Install]
WantedBy=multi-user.target

```

The log doesn't contain any error except a WARNING :

```
2019-10-03 12:05:38.516+0000 [id=1] WARNING o.e.j.s.handler.ContextHandler#setContextPath: Empty contextPath

```

But it always ends at being stuck at this page.

I've searched the web but my google fu seems bad today so here I am.

This sounds crazy to me, jenkins install is supposed to be easy and quite forward, but I can't make it works ...

\[Edit\] Might be a connection issue, we saw jenkins trying to connect to an ec2 instance... we have some proxy, but did not found the correct options (in Java command) to get through it, any tips welcome

# Answer
> 1 votes
OK, a good week-end does its job as usual: My issue was indeed caused by my company proxy (a good thing would be to add this requirement in the jenkins doc, i might have missed it anyway). So the fix is actually simple, just need to add the correct options to the jenkins start command, as in:

```
-Dhttp.proxyHost=<your_proxy_ip_here> -Dhttp.proxyPort=<your_proxy_port_here> -Dhttps.proxyHost=<your_proxy_ip_here> -Dhttps.proxyPort=<your_proxy_port_here>

```

Full example:

```
/usr/bin/java -Djava.net.preferIPv4Stack=true -Dhttp.proxyHost=10.21.92.40 -Dhttp.proxyPort=3128 -Dhttps.proxyHost=10.21.92.40 -Dhttps.proxyPort=3128 -jar /usr/share/jenkins/jenkins.war --webroot=/var/cache/jenkins/war --httpPort=8080

```

tcpdump was the savior here (+ a colleague :) )

---
Tags: jenkins, containers
---


# Question
Title: Reading two Repos in Azure Build pipeline
I created a Build pipeline in Azure using classic editor and selected empty job instead of using any template. This way provides option to connect only one repository at once. My requirement is to connect two repositories at once in the build pipeline and bring files from both of them to run some testing. There are so many templates available to chose from but I am not sure which is the right one to select, with which I can connect two repositories at the same time. I did Google for help and found that I could use Artifact with the combination of NuGet or nmp. I don't know how does that work.

# Answer
> 5 votes
I also use the classic editor, and I have implemented this for our builds. My use case was that I have a set of common set of build/release scripts stored in their own repository, and I want the ability to bring them into the application builds that I am executing.

I created a custom **Task Group** with an **Inline PowerShell script** to do this:

```
# Reference: https://blog.rsuter.com/script-to-clone-all-git-repositories-from-your-vsts-collection/
Write-Host "Cloning MyRepo from Azure DevOps..."

$sourcesDir = $env:SYSTEM_DEFAULTWORKINGDIRECTORY
Write-Host "Source directory: $sourcesDir"
$repoDir = Join-Path -Path "$sourcesDir" -ChildPath "MyRepo"
Write-Host "Target directory: $repoDir"

$url = "https://dev.azure.com/my_org/my_team_project/_git/MyRepo"

# Since using PAT, don't need real username, just non-empty username. Use "x".
$username = "x"
$pat = $env:PAT # passed into inline script as environment variable
$credentials = ("{0}:{1}" -f  $username, $pat)
$urlWithCreds = $url -Replace "://", ("://{0}@" -f $credentials)
Write-Host "Clone URL: $urlWithCreds"

git --version
git clone --progress $urlWithCreds
Set-Location "$repoDir"
git checkout master
git log -1 # show the latest commit

```

You will have to create a PAT with code permissions and pass it into the script as an environment variable. This task will clone the repository to a directory of the same name *within* your existing repo.

Putting it in a task group allows me to manage it in one place, and then all my more complicated build scripts can be under version control.

There is an active feature request for the capability to source from multiple repos. It looks like it is in the roadmap: https://developercommunity.visualstudio.com/idea/365522/allow-tfs-build-to-depend-on-multiple-repositories.html

---
Tags: azure-devops, azure, build-pipeline, azure-pipelines, artifacts
---


# Question
Title: How do I specify a database name for a SQL DB Deploy task?
Using Azure DevOps, I'm trying to setup a SQL DB Deploy task as a part of a new release pipeline. Database Name is a required field for this task. However, the field is read only, so I cannot manually input it.

There is a tooltip stating that "This setting is linked to the 'Database name' (Parameters.DatabaseName) pipeline parameter." I'm not sure what this means, but it is perhaps the key to my solution.

I tried adding a Pipeline variable named DatabaseName and also Parameters.DatabaseName, but this did not help.

How can I specify a value for the Database Name property?

# Answer
> 1 votes
It seems like Database Name is linked as a Process Parameter.

Under the "SQL Deployment" process, you should see the variable there in an editable state. You can either fill in the value there, or you can unlink and fill in the value in the "SQL DB Deploy" task.

---
Tags: azure-devops
---


# Question
Title: jenkins credentials not used as expected when cloning git repo
I am trying to setup a jenkins pipeline for my IaC code validation, for this I'd like jenkins to clone a gitlab (hosted) repository, for this purpose I setup a credential using username / key type, i used a jenkins user + a generated rsa key (same key on master and slave for simplicity).

When logged in the master or slave I can git clone using the following command and result :

```
root@jenkins-slave-1 [~]: su - jenkins
jenkins@jenkins-slave-1:~$ cd /tmp/
jenkins@jenkins-slave-1:/tmp$ git clone git@l-gitlab1.domain:salt/salt_states.git
Cloning into 'salt_states'...
remote: Enumerating objects: 1177, done.
remote: Counting objects: 100% (1177/1177), done.
remote: Compressing objects: 100% (483/483), done.
remote: Total 1177 (delta 676), reused 1153 (delta 665)
Receiving objects: 100% (1177/1177), 1.64 MiB | 0 bytes/s, done.
Resolving deltas: 100% (676/676), done.
jenkins@jenkins-slave-1:/tmp$

```

I can see the correct entry in auth log on gitlab instance, the one saying that a pubkey authentication was accepted.

BUT, when I run the pipeline (that is using the same credentials), it fails with this message :

```
Cloning the remote Git repository
Honoring refspec on initial clone
Cloning repository git@l-gitlab1.domain:salt/salt_states.git
 > git init /home/jenkins/workspace/salt_build_test # timeout=10
Fetching upstream changes from git@l-gitlab1.domain:salt/salt_states.git
 > git --version # timeout=10
using GIT_SSH to set credentials jenkins user's key (same key on master and slave(s))
 > git fetch --tags --progress -- git@l-gitlab1.domain:salt/salt_states.git +refs/remotes/origin/dev # timeout=20
ERROR: Error cloning remote repo 'origin'
hudson.plugins.git.GitException: Command "git fetch --tags --progress -- git@l-gitlab1.domain:salt/salt_states.git +refs/remotes/origin/dev" returned status code 128:
stdout:
stderr: Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,password).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

```

The gitlab auth log is showing that the authentication tried to use a password auth mode, not a pubkey. ...

```
Oct  8 11:49:23 l-gitlab1 sshd[18490]: Failed password for git from 172.27.200.46 port 56750 ssh2
Oct  8 11:49:23 l-gitlab1 sshd[18490]: Connection closed by 172.27.200.46 port 56750 [preauth]

```

Any ideas about what I am doing wrong ?

The error:

\[Edit\] I added a screenshot as my explanation are not very clear :(

# Answer
> 1 votes
The Gitlab plugin requires an auth token, you're trying to use an SSH key.

https://github.com/jenkinsci/gitlab-plugin#global-plugin-configuration

"PLEASE NOTE: This auth configuration is only used for accessing the GitLab API for sending build status to GitLab. It is not used for cloning git repos. The credentials for cloning (usually SSH credentials) should be configured separately, in the git plugin."

---
Tags: jenkins, jenkins-pipeline, git
---


title: How to test a Terraform Configuration?

text:
# Question
Title: How to test a Terraform Configuration?
If you had a Terraform configuration that had a moderate degree of complexity, how would you write tests around the configuration that could be executed as part of a Continuous Integration / Continuous Delivery pipeline?

As an example, you may have a multi-cloud configuration that specifies the following desired state:

* Azure Container Services to host Docker in Azure
* Azure Blob Storage
* SQL Azure
* EC2 Container Service to host Docker in AWS
* Amazon S3 Storage Service
* Amazon RDS SQL Server database

Potentially `terraform apply` could create the above from scratch, or transition from a partially deployed state to the above desired state.

I am aware that Terraform splits its work into the **execution plan** stage and the **application** phase which actually makes changes to the target architecture. Can this be used to write tests against the execution plan, if so are there frameworks to help write these?

# Answer
There is currently no full solution to this integrated into Terraform, but there are some building blocks that could be useful to assist in writing tests in a separate programming language.

Terraform produces state files in JSON format that can, in principle, be used by external programs to extract certain data about what Terraform created. While this format is not yet considered officially stable, in practice it changes infrequently enough that people have successfully integrated with it, accepting that they might need to make adjustments as they upgrade Terraform.

What strategy is appropriate here will depend a lot on what exactly you want to test. For example:

* In an environment that's spinning up virtual servers, tools like Serverspec can be used to run tests from the perspective of these servers. This can either be run separately from Terraform using some out-of-band process, or as part of the Terraform apply using the `remote-exec` provisioner. This allows verification of questions like "can the server reach the database?", but is not suitable for questions such as "is the instance's security group restrictive enough?", since robustly checking that requires accessing data from outside of the instance itself.
* It's possible to write tests using an existing test framework (such as RSpec for Ruby, `unittest` for Python, etc) which gather relevant resource ids or addresses from the Terraform state file and then use the relevant platform's SDK to retrieve data about the resources and assert that they are set up as expected. This is a more general form of the previous idea, running the tests from the perspective of a host *outside* of the infrastructure under test, and can thus collect a broader set of data to make assertions on.
* For more modest needs, one can choose to trust that the Terraform state is an accurate representation of reality (a valid assumption in many cases) and simply assert directly on that. This is most appropriate for simple "lint-like" cases, such as verifying that the correct resource tagging scheme is being followed for cost-allocation purposes.

There is some more discussion about this in a relevant Terraform Github issue.

In the latest versions of Terraform it is strongly recommended to use a remote backend for any non-toy application, but that means that the state data is not directly available on local disk. However, a snapshot of it can be retrieved from the remote backend using the `terraform state pull` command, which prints the JSON-formatted state data to stdout so it can be captured and parsed by a calling program.
> 24 votes

# Answer
We recently open sourced Terratest, our swiss army knife for testing infrastructure code.

Today, you're probably testing all your infrastructure code manually by deploying, validating, and undeploying. Terratest helps you automate this process:

1. Write tests in Go.
2. Use helpers in Terratest to execute your real IaC tools (e.g., Terraform, Packer, etc.) to deploy real infrastructure (e.g., servers) in a real environment (e.g., AWS).
3. Use helpers in Terratest to validate that the infrastructure works correctly in that environment by making HTTP requests, API calls, SSH connections, etc.
4. Use helpers in Terratest to undeploy everything at the end of the test.

Here's an example test for some Terraform code:

```
terraformOptions := &terraform.Options {
  // The path to where your Terraform code is located
  TerraformDir: "../examples/terraform-basic-example",
}

// This will run `terraform init` and `terraform apply` and fail the test if there are any errors
terraform.InitAndApply(t, terraformOptions)

// At the end of the test, run `terraform destroy` to clean up any resources that were created
defer terraform.Destroy(t, terraformOptions)

// Run `terraform output` to get the value of an output variable
instanceUrl := terraform.Output(t, terraformOptions, "instance_url")

// Verify that we get back a 200 OK with the expected text
// It can take a minute or so for the Instance to boot up, so retry a few times
expected := "Hello, World"
maxRetries := 15
timeBetweenRetries := 5 * time.Second
http_helper.HttpGetWithRetry(t, instanceUrl, 200, expected, maxRetries, timeBetweenRetries)

```

These are integration tests, and depending on what you're testing, can take 5 - 50 minutes. It's not fast (though using Docker and test stages, you can speed *some* things up), and you'll have to work to make the tests reliable, but it is well worth the time.

Check out the Terratest repo for docs and lots of examples of various types of infrastructure code and the corresponding tests for them.
> 16 votes

# Answer
As an update to this question, there is now Kitchen-Terraform which allows the testing of Terraform Configuration files without breaking production environments. The repository also includes a few examples for different Terraform providers.
> 13 votes

# Answer
In addition to all the other options mentioned, I would like to mention that InSpec 2.0 added support for cloud provider APIs. Basically, you can continue writing the IaC with Terraform, then write compliancy checks with InSpec for your cloud resources. In addition, InSpec supports writing tests for individual machines if you ever need it.

Here is an article from Christoph Hartmann (co-creator of Inspec) on how to use Inspec with Terraform: https://lollyrock.com/articles/inspec-terraform/
> 7 votes

# Answer
On Aws-Side there is https://github.com/k1LoW/awspec \- it should be possible, to feed in terraform.state and test, wheter terraform applied correct.

But I think, beyond testing on low level the tool, you used, it's probably a better idea, to think about how to test whole infrastructures.

We're discussing around this idea here:

https://github.com/DomainDrivenArchitecture/dda-cloudspec/blob/development/README.md

For testing invariants upfront, I do not know a ready to use solution ...

We did some experiments using a mix of `terraform plan -out=plan.dump` and `grep` for the absence of element names. There is a discussion about a more accessible plan format here: github.com/hashicorp/terraform/issues/11883

But at the moment we're using a manual plan review process for important parts of our infrastructure.
> 5 votes

# Answer
I saw this elegant, low tech method to test Terraform suggested by apparentlymart in a GitHub issue thread. It's not a appropriate for every situation but it's great for verifying module logic.

Create a root module that includes the module under test and verifies the under-test outputs. Here is a simple example using two files:

* `main.tf` that will run the tests
* `simple_module/outputs.tf` that represents a module under test

## ./main.tf

```
terraform {
  required_version = ">= 0.12"
}

module "simple_module" {
  source = "./simple_module"
}

locals {
  expected = 1
  got      = module.simple_module.module-returns-1
}

# Test Output
output "expect-1" {
  value = upper(local.expected == local.got)
}

output "expect-other" {
  value = "other" == local.got ? upper(true) : "FALSE. Got ${local.got}"
}

```

## ./simple\_module/outputs.tf

```
output "module-returns-1" {
  value = 1
}

```

## Run the tests

```
terraform init
terraform apply -auto-approve

```

```
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

expect-1 = TRUE
expect-other = FALSE. Got 1

```
> 0 votes

---
Tags: testing, terraform
---


# Question
Title: How to analyze the file activities due to process?
I need to analyze the file i/o operations caused due to a process. So I would like to know, which all files are affected by this program. Is there a tool for this particular use case?.

One of our debugging for a slow URL processing got the culprit being the file being read each time a URL was processed by the system. So I would like to get recommendations for a tool which would monitor the file activities per process. Any suggestions?

# Answer
> 1 votes
You can use `lsof` with the `-p` flag, like this : `lsof -p $PID` .

# Answer
> 0 votes
mainly you use ps for evaluating the pid of the process and use the proc filesystem to see the used files by the specific process. https://www.cyberciti.biz/faq/howto-linux-get-list-of-open-files/

---
Tags: linux
---


# Question
Title: How to check accepted cipher suites by an Azure App Service?
I am looking for a way to reliably check accepted cipher suites by an Azure App Service. On standard virtual machine I just remote to it and execute PowerShell command: `Get-TlsCipherSuite` which provides a list of currently accepted keys or check the system's registry.

I could pen test app service by ssllabs.com however I cross checked their results against results from `Get-TlsCipherSuite` for a certain machine and it's not 100% accurate: ssllabs didn't detect suites with Pre Shared Keys.

My questions:

* how to query Azure App Service accepted cipher suite?
* how do browsers know which cipher suites are available to it when it tries to go to a given domain?
* if that's not obvious out of answers to previous questions why ssllabs doesn't detect suites using PSK?

# Answer
Not sure if this is the right answer but let me give it a try with the limited knowledge i have on this.

1. these are the resources that I found when I am looking up for tls enabling for my app service. please see if any of this works. msdn how to configure ciphersuite for app service enforce tls vrsion
2. The client (browser) gives a list of cipher suites it can handle to the server and the server selects one, the decision is passed on to the client during the handshake.
3. so all the servers won't accept all the cipher suites, it based on server config to choose.
> 1 votes

---
Tags: azure, encryption, ssl
---


# Question
Title: How to update nodejs on Jenkins master node?
I am in the middle of trying to implement a build/deploy job in Jenkins by which the latest version of source code is pulled from git, built and then deployed to GCP. So far I am able to retrieve the lastest code from GIT but have run into issues when tyring to build. The application being built/deployed is an Angular 7 web site using npm for package management. So the normal steps would be to run npm install to make sure the packages are installed and then ng build -- prod to build the app.

My issue is that the version of npm/nodejs in the master node is out of date and not compatable with Angular 7. Even just trying the command to gert the current ng version fails:

```
+ /usr/bin/ng v
You are running version v8.1.4 of Node.js, which is not supported by Angular CLI v6.
The official Node.js version that is supported is 8.9 and greater.

```

I added a call to npm version to check the npm version is is trying to use:

```
+ echo current user
current user
+ whoami
jenkins
+ npm -v
5.0.3

```

To investigate the issue I used putty to remote onto the jenkins box and ran npm -v and ng v to see what is installed and get:

```
jenkins@****-int1:~$ npm -v
6.5.0-next.0

```

Which is different to that reported by npm -v in the shell script.

```
jenkins@****:~$ ng v

     _                      _                 ____ _     ___
    / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
   / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
  / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
 /_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|
                |___/

Angular CLI: 7.1.4
Node: 11.6.0
OS: linux x64
Angular:
...

Package                      Version
------------------------------------------------------
@angular-devkit/architect    0.11.4
@angular-devkit/core         7.1.4
@angular-devkit/schematics   7.1.4
@schematics/angular          7.1.4
@schematics/update           0.11.4
rxjs                         6.3.3
typescript                   3.1.6

```

So it should have nodejs 11.6, I ran node -v to double check:

```
jenkins@****:~$ node -v
v11.6.0

```

BUT if I run nodejs -v I get:

```
jenkins@****:~$ nodejs -v
v8.1.4

```

I thought that node and nodejs were the same thing? so why would I have two different versions installed and how to fix this?

To double check what my Jenkins script is using I added a call to node -v in the script and as expected it reports the older version:

```
+ node -v
v8.1.4

```

I'm not a DevOps person, I'm C# back end developer who's been lumbered with sorting this out due staff changes. I know very little about Linux, Jenkins or this stuff in general.

I tried to update the node installation via the Jenkins Shell script using

```
sudo npm cache clean -f
sudo npm install -g n
sudo n stable

```

But these were refused due to access rights (no sudo pwd I presume) and I don't know if that would've fixed the issue anyway.

Any help?

# Answer
I would have a look at the Tool Auto-Installation which allows Jenkins to install the tooling you need.

> Lets you configure tools so that agents will install them on demand whenever running a job that needs them. This could be especially useful when running a large farm of agents in a cloud which all start with a minimal operating system configuration.

Speaking of that also checkout the NodeJS Plugin which

> Provides NodeJS auto-installer, allowing to create as many NodeJS installations "profiles" as you want. The auto-installer will automatically install a given version of NodeJS, on every jenkins slave where it will be needed Allows to install globally some npm packages inside each installations, these npm packages will be made available to the PATH
> 1 votes

---
Tags: jenkins, node.js
---


# Question
Title: How do I get Azure Release Pipelines to move a .jar from a Build Pipeline job and place it on a Linux VM?
I know Azure release pipelines can deploy to on-premise servers and other cloud VMs. My pom.xml file enables Maven to create a .jar file. My Azure build uses this pom.xml file. The Azure build works fine by itself; there are no errors when it runs. I want to place the .jar file on to a Linux server via either the same Azure build pipeline job or a different Azure Release Pipeline. How do I do this?

I've looked at documentation. When I changed the build with a `pathToPublish:` stanza in my Azure build job, the job completes, but there is a warning:

> \[warning\]Directory '/home/vsts/work/1/a' is empty. Nothing will be added to build artifact 'MyBuildOutputs'.

I don't see how "publishWebProjects" would be relevant. Nothing I am doing is relevant to the web.

When I try to create a new and separate release pipeline in Azure (that will hopefully include the same build pipeline job), the different stages that I choose from never involve deploying to a VM. Plus none of the options are obvious to me as to how to configure a source (i.e., to use the .jar file from the previous step a build job). Maybe the source defaults to the Artifact thus created by the Maven build job that I configured. In the Pipeline job I configured a Git repo as an Artifact, but I am not 100% sure Azure knows I want the .jar file that it can produce with Maven to be the artifact to be deployed as the source of the pipeline job itself. Which Template, in the Pipeline options for a given stage, do I choose as the stage in a new release pipeline to move a .jar file to a Linux server?

# Answer
> 2 votes
You have two basic scenarios:

1. you have an Azure Pipelines agent running on the target machine
2. you have an SSH connection to the target machine

In the first case you just `cp` the file to the target location (given that the user running the agent has adequate permissions). In the second case, you use a task like Copy Files Over SSH task to copy the file. For this latter you have to setup an SSH service connection in advance.

---
Tags: azure-devops, azure, azure-pipelines
---


# Question
Title: How to deploy SQL Script in DevOps culture and Rollback situation too?
How we can manage database part in DevOps environment/CICD pipeline, Here taking about the SQL Script. For Example any deployment where Insert and Update SQL's, If we setup in CICD pipeline then I will execute everytime thus I will re-execute everytime & DevOps scenarios are fail in this situation.

Overall How we can setup pipeline with SQL script and Application in real world and In case of rollback, How can we manage it ?

# Answer
> 2 votes
There are a few different things you can do:

1. In your CI process, run your new SQL update scripts against a Dockerized SQL Server. This will allow your CI server to reject the changes if there are basic errors in the script.
2. If you are fully containerized look into tools like Kubernetes. These tools have ways to handle rolling back failed production updates.
3. Utilize a Blue Green Deployment strategy along with integration testing. This provides a fallback in case things go wrong and allows for different types of automated testing schemes in your CI process.

---
Tags: jenkins, continuous-integration, jenkins-pipeline, continuous-deployment, sql
---


# Question
Title: Is it possible to send email notification for Input (GO/NOGO) from a pipeline
There is User input functionality in the declarative pipeline which prompt on the Jenkins console but it's a headache to login every time on console and select the appropriate action.

So I want to implement such a thing via mail for User input, For example User will get the Input via mail where there are two URL's for "Approval" or "Reject" respectively, If User select Approval then proceeded otherwise not. Here need to pass input on the pipeline on runtime.

Are there any plug-ins for same ?

# Answer
You don't need any plugins. You can send an e-mail from your Pipeline using the `mail()` step. You can respond to the input step by using the REST API as documented in this answer on Stack Overflow. Put the two together and you should be able to craft an e-mail that contains buttons for responding to the input.
> 3 votes

---
Tags: jenkins, continuous-integration, jenkins-plugins, groovy
---


title: Cost-efficient hosting for CI/CD?

text:
# Question
Title: Cost-efficient hosting for CI/CD?
I'm paying over $300/month for Kubernetes to have CI/CD in GitLab. And it only covers web apps, Android apps and deployment to Google Cloud. Plus some staging environments, that do not require many resources and that I'm using rarely. It doesn't cover CD to Apple App Store though.

I'm curious if there is a way to optimize this? Maybe I should buy some hardware or optimize Google Cloud somehow?

# Answer
> 2 votes
Given that you mentioned rarely using some k8s resources, you could explore some alternatives:

* With `gcloud SDK` you could create scripts to be executed for the periods you want to reduce GKE nodes (even to zero) during periods you're not using it
* With Google Cloud Functions (GCF) you can create some python/Node.js/Go script to access GKE APIs and also execute the operations to stop/reduce your instances. These GCF could be triggered by schedulers configured by Cloud Scheduler. The cost here for such executions will be really low or even zero.

# Answer
> 2 votes
* Try to use only `n1-standard-1` nodes and let the autoscaler do it's magic.
* If you can accept some downtime (and eventually some failures of the CI/CD jobs that should be fixed when you retry) you can use preemptible nodes.
* Migrate stateless apps to Cloud Run.
* Follow this guide to reduce resource consumption on your cluster.

---
Tags: kubernetes, gitlab, google-cloud-platform
---


# Question
Title: How can I determine which command / step failed within a travis build
So I have a CICD travis build that failed... How do I know which line / step is the culprit?

# Answer
> 1 votes
Look at the Travis CI Logs. Often a failed build step will return "Error", "Failure", "-1", buried in a wall of text. If you want a better way to collect and these logs you can check out the following:

---
Tags: travis-ci
---


title: Ruby and Rest service in Docker

text:
# Question
Title: Ruby and Rest service in Docker
I am new to DevOps on the Ruby stack. I have Ruby on Rails application. It uses several Rest based service. Each of the services is deployed with Docker. There are no DevOps efforts now. We use Gitlab as SCM, I am assigned to do the DevOps to help the developers. We want to have three environments (development, staging, production).

I have set up an AWS Elastic Container Registry to host docker images. Two accounts (DevOps and developers) so that Gitlab CI/CD pipeline can build and upload images when developers commit code in dev, release branch respectively.

I am not sure how to handle the developer workflow. e.g. Developers testing the microservices locally. I believe the developers have to pull the images every time they need to test one of the microservices. How will a developer know that an image was updated when some other dev has updated the AWS ECR registry.

I am confused. I would like to get some suggestions on how to support the continuous development and do the continuous release for all environments. Any suggestions or pointers to books or courses that I can get more information around this is also helpful.

# Answer
Actually you need to implement per feature branch deployment. Because development environment use for testing app before QA. And feature branch deployment is very difficult to implement especialy at the begginging of devops.

May be at start will be enough to use docker-compose to build latest version of developer's application. On their localhost.
> 1 votes

---
Tags: docker, docker-compose, ruby-on-rails
---


# Question
Title: Docker container stuck in "new" state - Docker swarm - service stack
I am deploying a service stack to a Docker swarm like so:

```
$ dt stack deploy --with-registry-auth -c tmp.stack.docker-compose.yml tmp
Creating network tmp_my-network
Creating service tmp_my-service

$ dt stack ps tmp
ID                  NAME                IMAGE               NODE    DESIRED STATE       CURRENT STATE       ERROR               PORTS
yyjue6fdqu5u        tmp_my-service.1    nginx:latest                            Running             New 7 minutes ago

```

As you can see it's just an nginx, but the state of the service never leaves "new".

I can not stack deploy any other service either. No matter what I write into my compose file, they are all stuck in state "NEW".

I am not sure how to produce the following output from commandline, but in a running portainer I was able to get the following information:

```
Services > tmp_my-service > qlvqjofehvqys8kje3b32y50v > Logs
"Error grabbing logs: rpc error: code = Unknown desc = warning: incomplete log stream. some logs could not be retrieved for the following reasons: task qlvqjofehvqys8kje3b32y50v has not been scheduled"

```

This makes sense, as it looks like the service's tasks are never being scheduled, nothing is actually ever started. So I am assuming something (at orchestrator level) stops Docker swarm from carrying out the request to get any service started.

Where might I find more information about what could be wrong?

I have tried:

```
$ docker events
network create bcqcoakf5worjmfpelb199xrp (name=my-network)
node update rddzqxvlu2v2ylbcbywpa8kze (name=node-name)
service create x549akxlbonn9zvtxgh8oq2mm (name=my-service)

```

https://success.docker.com/article/why-do-my-services-stay-pending-when-trying-to-schedule-them-with-placement-contraints

https://success.docker.com/article/swarm-troubleshooting-methodology

Details of the task:

```
$ dt stack services tmp
dID                  NAME                MODE                REPLICAS            IMAGE               PORTS
x549akxlbonn        my-service      replicated          0/1                 nginx:latest

$ dt service ps my-service
ID                  NAME                IMAGE               NODE                DESIRED STATE       CURRENT STATE       ERROR               PORTS
m4d929n5su7r        my-service.1    nginx:latest                            Running             New 4 minutes ago

$ dt inspect --type task m4d929n5su7r
[
    {
        "ID": "m4d929n5su7re5igit3w2tnvg",
        "Version": {
            "Index": 2397643
        },
        "CreatedAt": "2019-10-10T15:58:33.416210977Z",
        "UpdatedAt": "2019-10-10T15:58:33.416210977Z",
        "Labels": {},
        "Spec": {
            "ContainerSpec": {
                "Image": "nginx:latest@sha256:aeded0f2a861747f43a01cf1018cf9efe2bdd02afd57d2b11fcc7fcadc16ccd1",
                "Labels": {
                    "com.docker.stack.namespace": "tmp"
                },
                "Command": [
                    "tail",
                    "-f",
                    "/dev/null"
                ],
                "Privileges": {
                    "CredentialSpec": null,
                    "SELinuxContext": null
                },
                "Isolation": "default"
            },
            "Resources": {},
            "RestartPolicy": {
                "Condition": "on-failure",
                "Delay": 5000000000,
                "MaxAttempts": 3,
                "Window": 120000000000
            },
            "Placement": {
                "Constraints": [
                    "node.role == manager"
                ],
                "Platforms": [
                    {
                        "Architecture": "amd64",
                        "OS": "linux"
                    },
                    {
                        "OS": "linux"
                    },
                    {
                        "Architecture": "arm64",
                        "OS": "linux"
                    },
                    {
                        "Architecture": "386",
                        "OS": "linux"
                    },
                    {
                        "Architecture": "ppc64le",
                        "OS": "linux"
                    },
                    {
                        "Architecture": "s390x",
                        "OS": "linux"
                    }
                ]
            },
            "Networks": [
                {
                    "Target": "bcqcoakf5worjmfpelb199xrp",
                    "Aliases": [
                        "my-service"
                    ]
                }
            ],
            "ForceUpdate": 0
        },
        "ServiceID": "x549akxlbonn9zvtxgh8oq2mm",
        "Slot": 1,
        "Status": {
            "Timestamp": "2019-10-10T15:58:33.41619488Z",
            "State": "new",
            "Message": "created",
            "PortStatus": {}
        },
        "DesiredState": "running"
    }
]

```

It all gives me the same info: Desired is "running", actual state is "new" but it won't give me more info on why.

**edit:** Under "placement \> Constraints" it says "node.role == manager" in the ouput. This is just temporary, as I have been trying to get it to run on individual nodes by constraining it. It does not work with this constraint, nor any other constraint, nor with no constraint at all.

# Answer
> 1 votes
After some more time I found my answer.

The service I wanted to deploy was a slightly different version of a service we already have. Stack file for the "original" service:

```
version: '3.7'

services:
my-service:
    image: my-repo:port/company/my-service
    ports:
    - 81:81
    networks:
    - my-network
    deploy:
    replicas: 1
    restart_policy:
        condition: on-failure
        delay: 5s
        max_attempts: 3
        window: 120s
networks:
my-network:
    driver: overlay
    internal: false
    attachable: true
    ipam:
    driver: default
    config:
        - subnet: 10.1.99.0/24

```

We took **the same file** but on a different git branch to deploy the slightly altered version (as makes sense from a "code" perspective).

From a Docker perspective this meant however that the stack with the original service with the **exact same network definition** was still running and we deployed another stack with **the exact same network definition** again, which lead to the service never being scheduled as, apparently, you must not re-declare the same network twice.

Of course it will get a different network name, but the overlay/subnet config will be identical.

It would be nice of Docker to inform you of this when you're trying to do that, but, as it so often is: Crap in, crap out. Our fault!

For future readers who do not have the exact same problem but their containers are getting stuck in "new" state: It might be safe to assume that it has something to do with your stack-file, your network definition or even something like correct YAML indentation. It is most certainly something "meta" and not something related to your Docker host and specifically not your image or your container (as when the task is stuck in "new" state, there is not even a container in existence as it never got scheduled).

---
Tags: docker, containers, docker-swarm
---


# Question
Title: How to copy a file locally to remote pod before it starts?
```
  # Copy /tmp/foo local file to /tmp/bar in a remote pod in namespace
<some-namespace>
  kubectl cp /tmp/foo <some-namespace>/<some-pod>:/tmp/bar

```

works, but then a pod should be running. The problem is that the file has to be copied before the pod starts. The jar file has to be loaded by an Application server. Destroying the pod after the copy will not work as the docker image is deployed again and the content will be wiped out.

A configMap was misused to copy the jar before the pod starts, but that only seems to work for configuration files. This was confirmed by comparing the checksum, e.g. local vs. remote.

# Answer
> 5 votes
Ok. Maybe you're not gonna like it but there are some things you can do.

One and most simple is to use a volume mount, eventually on a persistent storage but I think an emptyDir can work as well (or even better depending on the nature of the file you want to copy) and use an init container to copy the file to the volume before the app container start. I don't know how you generate that file and I can't speculate about the security implications but the file needs to be in the docker image used for the init container. So, for your init container, you will have a Docker file like

```
FROM ubuntu
COPY localfile /container/file
CMD cp /container/file /mnt/kube-volume

```

replacing localfile, container/file and /mnt/kube-volume with appropriate values; you build the container image locally -on your machine, so it can contain the file- and upload it to the (secure) docker repo along with the app container image.

Another method is to upload the file to a (secure) location and have an init container or a bootstrap process that download the file when you start your main application container in the pod.

The third option is to create a docker image that contains that file and use it for the app container in your pod.

# Answer
> 0 votes
If it's a small(\<1MB) file, you can create a configmap with "--from-file" and mount it into volume.

---
Tags: kubernetes
---


title: Use a pod template file in a Deployment

text:
# Question
Title: Use a pod template file in a Deployment
I've been looking at Deployments descriptions in Kubernetes. In the tutorials I've found, the definition of the replicated pods are always a copy-paste of a pod template file.

Is it possible to rather use a reference to the yaml file (or an url) defining a pod (I didn't found any resource telling so)?

If yes, should it be avoided (in order to keep label/selector in the same file for example)?

If no, is there a reason (because Kubernetes wouldn't know if you modified the pod template file for example)?

# Answer
> the definition of the replicated pods are always a copy-paste of a pod template file

This behavior is implemented by Kubernetes itself: it takes the `template` part of the deployment spec and creates `replicas` identical pods.

The corollary to this is that you don't need to manually write out the pod spec as a separate object. Including it in the `template` part of a deployment (job, statefulset, daemonset) is enough.

> Is it possible to rather use a reference to the yaml file (or an url) defining a pod

No, individual Kubernetes objects must be self-contained (except where there are facilities to do things like inject Secret values into environment variables). The Kubernetes server doesn't know about your local file system.

Using one of the templating systems out there (Helm is more-or-less standard but not the only choice) could help reduce this duplication, if you really need near-identical deployments and also bare pods. You could write something like

```
---
apiVersion: v1
kind: Pod
{{ template "pod-spec" . }}
---
apiVersion: apps/v1
kind: Deployment
spec:
  template:
{{ include "pod-spec" . | trim | indent 4 }}

```
> 1 votes

# Answer
Yes you can create resources (including pods) from a url yaml using kubectl. However, just because you pointed to a remote url does not mean that the resource will automatically update. You will need to either re-deploy when the remote yaml is updated, develop/integrate with a service that handles this, or look into using a tool such as helm to help manage changes.

It does not need to be avoided, you just need to know that it will not automatically update if the source changes.
> 1 votes

---
Tags: kubernetes
---


# Question
Title: How to create the smallest working docker image every time?
**Aim:** to create the smallest working docker images every time

**Current**

```
REPOSITORY          TAG       IMAGE ID            CREATED             SIZE
a-docker-image      latest    x                   42 minutes ago       1.92 GB

```

**Attempt**

Adding a cleanup step at the end of the Dockerfile:

```
#clean
RUN apt-get purge -y wget
RUN rm -r a-build-dir
RUN apt-get purge -y a-package

```

reduced the image size a little bit:

```
REPOSITORY          TAG       IMAGE ID            CREATED             SIZE
a-docker-image      latest    y                   2 minutes ago       1.86 GB

```

**Discussion**

I have built various docker images. Every time I try to decrease the size of the created image, but I always feel it is too large. I am looking for a script that has already been created by someone on github that removes all superfluous packages from the image so the size of the created image will be as small as possible.

As I said I always try to reduce the size of the image, but I want to apply this consistent so that every image that I create from now on will be as small as possible.

**Question**

How to create the smallest working docker image every time?

# Answer
There's a variety of techniques involved, with no single solution. You will likely want to do several of the following:

---

First, optimize your image layers for reuse. Put frequently changing steps later in the Dockerfile to increase the chances that early layers are cached from previous builds. A reused layer will show up as more disk space in a `docker image ls`, but if you examine the underlying filesystem, only one copy of each layer is ever stored on disk. That means 3 images of 2 GB each, but which only have 50 MB different in the last few layers the build, will only take up 2.1 GB of disk space, even though the listing makes it appear that they are using 6 GB since you are double counting each of the reused layers.

Layer reuse is why you see images with infrequently changing build dependencies install those first before copying in code. See any python example that has a pattern like:

```
FROM python
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
# note how the code is copied only after the pip install
# since code changes but requirements.txt doesn't
COPY . .
CMD ["gunicorn", "app:app"]

```

---

Pick a minimal base image. This is why you see people go from `ubuntu` to `debian:slim` (the slim variants are smaller, shipping with fewer tools), or even `alpine`. This reduces the size of your starting point, and is very helpful if you are constantly pulling new versions of the base image. However, if your base image rarely changes, layer reuse removes much of the advantage of a minimal base image.

The smallest base image you can pick is `scratch`, which is nothing, no shell or libraries, and is only useful with statically compiled binaries. Otherwise, pick a base image that includes the tools you need without lots of tools you don't need.

---

Next, any step that changes or deletes a file should be combined with previous steps that create that file. Otherwise the layered filesystem, which uses copy-on-write even on things like a file permission change, will have the original file in a previous layer and the image size will not shrink when you remove files. This is why your `rm` commands have no effect on resulting disk space. Instead, you can chain the commands, like:

```
RUN apt-get update \
 && apt-get install -y \
      a-package \
      wget \
 && ... \
 && apt-get purge -y wget \
 && rm -r a-build-dir \
 && apt-get purge -y a-package

```

Note that overuse of command chaining can slow down your builds since you need to reinstall the same toolset any time a prerequisite changes (e.g. the code being pulled with wget). See multi-stage below for a better alternative.

---

Any file you create that you do not need in your resulting image should be deleted, in the step that creates it. This includes package caches, logs, man pages, etc. To discover what files are being created in each layer, you can use a tool like wagoodman/dive (which I have not personally vetted and would express caution since it runs with full root access on your host), or you can build your docker images without pruning the intermediate containers and then view the diff with:

```
# first create and leave containers from any RUN step using options on build
docker image build --rm=false --no-cache -t image_name .
# review which layers use an unexpectedly large amount of space
docker image history image_name
# list all containers, particularly the exited ones from above
docker container ps -a
# examine any of those containers
docker container diff ${container_id}
# ... repeat the diff for other build steps
# then cleanup exited containers
docker container prune

```

With each of those intermediate containers, the diff will show what files are added, changed, or deleted in that step (these are indicated with an `A`, `C`, or `D` before each filename). What diff is showing is the container specific read/write filesystem, which is any file changed by the container from the image state using copy-on-write.

---

The best way to reduce image size is eliminating any unneeded components, like compilers, from your shipped image. For that, multi-stage builds let you compile in one stage, and then copy only the resulting artifacts from the build stage to a runtime image that has only the minimum needed to run the application. This avoids the need to optimize any of the build steps since they are not shipped with the resulting image.

```
FROM debian:9 as build
# still chain update with install to prevent stale cache issues
RUN apt-get update \
 && apt-get install -y \
      a-package \
      wget \
RUN ... # perform any download/compile steps

FROM debian:9-slim as release
COPY --from=build /usr/local/bin/app /usr/local/bin/app
CMD [ "/usr/local/bin/app" ]

```

Multi-stage is ideal with statically compiled binaries which you can run with scratch as your base image, or transitioning from a compile environment like JDK to a runtime like JRE. This is the easiest way to dramatically reduce your image size while still having fast builds. You may still perform chaining of steps in your release stage if you have steps that change or delete files created in previous steps, but for the most part, the `COPY` from another stage isolates the release stage from any layer bloat experienced in the earlier build stages.

---

Note, I do not recommend squashing images since this reduces the size of one image at the expense of eliminating layer reuse. That means future builds of the same image will require more disk and network traffic to send updates. To go back to the first example, squashing may reduce your image from 2 GB to 1 GB, but not 3 images may take up 3 GB instead of the 2.1 GB.
> 7 votes

# Answer
A `Dockerfile` creates a new layer for each of the commands in the file. Since layers are well, *layered* on top of each other - you cannot remove files that a previous layer added. This is why when you install packages, or download files, or create builds each in a separate command - these are still there in the image, even if in a future layer you removed them.

So if you just change this:

```
RUN apt-get update -y
RUN apt-get install -y wget a-package
# ...
RUN apt-get purge -y wget
RUN rm -r a-build-dir
RUN apt-get purge -y a-package

```

To this:

```
RUN apt-get update -y \
    && apt-get install -y wget a-package \
    && mkdir a-build-dir \
    && wget http://some-site/very-big-source-code.tar.gz \
    && tar xzvf very-big-source-code.tar.gz \
    && do-some-compilation \
    && apt-get purge -y wget \
    && cd .. \
    && rm -rf a-build-dir \
    && apt-get purge -y a-package

```

You will get a much smaller image.

---

Another option, is to **squash** the image after you built it. Q: How does the new `docker --squash` work?

---

Yet another option, is to choose a slim base image. For example, images that use Alpine Linux as their base instead of Debian, take just 10-15mb instead of 180-250mb. And this is before adding your own application and data. Many *official* base images on Docker Hub have an alpine version.
> 27 votes

# Answer
Probably not exactly an answer, but worth giving the alternatives.

Chef's **habitat** has been created with this in mind, creating a package with all needed dependencies without the extraneous distro/base image load you don't wish.

Extracts on what matters here, the container size from **this blog post** with a simple nodejs app:

> ```
> michael@ricardo-2:plans_pkg_part_2$ docker images
> REPOSITORY           TAG                 IMAGE ID            CREATED             SIZE
> mfdii/node-example   latest              36c6568c606b        40 minutes ago      655.9 MB
> node                 latest              04c0ca2a8dad        16 hours ago        654.6 MB
> mfdii/mytutorialapp  latest              534afd80d74d        2 minutes ago       182.1 MB
>
> ```

`mdfii/node-example` is a docker image from a classic dockerfile while `mfdii/mytutorialapp` is the docker image produced with habitat.

If size is your main concern and you're up to take the learning curve of Habitat plans, this could be a solution for you.
> 4 votes

# Answer
One could also use dive

```
docker run --rm -it \
    -v /var/run/docker.sock:/var/run/docker.sock \
    wagoodman/dive:latest <dive arguments...>

```

to get a report about what waste could be removed from a docker image in order to reduce the size.
> 0 votes

# Answer
If you want to have reusable development layers but reduce your disk usage for delivery, you can produce a merged "delivery layer" like this:

1. Make sure you have a container that uses your image (if you don't have one, maybe use something like `docker run IMAGE echo`, if the echo command is available)
2. Find the container id (maybe using `docker container ls -l`)
3. Pipe `docker export` to `docker import` to create the merged layer (something like `docker export 20f192c6530a | docker import - project:merged`)

This will keep your development layers around but give you a smaller, merged image that you can deliver.
> 0 votes

# Answer
Multi stage builds. Use the image that has all your build components to build your application, and a lighter runtime image. Copy just your build artifact to the runtime image. No need to delete anything.

https://docs.docker.com/develop/develop-images/multistage-build/
> 0 votes

# Answer
Use `docker ps` to check the current running containers. For example:

```
FROM ubuntu16

MAINTAINER sreeni (email/domain)

RUN apt-get update

RUN apt-get install -y nginx

ENTRYPOINT [“/usr/sbin/nginx”,”-g”,”daemon off;”]

EXPOSE 80 (port)

```

Use below docker command to run the container:

```
docker run -d -p 80:80 --name web server ubuntu16

```

After that, check localhost or ip address:80 (open browser and check)
> -1 votes

---
Tags: docker
---


title: ansible-pull ignore group_vars vault

text:
# Question
Title: ansible-pull ignore group_vars vault
How do I ignore the `group_vars` vault file when running ansible-pull?

My repository (among other stuff) does machine init, joining the domain and software updates. I would like the software part to be handled by ansible-pull. No passwords are required for this, and no vault is necessary.

Unfortunately, initial machine setup after a fresh install does require some passwords which are set in `group_vars/all.yml` and is encrypted by vault. Whenever I run the `ansible-pull` command I run into vault decryption issues as there is no password provided.

Any advice on how to overcome this?

# Answer
Eventually I just got rid of the group\_vars all together. I specified a new role that includes several other roles with 'include\_role:' and added the vars directly to those roles with inline-vault variables:

```
- name: include ipaclient
  include_role:
    name: ipaclient
  vars:
    my_user: user
    my_password: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          123456123456123456123456123456132456...

```
> 1 votes

---
Tags: ansible, ansible-vault
---


# Question
Title: How do I create separate versions of lambda functions through terraform?
I'm Running Terraform v0.11.14

```
resource "aws_lambda_function" "test_lambda" {
  filename         = "lambda_function.zip"
  function_name    = "test_lambda"
  role             = "${aws_iam_role.iam_for_lambda_tf.arn}"
  handler          = "index.handler"
  source_code_hash = "${data.archive_file.lambda_zip.output_base64sha256}"
  runtime          = "python2.7"
  version          = "1"
}

```

On plan I get the error

```
Error: aws_lambda_function.test_lambda: "version": this field cannot be set

```

I couldn't find an aws\_lambda\_function\_version resorce so presume you create versions using the lambda fuction resorce and specifying the version.

I'm not sure what value I give to version to build versions of my lambda.

Im tring to get it to work for one version for now as a first step. I plan on adding count to create multiple versions as my next step if I can get it to create one version first?

# Answer
> 1 votes
I just needed to set

publish = true

I then enconter another issue for creating seprat versions but finding this answers my initial question

---
Tags: aws-lambda, versioning
---


# Question
Title: How to properly automate CM based on log events?
Let's say I have a number of tools like Docker bench security, some static security scanners, etc. Now I have let's say free Ansible edition (no expenses are considered).

I have lots of logs, but not much automated, actionable info that triggers scripts. What is a proper way to automate link between static scanners and CM tools like Ansible and Jenkins?

Should it go like:

1. Parse logs for set of errors and then automatically trigger "correcting scripts"?
2. Just run a playbook once in a while as it checks the state before doing anything?
3. Use tools/plugins for Ansible?
4. Something else?

# Answer
There is a natural tension here between:

(a) **fixing something that is wrong automatically**: because for example, a CVE has been created with a high enough severity to need to do something about it.

(b) **breaking the system**: because the automation introduced a breaking change or a defect.

I have a tendency towards systems being under constant development, even if no features are being added to them they need to kept current to avoid exposure to vulnerabilities. The rate of change to the 3rd party components of the majority of systems I cover is so high that's its actually worth rebuilding everything and redeploying it daily.

This introduces a couple of requirements:

1. It is highly automated,
2. 99% of user journies are covered by end-to-end tests so each build has a high degree of confidence that it will work,
3. We don't roll out to all users, we employ canary release and a rollback mechanism,
4. We use version *constraints* in package manifests, etc... i.e. the `version of React must be >= 16.7.1 and < 17.0.0`
5. We manage by exception, i.e. if the end-to-end tests fail or the canary release fails humans get involved.

This, in turn, makes systems like trivy, OWASP ZAP and SonarQube more about generating metrics and stopping vulnerabilities from going live than triggering automated remediation.

We do then have a set of rules like:

1. The live application must not have an unmitigated CVE with a CVSS score of 7.0 or higher.
2. The list of CVEs with a score of 4.0 or higher will be reviewed on a weekly basis to assess if remediation should be undertaken (i.e. change to a different library, introduce a new safeguard, remove a defunct library, fix the CVE at source, etc.)

This then gives my SRE teams the operational boundaries to work within, they may in practice:

1. just re-run the deployment ad-hoc to fix it **now** rather than within 24h,
2. override a default, for example, if there is a known vulnerability deploy something with a less critical but still higher than CVSS 7.0 known vulnerability,
3. Introduce a new safeguard, i.e. a Web Application Firewall Rule to mitigate,
4. Swap out the dependency for something less vulnerable,
5. Disable that functionality entirely until a fix can be introduced.
> 1 votes

---
Tags: jenkins, ansible, jenkins-pipeline, configuration-management
---


# Question
Title: How to bring a docker data container to Kubernetes?
I have an application that relies on 4 docker data containers for the data it needs to run some tests. At the moment I have their docker-compose file and their images but I don't have access to the data containers Dockerfile or actual data.

How this application works in docker-compose is as below: The application is called rtsp\_server and it loads different volumes from the data containers such as st-andries4(I just wrote one of them).

This is their docker-compose:

```
# The application
rtsp_server:
  image: rtsp_server_image
  volumes_from:
    - rtsp_data_st_andries4:ro
  ports:
    - "8554:8554"
  command: python ./rtsp-server.py /rtsp_data/st-andries-4 st_andries_4

# Data container
rtsp_data_st_andries4:
  image: st_andries_4:1
  volumes:
    - /rtsp_data/st-andries-4
  command: /bin/true
  mem_limit: 10000000000

```

I already brought the application, rtsp\_server, to Kubernetes and it works fine but I need to also bring the data containers to k8s but I don't know how to do it.

# Answer
You'll want to look through the Kubernetes Volume documentation to see what options are available.

In particular look into the PersistentVolumeClaim. This allows you to setup a shared data volume for your pods. Here's an example snippet from a k8s deployment YAML using a volume:

```
spec:
      containers:
      - image: **yourdbcontainer**
        name: mysql-db
        env:
        - name: example
          value: example_value
        ports:
        - containerPort: 3306
          name: mysql-db
        volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql
      volumes:
      - name: mysql-persistent-storage
        persistentVolumeClaim:
          claimName: mysql-pv-claim

```
> 1 votes

---
Tags: docker, kubernetes, docker-compose
---


# Question
Title: How can i check if all CustomResourceDefinitions have been created properly?
I have installed with success cert-manager following the steps indicated here.

Now, i am doing an script that checks if every steps are done properly, so my question is: How can i check if all the CustomResourceDefinitions are created via kubectl, for example?

I use google kubernets engine v1.13

# Answer
> 0 votes
This command works: `kubectl get CustomResourceDefinition | awk '{ print $1 }'`

You can use `-o yaml` or `-o json`

---
Tags: kubernetes, google-cloud-platform
---


title: Questions regarding Kubernetes storage

text:
# Question
Title: Questions regarding Kubernetes storage
I have two questions regarding K8s storage:

1. Can I set up an NFS server on one of the worker nodes in a bare-metal cluster?
2. I am running different versions of our application which consists of different components on different namespaces. If I want to use StorageOS do I need to install one StorageOS per namespace?

Thank you.

# Answer
1. Yes. Check out this example for a nice tutorial to get started.
2. Per the StorageOS documentation you shouldn't even have to worry about this. Just let StorageOS handle the namespaces for you with one StorageOS install.
> 2 votes

---
Tags: kubernetes, storage
---