# Question
Title: Coupling docker registry and source control
What are the best practices (if any) for coupling a docker image registry with a SCM service (such as bitbucket)?

I know a docker registry can live in artifactory, but how can I assure that the two are as tightly coupled as possible?

For example I am worried that without due-diligence from developers the latest Docker image in the registry will not reflect the current state of the Dockerfile in SCM or vice-versa. Also it is imperative that we can always trace back to the base Dockerfile (that we made) for any image in the registry.

# Answer
> 3 votes
I was able to get an answer over here forums.docker.com Credit goes to dmaze.

Set up some sort of automated build system (“continuous integration” in the current trendy term). Docker is sufficiently mainstream at this point that any of the cloud-based or locally-installed CI systems can do it.

In your Dockerfile, use a LABEL to record the source of the build. That probably includes the commit hash from distributed source control (git, Mercurial), the branch name if relevant, any release tags if present, and possibly details like the timestamp of the last commit. docker history and docker inspect should be able to show these.

When you docker push your images, push them at least twice, with the commit hash and with the branch name as the “version” part (quay.io/mycorp/imagename:123abc7, quay.io/mycorp/imagename:dmaze-test). If release tags are readily available, the CI system should push images with these tags too.

Make sure the Dockerfile is committed to source control, of course, and try to have a stable path to get any external dependencies that may be present.

Now you can go both ways: given an arbitrary commit, if your CI system built it, you can docker run the image it built; and if you have an image, you can find where exactly in source history it came from, and git checkout or hg up to that specific version, and docker build a near-identical copy of it yourself.

# Answer
> 5 votes
If you want a beautiful integration between Docker and SCM, GitLab provides it's own built-in Docker registry. This makes publishing a Docker image in the build pipeline a breeze.

The other big advantage of GitLab Docker registry is that it supports multiple Docker repositories for each GitLab repo. This allows you to create a new Docker repo for each branch, or each commit, each environment, or anything you need.

My company leverages this by pushing our images to repos based on the branch, tagged with their commit. Here is an example: `project-name/branch-name:commit-SHA`

Or if you have multiple Docker images being build for a particular branch (like a front-end Angular app and it's back-end API that it talks to), you can scope it even further, like so:
`project-name/branch-name/api:commit-SHA`
`project-name/branch-name/front-end:commit-SHA`

There are no limits to the number of slashes (/) you can use for scoping. This makes it very easy to tell the exact commit of an image when it's deployed. For this reason, my company almost never uses the generic 'latest' tag. We prefer the specificity of which image is deployed where.

You can still apply this same 'multiple docker repo' logic to any Docker registry. You'll have to look up that specific system's capabilities for creating new repos on the fly, and how easy it is to integrate with your CI/CD pipeline.

---
Tags: docker, git, scm, registry
---


# Question
Title: Which server to set up for distribution of windows based application updates, patches and upgrades?
We are looking forward to set up some server to distribute application patches, updates, and upgrades. Our applications would be checking for their updates by querying this server. Presently, developers here are considering Chocolatey, and just rejected Squirrel for windows due to its narrow scope.

How is this being done in the world generally with best practices?

# Answer
I recommend Chocolatey. There are more and more companies that are adopting this. For example, if one would like to install Gradle and navigates to the installation page:

> Chocolatey is “the package manager for Windows”.
>
> ```
> $ choco install gradle
>
> ```
> 1 votes

---
Tags: installation, upgrade
---


# Question
Title: Deployment custom App in Microsoft Teams
I am developing a chatbot Azure service, which I want to integrate within Microsoft Teams. So far everything is working but the "re-install" of the package in the Microsoft Teams.

I created a publish "folder-profile". Then I zip the result with the manifest.json and the icon files inside. I go to the Manage Team section and in the Apps tab I select upload a custom app. Then I choose the .zip file and the service seems to be there (actually it is there).

It works, but when I create a new version and I repeat the described steps, it seems like Microsoft Teams is still using my old code.

I test the chatbot in the Chat by using @"APP-ID" and I see how my changes work, but installed as an App for the "Team" keeps the old version.

I tried to uninstall it, check if the bot is gone(it is gone) and then upload again, but some kind of cache is there and the bot behaves like in the previous version.

Any idea how to deploy new versions of my app in Microsoft Teams?

# Answer
One could use NuGet. This tool is embedded in various tools, including Chocolatey.
> 1 votes

---
Tags: deployment, azure
---


title: Parallelising Packer with Ansible

text:
# Question
Title: Parallelising Packer with Ansible
**Issue**

I have an Ansible role which covers Windows, macOS and Ubuntu OS. I also have a Packer template file for each OS (due to small nuances and changes in the template file to handle provisioning) which handles standing up and provisioning the VM with Ansible onto VSphere.

So to build all three VMs, it goes something like:

```
  Windows build                Mac build                 Ubuntu build
---------------------     ---------------------      --------------------
| Packer -> Ansible | --> | Packer -> Ansible | ---> |Packer -> Ansible |
---------------------     ---------------------      --------------------

```

Which isn't very efficient and can take a while.

**Solution**

I'm thinking of parallelising this somehow, by potentially having Packer standup all three VMs, but not provision it - only get the IP address of the VMs, which then (with some manipulation), put the IP addresses into an Ansible inventory file, and the ansible playbook is then run against that.

Something like:

```
          --------------
Packer -> | Windows VM | <----
   |      --------------     |
   |      --------------     |
   -----> |  macOS VM  | <----
   |      --------------     |
   |      --------------     |
   -----> | Ubuntu VM  | <----
          --------------     |
                             |
                          Ansible parallel provisioning

```

Is this an optimal way of using Packer and Ansible together? or is there a better way of parallelising these builds?

# Answer
> 1 votes
What about using a CI tool? Create a parallel job and the jobs will run parallel. For example, one could use Jenkins:

https://jenkins.io/blog/2017/09/25/declarative-1/

> ```
> stage('run-parallel-branches') {
>   steps {
>     parallel(
>       a: {
>         echo "This is branch a"
>       },
>       b: {
>         echo "This is branch b"
>       }
>     )
>   }
> }
>
> ```

but there are other CI tools that could do the same.

---
Tags: continuous-integration, ansible, packer
---


# Question
Title: Compatibility of http-builder-ng Groovy module with Jenkins 2
Did somebody had a chance to use http-builder-ng Groovy module in Jenkins pipeline?

When I use this module with any kind of request/URL it always returns me null object when the same source code from my Desktop is being run fine.

Here is debug output:

```
<groovyx.net.http.UriBuilder$Basic@4bc2413c scheme=http port=-1 host=api.open-notify.org path=/astros.json query=[:] fragment=null userInfo=null parent=groovyx.net.http.UriBuilder$ThreadSafe@69c6847a useRawValues=null>

```

The older version of this module developed by groovy.codehouse works fine, but it doesn't have some methods which I need.

# Answer
> 1 votes
I would recommend against importing if at all possible. You can use built-in functionality to achieve the same results. Also be aware of "In-process Script Approval"

https://stackoverflow.com/a/42662243/1678094

```
// GET
def get = new URL("https://httpbin.org/get").openConnection();
def getRC = get.getResponseCode();
println(getRC);
if(getRC.equals(200)) {
    println(get.getInputStream().getText());
}

// POST
def post = new URL("https://httpbin.org/post").openConnection();
def message = '{"message":"this is a message"}'
post.setRequestMethod("POST")
post.setDoOutput(true)
post.setRequestProperty("Content-Type", "application/json")
post.getOutputStream().write(message.getBytes("UTF-8"));
def postRC = post.getResponseCode();
println(postRC);
if(postRC.equals(200)) {
    println(post.getInputStream().getText());
}

```

You can also override the request method:

https://stackoverflow.com/a/32503192/1678094

```
conn.setRequestProperty("X-HTTP-Method-Override", "PATCH");
conn.setRequestMethod("POST");

```

---
Tags: jenkins-pipeline, groovy, java, jenkins2
---


# Question
Title: How to create Chef Package for Redhat. I have 2 rpms agents and 2 yaml files
1. Installation steps. There are two ELK agents to install:

* Metricbeat

    1. Copy the attached file `metricbeat-6.0.0-x86_64.rpm` to the target server
        2. Execute the command: `sudo rpm -vi metricbeat-6.0.0-x86_64.rpm`
        3. Copy the attached file `system.yml` to `/etc/metricbeat/modules.d/`
        4. Copy the attached file `metricbeat.yml` to `/etc/metricbeat/`
        5. Execute the command: `sudo systemctl enable metricbeat`
        6. Execute the command: `sudo service metricbeat start`
    * Filebeat

    1. Copy the attached file `filebeat-6.0.0-x86_64.rpm` to the target server
        2. Execute the command: `sudo rpm -vi filebeat-6.0.0-x86_64.rpm`
        3. Execute the command: `sudo mv /etc/filebeat/module.d/system.yml.disabled /etc/filebeat/module.d/system.yml`
        4. Copy the attached file `filebeat.yml` to `/etc/filebeat/`
        5. Execute the command: `sudo systemctl enable filebeat`
        6. Execute the command: `sudo service filebeat start`

I want to Automate this using Chef. The Chef environment is ready and working.I want to deploy this binaries on 150 servers. Would be helpfull if someone explain with step by step.

# Answer
Very basic recipe exemple to install filebeat on ubuntu (I do use more or less the same approach for elasticsearch, kibana and logstash):

```
apt_repository 'elk' do
  uri 'https://artifacts.elastic.co/packages/5.x/apt'
  distribution 'stable'
  components ['main']
  key 'https://artifacts.elastic.co/GPG-KEY-elasticsearch'
  cache_rebuild true
end

package 'filebeat' do
  action :upgrade # To keep up to date with releases from elastic.co
end

service 'filebeat' do
  action [:enable, :start]
end

template '/etc/filebeat/filebeat.yml' do
  source 'filebeat.yml.erb'
  mode '0600'
  notifies :restart, 'service[filebeat]'
end

```

Basically, adding a repository, installing package, enabling and starting the service, render a config and restart the service when it changes. On ubuntu the package set the service scripts so I don't have to manage it here.

The template is my default configuration using some node attributes depending on environment. See https://docs.chef.io/resource\_template.html for more details on how to use them.

According to your question I would strongly sugest you to follow the tutorial paths at https://learn.chef.io to grasp the basics about chef.
> 1 votes

# Answer
https://www.google.nl/search?q=how%2Bto%2Bcreate%2Ba%2Bchef%2Bpackage

returns:

https://docs.chef.io/resource\_package.html

and

https://docs.chef.io/resource\_examples.html

The latter provides code snippets, like how to copy files:

> **Create a file from a copy**
>
> The following example shows how to copy a file from one directory to another, locally on a node:
>
> ```
> file '/root/1.txt' do
>   content IO.read('/tmp/1.txt')
>   action :create
> end
>
> ```

More example could be found in the supermarket. In order to write chef books one could investigate how some recipes are structured like mysql.
> 0 votes

---
Tags: chef, automation
---


# Question
Title: PTC with Jenkins on Deployment based on the Build request which raised on PTC
I've build which is checked-in on the PTC Integrity and I've raised the CR or SR ticket(Build & Deploy Request) on the PTC life manager by developer for deploy his build on some XXXX server and the version of the build is xxxx.

As of now as a deployment team we are following the below process

If we get a ticket we assigned it to us. And then collect all those details which is on the ticket by manually and doing deployment also by manually.

Instead of this now we are planning to do this on automation.

How do I get the following details on the Jenkins to do my deployment. I've some bat script to do the deployment. To kick off that bat file, it needs the following details:

1. Version of the build
2. Which server the Build needs to deploy
3. Where the build file is located

I need to collect all those above details from the PTC ticket which I had created earlier Is there any possiblity from Jenkins can I able to get those values from the ticket number.

I'm very new to PTC. I came to know from google there is a plugin available for PTC with Jenkins

Could someone please help on this

# Answer
> 3 votes
You can pass your data using environment variables or through job parameters.

Here's an example of setting environment variables in a pipeline:

```
pipeline {
    agent any
    environment {
        FILE = "Makefile"
        SERVER = "example.com"
        VERSION = "v1.1.0"
    }
    stages {
        stage ('build') {
            steps {
                echo "VERSION is: $VERSION"
                bat "script.exe ${version} ${server} ${file}"
            }
        }
    }
}

```

If you want to use parameters instead you could do so in the following pipeline example:

```
pipeline {
  agent any
  parameters {
    string(description: 'file', name: 'file')
    string(description: 'server', name: 'server')
    string(description: 'version', name: 'version')
  }
  stages {
    stage("foo") {
      steps {
        echo "VERSION is: ${params.version}"
        bat "script.exe ${params.version} ${params.server} ${params.file}"
      }
    }
  }
}

```

See the following documentation for building a parameterized pipeline: https://github.com/jenkinsci/pipeline-model-definition-plugin/wiki/Parametrized-pipelines

If you're using a freestyle job it's slightly different but you can find detailed instructions here: https://wiki.jenkins.io/display/JENKINS/Parameterized+Build

---
Tags: jenkins, jenkins-plugins
---


# Question
Title: Is it possible to load balance TLS/SSL based logstash communication to elasticsearch?
if i have filebeat -\> logstash -\> elasticsearch pipeline with TLS or HTTPS encryption , is it possible to load balance between 2 destination elasticsearch clusters in active-standby ?
what i mean is, in case one of the clusters goes down i would manually or automatically switch all the data streams to the other cluster.

im looking at various proxy setups but im not sure i understand the terms correctly, and TLS would not solve the problem.

# Answer
You can do this a couple of ways, but I have never seen anyone do this with a CNAME.

One option, if you have control over your IP addressing is to use BGP to fail-over (or load balance) an IP to your cloud hosting providers.

A second option is to invest in a DNS-based geographic load balancing solution, such as the GTM from F5, the GSLB solution from A10, or the GSLB solution from Kemp (F5 is the most commonly used in the industry). With these solutions you will set your name server (NS1 and NS2) for your domain to be the GSLB solution of your choice. You will set a very low TTL for your DNS records and configure health monitors to your ELK stack from the GSLB solution. These can be as complex as logging into ELK and scraping for data or as simple as making sure you can establish a TCP connection and/or ping your server. When this health monitor fails, the GSLB solution will failover traffic to the alternate data center automagically. You will be able to do this as an HA pair with one GSLB solution at once DC as NS1 and the other at your secondary site as NS2. The GSLB will then forward lookup to your previous nameservers which will publish IP addresses for both sites. These will be rewritten to eliminate the IP that the GSLB does not want to direct traffic to at the time (again, this can be used for active-active sites - you can even use geolocation to send users in Europe to a Eurpoean cloud provider and US based users to a US based provider, or do a simple active/standby)

A final way to do this might be with local load balancing solution, such as F5's LTM, A10's SLB, or Kemp's solution. You may also be able to use nginx for this for a cheap solution. In this scenario, you might have this solution onsite in an office with the URL to the load balancer served up by your local DNS server to a local IP (a Virtual IP) on your load balancer. You would then configure the load balance to direct users to the desired cloud provider by configuring a health monitor (of whatever complexity). You would then need to configure this load balancer to bridge your SSL connection. You would add your certificate and private key to the load balancing solution. Users would establish an encrypted connection to the load balancer. This encryption would be stripped off by the load balancer and a new encrypted connection established to the cloud provider by the load balancer, and the data forwarded along over the new SSL tunnel.
> 1 votes

---
Tags: load-balance, logstash, ssl, elastic
---


title: How to connect to a docker container?

text:
# Question
Title: How to connect to a docker container?
I have the following `Dockerfile`:

```
FROM ubuntu:latest
MAINTAINER Valter Silva
RUN apt-get update
RUN apt-get install -y apache2
ADD index.html /var/www/html/
CMD /usr/sbin/apache2ctl -D FOREGROUND
EXPOSE 22
EXPOSE 80

```

This is how I build my `image`:

```
docker build --tags mob:latest .

```

Next, this is how I bring my container up:

```
docker run -d --name website mob

```

Then I check my container's ip via the following command:

```
docker network inspect bridge

```

I would like to execute a `curl` on this ip to ensure that my website is up and running:

```
curl 172.17.0.2

```

But for some reason, my command keeps hanging. I'm not sure why this is happening. I'm running my `docker`on a MacOS system.

# Answer
You can get to know if your site is up and running in the following way,

Map both the ports of inside container to the host using **-p** option and try to `curl localhost:port`

*Scenario:-*

Run your container with the following command,

```
docker run -d --name website -p 80:80 -p 22:22 mob

```

Explanation:- `-p host_port:container's_port`

Now, `curl localhost:80` or `localhost:22`

If your service is running properly, you will have got the response

P.S:- Make sure nothing is running in 80 and 20 on host before running the **docker run** command
> 8 votes

# Answer
It depends on what you want to archive. If your container is intended to be accessed from outside, you need port mapping: `-p $HOSTPORT:$CONTAINERPORT` So you can use `-p 80:80` as an argument for `docker run` to expose port 80.

If this is not the case (e.g. backend server that is accessed by a reverse proxy) you could simply use `docker exec` to execute your `curl` command in the container. This also has the advantage that no external IP is needed, simply use `localhost`. It's also an better option to expose port 22 for ssh.

```
docker exec -it $CONTAINERNAME bash

```

where `bash` can be replaced by any login shell for interactive mode. You can also directly pass a command here like this:

```
docker exec -it $CONTAINERNAME curl http://localhost

```

The `execute` command has the benefit that it can be used in already running containers. So for example you could check if a service in the container is gone after a period of time.
> 2 votes

# Answer
Your command (curl 172.17.0.2 ) is hanging not due to expose port .It is clear
for security reasons, in Dockerfile EXPOSE command is not working, you need to manually need to add -p host\_port:container\_port in docker run command & it helps connect your container outside your machine.

If service is not running on 80 port , then you get message like this shell\> curl 172.17.0.2 curl: (7) Failed to connect to 172.17.0.2 port 80: Connection refused

curl command is hanging due to non interactive & non TTY shell .

Run your docker run command with -it option :

docker run -d -it -p 80:80 --name website mob
> 1 votes

---
Tags: docker, dockerfile, docker-machine
---


title: Personalize Jenkins job view per user

text:
# Question
Title: Personalize Jenkins job view per user
I am creating a Jenkins job.

This will build some artifacts and will deploy to one of our environments, i.e. `qa`, `staging`, `production` etc. (I am using a choice parameter for this purpose)

However we need to restrict access to environments to specific user, i.e. the `QA` team should only be able to deploy to the `qa` env and so on; hence, the choice parameter should display to each user, **only** the environments he/she can deploy to.

Is this feasible in Jenkins?

Is there any **other tool** that could someone use to create jobs that present a custom UI (hiding/revealing options/choices) to users based on permissions?

# Answer
Have a look at the Role Strategy Plugin. It allows you to assign roles per project or node, so you can configure detailed permission as required.

There are other CI tools with extended permission-management as well, but I see no need for them: Jenkins is an open source, stable CI system which works well on all important platforms and became a de facto standard. It has many plugins to archive nearly anything. Even rare use-cases can be handled by writing custom plugins/batch scripts.
> 1 votes

# Answer
If you can't find a plugin to do what you want, there is a more manual solution.

Create multiple upstream jobs that control the parameters and pass them to the downstream deploy job. Then you can restrict each group to only see the relevant upstream job.

It will result in semi-duplicate upstream jobs, but will guarantee appropriate separation.
> 1 votes

---
Tags: jenkins, continuous-integration
---


# Question
Title: Multiple Kafka brokers behind a single OpenShift service?
Is it possible to have several brokers behind a single TCP/IP port?

I.e., offer a single endpoint for Kafka clients, and having a scalable amount of brokers behind that?

In my company, some people are configuring a (scaling) Kafka server for us, and they insist that this is not possible, and that we need to configure every single broker as a separate endpoint...

This is in a OpenShift context; it would be nice to just be able to scale the Kafka pods up and down as needed, while using the usual loadbalancing to offer a single port to the outside world.

***EDIT:*** I am familiar with HAProxy; the question is about Kafka, not OpenShift/ HAProxy. Is there anything special to be configured (in the Kafka broker image)

* Are they all synchronized amongst each other?
* Are there examples for that somewhere?

Googling for "OpenShift Apache Kafka" etc. brings up plenty of examples, but all that I found seem to have a different service for each Apache Kafka broker (i.e., explicitly running 3 separate Kafka deployments inside OpenShift, with separate OpenShift services, one each). This is what I wish to avoid. I want to horizontally scale the number of Kafka processes behind HAProxy, and have one single external IP.

# Answer
Yes, you can. `HAProxy` is purpose built to balance *any* TCP or HTTP connection. My old company used to run every component of Kafka with HAproxy.

Here is a whole project dedicated to running auto-scaling Kafka in Docker using HAproxy: https://github.com/pozgo/docker-kafka

You can even see their HAproxy config here where they specify the broker check (port 9092): https://github.com/pozgo/docker-kafka/blob/master/haproxy.cfg
> 1 votes

---
Tags: openshift, kafka, loadbalancing
---


# Question
Title: What is the proper term for lead time from commit to deployment?
I know for example following DevOps relevant concepts related to time:

* MTTR - mean time to recovery
* Sprint length - fixed agreed time to hand over features to customer (what this specificall implies is part of DoD I think)

Now my question - how do you name the lead time from a code commit hitting the repository to a first running deployment where you could do a first acceptance test to the feature? To a production system?

# Answer
From code repository to any environment, as a running software would be "Time to Deploy" ( which is different from deployment time, btw ).

From code repository all the way to production environment, the best aproximation is time to market (*TTM*), taking aside that time to market does involve the conceptual design of a product.

But as you said you are tagging an pipeline, I should advise:

**Use the tags that makes sense: to you, your team and to your product.**

A common pipeline could be : BUILD \> TEST \> DEPLOY

But we could tag the same pipeline, making more steps visible, ie : SOURCE CHECKOUT \> CODE CHECK \> UNITARY TESTS \> BUILD \> INTEGRATED TESTS \> DEPLOY \> ACCEPTANCE TESTS

As a rule, if you have trouble on one step and must check the output to understand whats going on, this step could be divided in one or more tags.

If makes sense to you, you could even tag something like SOURCE CHECKOUT IDLE TIME ( the time took for the build machine to connect to the repository server, before downloading the source )

Hope this could be useful, happy tagging!
> 1 votes

---
Tags: terminology, continuous-delivery, metrics, speed
---


# Question
Title: How to export EC2 snapshot into S3 bucket?
I've created a snapshot of EC2 instance via:

```
aws ec2 create-snapshot --volume-id vol-xyz

```

In `aws ec2 create-snapshot help` we can read:

> Creates a snapshot of an EBS volume and stores it in Amazon S3.

Although I cannot see in which bucket it has been imported.

In addition, there is a command `aws ec2 import-snapshot` which imports a disk from the S3 bucket.

So basically I'm looking for some way of exporting EC2 snapshot into S3 bucket where I can see the image file in order to download it.

How I can do it?

# Answer
> 4 votes
EBS Snapshots aren't really disk image files. They're logical containers of pointers to chunks of compressed and deduplicated backup data that EBS stores in S3 when you create snapshots -- but these are stored in buckets that EBS owns and controls.

They are user-accessible via only one mechanism -- creating a new EBS volume from the snapshot. When you do this, the new volume is linked back to its source snapshot and EBS starts loading the data onto the new volume in the background, from the snapshot storage in S3. If you access a part of the volume that EBS hasn't yet restored, it immediately goes and fetches the chunks and puts them on the volume.

To create a disk image file of a snapshot, you create a new volume from the snapshot, attach it to a running instance without mounting it, and then use the same tools you would use for a standard hard drive -- tools like `dd` to read raw blocks from the snapshot, and write them to a file, which you can then download from the instance, e.g. using `scp`.

---
Tags: amazon-ec2, aws-cli
---


# Question
Title: Multi jobs over local (IntelliJ) and Jenkins
I have to build a way for my tests to work with multi jobs over any platform using IntelliJ and Jenkins

I have heard that I can do this with JUnit 5. I have not been able to find any references about this.

Can someone with experience in setting up multi jobs in Jenkins point me to some good references?

# Answer
You can check out this **MultiJob** plugin for Jenkins. Or if its the same job you want to run in different configuration you can make use of Multi-configuration job type in Jenkins.

I have tried searching for JUnit5 supporting multi jobs scenario as specified didn't find any decent information on the same.
> 1 votes

---
Tags: jenkins
---


# Question
Title: How can a Windows server be a slave to Jenkins running on Linux when there in an error installing the slave-agent.jnlp file?
I want to have a Windows Server 2016 slave node with a master Jenkins (version 2.89.2) running on Red Hat Enterprise Linux 7.x. I am trying to install the slave-agent.jnlp file from the Jenkins web UI on the Windows server. I run this command for PowerShell opened as administrator:

```
javaws -verbose C:\Users\Administrator\Downloads\slave-agent.jnlp

```

I receive this error:

```
Error: The field <jnlp><application-desc><argument> has an invalid value:

Launch File

<jnlp codebase="http://FQDNofJenkinsServer:8080/computer/FQDNofWindowsSlave/" spec="1.0+"><information><title>Agent for FQDNofWindowsSlave</title><vendor>Jenkins project</vendor><homepage href="https://jenkins-ci.org/"/></information><security><all-permissions/></security><resources><j2se version="1.8+"/><jar href="http://FQDNofJenkinsServer:8080/jnlpJars/remoting.jar"/></resources><application-desc main-class="hudson.remoting.jnlp.Main"><argument>32ba31ca5f150c2b3491802b0b60e56574d970a9e601cebad656a113fe325a85</argument><argument>FQDNofWindowsSlave</argument><argument>-workDir</argument><argument/><argument>-internalDir</argument><argument/><argument>-url</argument><argument>http://FQDNofJenkinsServer:8080/</argument></application-desc></jnlp>

Exception

BadFieldException[ The field <jnlp><application-desc><argument> has an invalid value: ,]
    at com.sun.javaws.jnl.XMLFormat$10.visitElement(Unknown Source)
    at com.sun.javaws.jnl.XMLUtils.visitElements(Unknown Source)
    at com.sun.javaws.jnl.XMLFormat.buildApplicationDesc(Unknown Source)
    at com.sun.javaws.jnl.XMLFormat.process(Unknown Source)
    at com.sun.javaws.jnl.XMLFormat.parse(Unknown Source)
    at com.sun.javaws.jnl.LaunchDescFactory.buildDescriptor(Unknown Source)
    at com.sun.javaws.jnl.LaunchDescFactory.buildDescriptor(Unknown Source)
    at com.sun.javaws.Main.launchApp(Unknown Source)
    at com.sun.javaws.Main.continueInSecureThread(Unknown Source)
    at com.sun.javaws.Main.access$000(Unknown Source)
    at com.sun.javaws.Main$1.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

```

I expect the command to work. I do not understand the error message. What should I do?

# Answer
You have two arguments that are empty:

```
<application-desc main-class="hudson.remoting.jnlp.Main">
  <argument>32ba31ca5f150c2b3491802b0b60e56574d970a9e601cebad656a113fe325a85</argument>
  <argument>FQDNofWindowsSlave</argument>
  <argument>-workDir</argument>
  <argument/>
  <argument>-internalDir</argument>
  <argument/>
  <argument>-url</argument>
  <argument>http://FQDNofJenkinsServer:8080/</argument>
</application-desc>

```

Your error message has a comma in it:

> BadFieldException\[ The field has an invalid value: ,\]

Other bad argument error messages on the internet have comma separated lists of bad arguments:

> BadFieldException\[ The field codebase has an invalid value: $$codebase,$$codebase\]

So I'd guess those two empty arguments are the problem.
> 1 votes

# Answer
Jenkins provides `agent.jar` and a secret required to make a connection from the node to the master. Make sure the node is configured properly (*Manage Jenkins**Manage Nodes**Node*)

Ensure Java 1.8 is present on the Windows agent. Download the `agent.jar` file on the Windows machine, and run the command Jenkins provides, using Windows command prompt.

```
java -jar agent.jar -jnlpUrl http://localhost:8080/computer/example/slave-agent.jnlp -secret e6073ce49423ed54df4a24f918973f50dc760dd440013d3b30403c4a2c7501fd -workDir "c:\jenkins"

```
> 0 votes

---
Tags: jenkins, windows, linux, jenkinsfile, jenkins2
---


title: CI/CD with a GPU cluster

text:
# Question
Title: CI/CD with a GPU cluster
With typical continuous integration environments, you configure an environment capable to execute compilation and test batches (agent, slave..) coordinated by a scheduler (master, server..)

But what if your "client" environment is a Graphics Processing Unit (GPU) cluster used to perform model trainings in different configurations? Is there any difference or would you just for example let the head cluster node incorporate a Jenkins slave? (or Bamboo agent etc)

# Answer
Since you're talking about CI/CD I presume you have the possibility to automate the model trainings in those configurations. Let's call the scripts able to do that `train_model_config_A`, `train_model_config_B`, etc.

Then you could have a wrapper script which checks an environment variable used to select which client environment you desire and invokes the corresponding `train_model_config_<blah>` script. Ideally translating the outcome of the training (whatever that is) into one or more results of the pass/fail type. Then such wrapper script can be integrated in a CI/CD pipeline like a custom test step/stage (or even a build one, if it produces any artifacts you might want to archive). Just like any test executed on a testbed incorporating some non-generic piece of hardware. In other words the CPU cluster makes no real difference.

You might not need to install the slave directly on the GPU cluster if the `train_model_config_<blah>` scripts (and thus the wrapper script as well) can be executed on some other hosts and remotely control the GPUs - you can then have the slave on some other host, leaving the GPU cluster free to only do its stuff.
> 2 votes

---
Tags: continuous-integration, cloud
---


# Question
Title: Is it possible to set a minimum Python version in Pygradle?
Pygradle is used to build a python project.

**Problem**

The problem is that the CI runs a different Python version, resulting in different build outcomes, e.g. the build succeeded on the laptop, but failed on CI.

*CI*

```
============================= test session starts ==============================
platform linux -- Python 3.6.3, pytest-2.9.1, py-1.4.29, pluggy-0.3.1
rootdir: /home/travis/build/user/project, inifile:
plugins: cov-2.2.1
collected 1 items

```

*Laptop*

```
============================= test session starts ==============================
platform linux2 -- Python 2.7.5, pytest-2.9.1, py-1.4.29, pluggy-0.3.1
rootdir: /home/user/dev/project, inifile:
plugins: cov-2.2.1
collected 1 items

```

**Discussion**

To ensure that the build fails both locally on CI and on the laptop, the same python version should be used.

**Question**

Is it possible to set a minimum Python version in Pygradle?

# Answer
Side note: personally I dislike using a *minimum* version: you never know beforehand how the language will evolve and if a certain future version will still be compatible with your code. For example many 2.7-compatible scripts can't actually work with python 3.X, setting 2.7 as the minimum version won't cut it.

I prefer to explicitly list as allowed the (existing) versions that I can test and confirm to be compatible. Any future version can be added later on, after such confirmation is obtained.

If you agree with the above then the `com.linkedin.python` plugin can apparently perform such version checks (it doesn't looks like it supports a *minimum* version):

> **Default and allowed Python version**
>
> This plugin enforces a set of default and allowed Python versions. For example, you can specify `pythonVersion = '3'` and you will get whatever the default Python 3 version is. Similarly `pythonVersion =
>  '2'` gets you whatever the default Python 2 version is.
>
> This plugin also enforces a set of allowed Python versions. If you choose a Python version that is not allowed, you will see an error messages such as:
>
> ```
> Python 3.2 not allowed; choose from [2.6, 2.7, 3.4, 3.5, 3.6]
>
> ```
>
> If you see this error message, you must adjust your `pythonVersion` setting to one of the allowed values.
>
> Consumers of this extension can change the default Python 2, Python 3, and allowed versions by making a call on the `PythonDetails` object, either the one returned by `PythonExtension.getDetails()` or on any `PythonDetails` instance you create, e.g. in Groovy:
>
> ```
> pythonDetails.setPythonDefaultVersions('2.7', '3.6', ['2.7', '3.5', '3.6'])
>
> ```

But I'm not familiar with pygradle, I can't really tell exactly how the plugin works and/or is configured to do the job.
> 0 votes

---
Tags: pygradle
---


# Question
Title: How to access ephemeral storage of EC2 instance?
I've got ephemeral storage setup in *Scalr* as part of the role. The instance is spinned up by *Chef* and it seems it is running fine (it's using `ephemeral0` device mounted as `D:` drive).

I've tried to use `aws ec2 describe-instances` command, but here is no any indication that the ephemeral device is attached to the instance (unless I don't know what I'm looking for).

How I can access ephemeral storages of the running instances using via AWS CLI/API?

# Answer
> 1 votes
You are correct that the `describe-instances` API doesn't include information about ephemeral volumes. Once an instance is created, the only way to know that it has such volumes is available from the metadata service that is accessible from the instance itself.

According to User Guide for Linux Instances, the mapped ephemeral volumes are shown in instance metadata.

Since you created the instances using Chef, another option would be to check the Chef state database - most probably the information about a created instance includes the ephemeral device information. But this method highly depends on your Chef version and the cookbook you use to create the instances, so YMMV.

---
Tags: chef, aws-cli, scalr
---


# Question
Title: Ansible Remote provisioning not working in Docker container build
I have a Packer template to start with a Docker container and provision it. The shell script provisioners seem to be working, but, while the playbook runs and seems to succeed, unlike the shell provisioners, it seems to not be applying to the Docker container.

packer-template.json

```
{
  "builders": [
        {
            "type": "docker",
            "image": "chenjr0719/ubuntu-unity-novnc",
            "commit": "true"
        }
    ],
   "provisioners": [
        {
            "type": "shell",
            "inline": ["sudo apt-get update"]
        },
        {
            "type": "shell",
            "script": "setup_ansible.sh"
        },
        {
            "type": "shell",
            "script": "add_ansible_roles.sh"
        },
        {
            "type": "ansible",
            "playbook_file": "./kubeadm.yml"
        }
   ]
}

```

setup\_ansible.sh

```
sudo apt-get install software-properties-common -y
sudo apt-add-repository ppa:ansible/ansible -y
sudo apt-get update
sudo apt-get install ansible -y

```

add\_ansible\_roles.sh

```
sudo ansible-galaxy install djx339.k8s-kubeadm-master

```

kubeadm.yml

```
- name: install kubeadm
  hosts: localhost
  roles:
    - { role: djx339.k8s-kubeadm-install }

```

Output from the playbook

```
docker: PLAY [install kubeadm] *********************************************************
docker:
docker: TASK [Gathering Facts] *********************************************************
docker: ok: [localhost]
docker:
docker: TASK [djx339.k8s-kubeadm-install : include_tasks] ******************************
docker: included: /home/dw/.ansible/roles/djx339.k8s-kubeadm-install/tasks/setup-Debian.yml for localhost
docker:
docker: TASK [djx339.k8s-kubeadm-install : Sysctl] *************************************
docker: ok: [localhost] => (item={u'name': u'net.bridge.bridge-nf-call-iptables', u'value': 1})
docker:
docker: TASK [djx339.k8s-kubeadm-install : Add kuebeadm apt key] ***********************
docker: ok: [localhost]
docker:
docker: TASK [djx339.k8s-kubeadm-install : Add kubeadm apt repo] ***********************
docker: ok: [localhost]
docker:
docker: TASK [djx339.k8s-kubeadm-install : Install kubeadm] ****************************
docker: ok: [localhost] => (item=[u'kubelet', u'kubeadm', u'kubectl', u'kubernetes-cni'])
docker:
docker: PLAY RECAP *********************************************************************
docker: localhost                  : ok=6    changed=0    unreachable=0    failed=0

```

Running `kubeadm` in the docker container throws an error.

Why isn't the playbook applying? It works when I run it locally.

The error which I'm getting is:

> kubeadm not found

So basically it's not installing `kubeadm`.

---

Note: just noticed the mismatch between the role I'm adding in the `add_ansible_roles.sh` script and the playbook.

# Answer
It appears that you are trying to run ansible locally **inside** the container but instead you are running it locally on the machine you use to run packer on(the provisioning one).

To run ansible in local mode you need to update the ansible part of your packer file to:

```
    {
        "type": "ansible-local",
        "playbook_file": "./kubeadm.yml"
    }

```

Also, I would suggest to add `connection: local` to your ansible playbook, to avoid unnecessary attempts for ssh connections.

```
- name: install kubeadm
  hosts: localhost
  connection: local
  roles:
    - { role: djx339.k8s-kubeadm-install }

```
> 3 votes

---
Tags: docker, ansible, packer
---


# Question
Title: Does Oracle have an API that shows the latest stable JDK version?
There is an ansible role that installs jdk8, but is incompatible with jdk9. The role parses a couple of html pages and the commands are incompatible with java9. It was quite hard to make it work for jdk9, and now the PR is broken for jdk8.

https://github.com/ansiblebit/oracle-java/issues/63

In order to prevent that a lot of work has to be done over and over again, it would be great if there is an API that shows the latest jdk version. Then this version could be used in the ansible role and then this will be more stable.

# Answer
http://api.releasesoftwaremoreoften.com/latestversion?name=oraclejdk8 http://api.releasesoftwaremoreoften.com/latestversion?name=oraclejdk9

Running:

```
curl -s http://releasesoftwaremoreoften.com/latestversion?name=oraclejdk9 | jq .latestVersion

```

returns:

```
9.0.4

```

and

```
curl -s http://releasesoftwaremoreoften.com/latestversion?name=oraclejdk8 | jq .latestVersion

```

returns:

```
8u162

```
> 1 votes

---
Tags: automation, oracle-jdk, api
---


# Question
Title: Using Jenkins to control docker image build flow
I'm trying to figure out the proper way to get build environments into a CI pipeline.

What I want to achieve is:

1. someone pushes to the docker source repo (not registry, a repo with the docker folder)
2. jenkins kicks off a build of that docker image
3. if successful docker then goes and runs a software build using the newly built docker image
4. if successful the docker image is tagged and pushed to the docker registry

All I have so far is a Jenkins file that has four stages

1. checks out out docker source repo
2. builds docker image from source
3. Checks out software source code (code that builds in the docker environment)
4. builds software source code in docker container
5. If successful tags and pushes new image to docker registery

My main concern is everything I'm doing is going to be in the 'script' blocks, except the git checkouts, and it feels super hacky to be building and kicking off docker builds with jenkins using bash...

are there any tools/already defined procedures to do things like this? Is this the best way to do it? I am looking for a best practice approach as this will be put into production and I want it to be as resilient and long lasting as possible.

# Answer
I have found a solution so I'll leave it here for others.

using docker

```
#!/usr/bin/env groovy

pipeline{
agent buildsvr

  stages{
    stage('Checkout and build Docker Source'){
      steps{
        checkout([
          $class: 'GitSCM',
          branches: [[name: '*/master']],
          doGenerateSubmoduleConfigurations: false,
          submoduleCfg: [],
          userRemoteConfigs: [[
            credentialsId: 'asdf787c-5fd4-4dfb-81ad-873jhf107223',
            url:
'git@mydockersourcerepo.git']]])
        sh 'pushd docker && docker build -t ${image}:latest . && popd '
      }
    }
    stage('Checkout Target Source'){
      steps{
        script{
          checkout([
            $class: 'GitSCM',
            branches: [[name: 'mybranch']],
            doGenerateSubmoduleConfigurations: false,
            extensions: [[
              $class: 'SubmoduleOption',
              disableSubmodules: false,
              parentCredentials: false,
              recursiveSubmodules: true, reference: '',
              trackingSubmodules: true
              relativeTargetDir: 'source']],
            submoduleCfg: [],
            userRemoteConfigs: [[
              credentialsId: 'Bitbucket-Jenkins',
              url: 'ssh://git@sourcerepo.git']]])
        }
      }
    }
    stage('Build Source Target'){
      steps{
       script{
          sh 'docker-compose up'
        }
      }
    }
  }
  post{
    success{
      script{
        sh 'docker tag ${image} 10.90.239.223:5000:/${image} && docker push
10.90.239.223:5000:/${image}'
      }
    }
  }
}

```
> 2 votes

---
Tags: docker, jenkins, jenkins-pipeline, docker-compose
---


# Question
Title: How to integrate docker-compose or docker run command arguments into SCM and Jenkins build?
When I do `docker run` I have to give extensive argument options for my container setup. To mitigate this and ensure all developers are using the same arguments I switched to docker-compose.

Now when I try to go run the same build from jenkins I can no longer use the docker plugin since no jenkins plugins seem to support `docker-compose up`, they all need to use the `docker` commands for the remote API instead of the `docker-compose` commands.

Unless there are reasons not to, I would prefer `docker-compose up`. I am aware I could create a bash script the does `docker run <arguments>` and let my jenkinsfile parse the script and load in the proper args but this feels ultra hacky.

How can I use one or the other and keep them both in SCM and ensure that if a change is made to docker source (either the `docker-compose.yml` or possible a bash script running `docker run` with the proper arguments), it will work in both places? Are there tools for this? a plugin I am missing?

Also I desire to use CoreOs which doesn't natively support jenkins (no java and other things).

# Answer
> 0 votes
If you have docker compose installed on the Jenkins Build machine you can use `sh docker-compose up` to run docker-compose from a new shell.

In general, you can run any command from your JenkinsFile using the `sh` command.

---
Tags: docker, jenkins, jenkins-plugins, jenkins-pipeline, docker-compose
---


title: Automated patching of docker images

text:
# Question
Title: Automated patching of docker images
Docker and other container images have become important building blocks especially for software production pipelines. Unfortunately, several researchers (e.g. BanyanOps) have shown how vulnerable docker images can be , including those from the official distributors. I was researching into the possibilities for automatic patching of such vulnerable images when discovered. According to this link, container patching is very sensitive and error-prone. Infact, there seems to be no solution as all the papers/reports about container image vulnerabilities do not proffer patching approaches. I will be grateful if someone has some opinions, options or even a possible solution.

**Note**: I am interested in how patching is conducted with or without update notifications.

# Answer
As this question is quite high-level, I give a high level answer.

Technically, if there is an incident, you would normally enforce a shutdown of a running container and a replacement with a newer image with an implementation of the change request resulting from the incident.

How much automation will be possible to get there, I think it really dependends on single cases or at best their classes.

**Best is therefore I think to assess different scenarios, group them into classes and derive solution workflows. Once a problem catalogue and a required set of organization policies are there you will be able to sort out which tools can help to realize them and which automations grades are possible.**

For example:

* discovered system environment level vulnerabilities are normally patchable through operation system security upgrades. You could have batch job delivering daily updated base images.
* application environment level changes like "let's migrate from Java 8 to Java 9" is an organizational decision which requires an organizational process
* application code level vulnerabitites requiring updating to patched versions requires some ways of dependency management governance in your organization, tools for quality gates are available but without an organizational decision nothing will happen here.
* default credentials in production - again where do they come from? This is very special case and possibly a design failure; not sure how solve this in an automated way.

So I have to conclude that this question is more about culture than automation.
> 4 votes

---
Tags: docker, continuous-integration, culture, build-pipeline
---


# Question
Title: Does QA in DevOps roles means CI/CD or more?
One of views on DevOps is to say that it combines fileds of Development, Operations and Quality Assurance.

Now many DevOps roles have focus either

* on development with the left shift in terms of operations awareness (up to a No-Ops full stack, *you build it - you run it* (c) Besos, Amazon's CTO),
* or a sysadmin who is expected to write CaSC (Configurarion as Code), IaC (infrastructure as code) and automation codes to reduce work required in operations.

In both cases you see lots of tools typical for these roles with and without DevOps context.

What does QA DevOps roles do today as a formal commitment? Test automation always has been there, for example. Is QA just meant here to support CI/CD infrastructure and teams working on it, or something else?

# Answer
> 2 votes
First, DevOps is a journey, not a destination. Not every organization approaches similar problems with the same solution; while automation is a component of a DevOps platform, it's not the single defining characteristic.

The role and responsibilities of QA professionals haven't really changed; they're still responsible for validating that code is ready for release. What HAS changed is the focus on both automation and smaller releases. Much like operational roles, QA staff should now have the opportunity to focus on adding additional automatic quality checks into the development life cycle.

In a shop with a mature CI/CD pipeline, QA becomes more about designing great tests, and letting the software do the actual heavy lifting. As Tensibai commented above, "some QA tests can't be automated" (yet), so there will still need to be some engineering responsibilities, but the goal should be to utilize automation in every aspect of delivery in order to facilitate adding additional quality.

# Answer
> 1 votes
This is an open-ended question, which is why the following answer is an opinion.

The focus of QA (quality assurance) in the DevOps field is to monitor the deployment metrics to ensure that faults are caught before they can manifest. The most direct metrics of application health include application availability and errors. These can be directly extracted from error logs and automated monitoring of served web pages (that status code 200 or the login form are always returned).

The next level allows you to anticipate errors, bottlenecks and resource usage. This is very deployment specific. In some cases you may have a micro service that receives a high load or in others a single page may make several database queries. Monitoring your servers' CPU load and memory usage can also be used as metrics. These may act as key indicators to your applications' health.

Other aspects delve into feature and device usage which enable you to make informed decisions about depreciating these in order to limit your application's complexity. Since complexity is directly linked to bugs and development velocity, it makes sense to limit it where possible.

Sources: Telerik and Stackify

# Answer
> 1 votes
The key aspect in a Dev(Sec)Ops-oriented organisation is the inter-field communication and collaboration. From this perspective, the QA role can include activities such as:

* continuous improvement of automated QA coverage/effectiveness, reliability, performance (which ideally should target CI/CD, but not always is/can be)
* direct, active/constant involvement with Development:
    + for TDD-based development
    + for reproducing difficult customer-found defects, developing strategies for verifying their fixes and plugging the holes that allowed such bugs to reach the customer
* feature, roadmap, resource planning, in which all fields should be involved

For organisations building software delivered for deployment and operation by 3rd party customers (like IT infrastructure equipment or consumer products, for example) the QA role could bring their expertise (ideally alongside Development) in customer-facing interactions (maybe just in an assisting role) to improve the understanding of how the customer is using the product and the challenges they face and bring that feedback into the aspects of the product lifecycle under their influence.

For organisations which also deploy and operate the software being produced the QA role may extend to/blend into the Security/Operations field while closing the loop back towards Development and itself, for example:

* ensuring the product quality matches expectations
* managing SLAs
* constantly aiming for reducing deltas between the production environment and the Development/QA one(s), regardless of them being inside of the CI/CD pipeline or not

---
Tags: culture, methodology, job-role
---


# Question
Title: Count of currently active jenkins agents
Is there an easy way to get a count of how many agents are currently up? I know I can go to `www.myjenkins.com/computer/` to see the agents, but counting them manually is tedious.

Is there an API call or something I can run in the script console that just outputs the count for me?

# Answer
> 3 votes
You can sufix `computer/` with `api/json` and run some script on this response. Count the computers array or group by some one of his attributes, like `idle` or `offline`.

`https://www.myjenkins.com/computer/api/json`

---
Tags: jenkins
---


title: How to define constants in Ansible?

text:
# Question
Title: How to define constants in Ansible?
If one writes a constant in Python, then one uses uppercase:

```
SOME_CONSTANT = "helloworld"

```

Now I wonder how to define constants in Ansible. My first impression was to use the same convention as in Python as Ansible is written in python, but when one checks the variables information page then this does not seem to be a best practice.

**Current approach**

https://github.com/030/ansible-firefox/blob/master/vars/main.yml

```
---
firefox_download: /tmp/firefox-{{ firefox_version }}.tar.bz2
firefox_bin: "{{ firefox_home }}/firefox/firefox"

```

https://github.com/030/ansible-firefox/blob/master/tasks/main.yml

```
- name: Download.
  get_url:
    url: https://ftp.mozilla.org/pub/firefox/releases/{{ firefox_version }}/linux-x86_64/en-US/firefox-{{ firefox_version }}.tar.bz2
    dest: /tmp/firefox-{{ firefox_version }}.tar.bz2
checksum: "{{ firefox_checksum }}"

```

**Discussion**

I am considering to define `https://ftp.mozilla.org/pub/firefox/releases/{{ firefox_version }}/linux-x86_64/en-US/firefox-{{ firefox_version }}.tar.bz2` as a constant in the `vars/main.yml`, but I did not see any other ansible roles that define constants as:

```
FIREFOX_DOWNLOAD_URL: https://ftp.mozilla.org/pub/firefox/releases/{{ firefox_version }}/linux-x86_64/en-US/firefox-{{ firefox_version }}.tar.bz2

```

while this is valid python and the convention to write constants.

One could argue that an Ansible constant could be defined in the `default/main.yml`, but on the other hand, as a CONSTANT is immutable it should not be possible to overwrite it and should reside in `vars/main.yml`.

# Answer
As pointed out in the other comment, there are no constants in python. It's just a convention, but defining a variable in capitals, doesn't mean you can't reassign the value.

There are many ways to pass/define variables in ansible (command line, `vars` in playbook, include a `yaml`, `vars` dir in a role, `defaults` dir, etc) however, there isn't a way to define a **constant**.

The closest to setting a constant in the way you ask for it is setting your variables as defaults(that is `defaults/main.yml` inside the role structure). The defaults of a role are the type of variables that have the lowest precedence and are being overwritten by any other way you can define a var but are a great place to put a variable that is unlikely to change and won't clutter your "important" role vars.

Regarding the way you define them - I'd say it's up to you. Just stick to it. I haven't seen anybody using capitals and I would probably make the variable lower case, however, since ansible provides tremendeous amount of flexibility and freedom to do what you like the way you like it, it really is up to you.
> 3 votes

# Answer
Note that `SOME_CONSTANT = "helloworld"` is just following a PEP8 convention, one can always follow that statement with `SOME_CONSTANT = "something_else"`. Python doesn't really have constants, see How do I create a constant in Python?

I'd use lowercase (i.e. following the Python variable naming convention), to prevent confusion:

* inline with the ansible documentation
* no surprises when (some of them) are internally changed to lowercase, for example in Local Facts (Facts.d):

> **Note**
>
> The key part in the key=value pairs will be converted into lowercase inside the ansible\_local variable. Using the example above, if the ini file contained `XYZ=3` in the `[general]` section, then you should expect to access it as: `{{ ansible_local.preferences.general.xyz }}` and not `{{ ansible_local.preferences.general.XYZ }}`. This is because Ansible uses Python’s ConfigParser which passes all option names through the optionxform method and this method’s default implementation converts option names to lower case.
> 1 votes

---
Tags: ansible
---


# Question
Title: What are Pros and Cons of various VCS and CI/CD tools to store, change and deploy configuration snippets of more than 1000 customers in a VCS?
Follow-up to this question.

We are in the middle of a transition to Configuration as Code (CaC). At the moment a lot of sites are configured manually. A side effect of this is that so now and then configuration gets broken and nobody knows what happened. By applying CaC the Version Control System (VCS) could be consulted when an environment is broken and then it is easier to figure out the cause of the issue. Another advantage is that people cannot change the configuration directly on the production systems.

One of the comments to this Q&A was whether the configuration is shared between customers. At the moment this is not the case and this is something we would like to prevent as sharing too much could also be a disadvantage as this could cause tangled configuration, i.e. spaghetti.

In summary, there are multiple configuration files per customer site.

**Process**

When the configuration is changed and merged into master or trunk then the configuration has to be deployed.

There are more than 1000 customer environments that have to be migrated. One approach is to create 1000 VCS repositories. An advantage of this is that a configuration change will result in a deploy to a specific environment, e.g. code pushed to VCS customer0001 will deploy the configuration to customer0001.

**Discussion**

As there are more than 1000 customers, I wonder whether 1000 repositories will be a good idea in terms of costs, maintenance and so on. What about creating a separate VCS for the configuration and create a folder per customer, i.e. a monorepo. E.g.:

*VCS repository*

* customer0001
    + configFileA
    + configFileB
    + configFileC
* customer0002
    + configFileA
    + configFileB
    + configFileC
* customer0004
    + configFileA
    + configFileB
    + configFileC
* customer0005
    + configFileA
    + configFileB
    + configFileC
* customer0998
    + configFileA
    + configFileB
    + configFileC
* customer0999
    + configFileA
    + configFileB
    + configFileC
* customer1000
    + configFileA
    + configFileB
    + configFileC

An advantage is that it is clear that all configuration resides in one repository, but how to trigger a certain build if a certain customer folder change and deploy it to the environment of the customer?

It is possible to create a custom groovy script in Jenkins that does some filtering or use branches in certain VCS repositories like bitbucket, but I wonder whether there are more efficient solutions.

**Preferred answer**

A table that gives an overview of the pros and cons:

```
| VCS | Monorepo | CICD     | Pros               | Cons                  |
|-----|----------|----------|====================|=======================|
|     |          | Circleci |                    |                       |
|-----|----------|----------|--------------------|-----------------------|
|     | true     | Jenkins  |                    | groovy script required|
                                                 | to filter customer    |
                                                 | folders               |
                            |--------------------|-----------------------|
|     | false    | Jenkins  | code change results|more than 1000 jenkins |
                            | in deploy to       |jobs required          |
                            | associated customer|                       |
                            | environment        |                       |
|                           |--------------------|-----------------------|
| git |          |          |                    |                       |

```

# Answer
> 3 votes
Personally I'd keep them in separate repositories, for clear customer isolation:

* minimal/no risk of unwanted interference between customers
* different access control for different customers is possible
* different CI/CD pipelines and/or configs for different customers is possible
* simpler/standard CI/CD configurations
* clean per-customer repository history
* easier calculation of per customer resource usage
* lower maintenance risks - better to have just a few customers down than all of them at once

I see some cons as well, but IMHO not enough to tip the balance:

* more resources needed (one CI/CD pipeline per customer), but these costs can be reflected in the service pricing. And they should get a lot smaller due to automation.
* sharing of configs/procedures across customers becomes difficult

If a unified VCS vision/presentation is desired I'd still keep separate per-customer repositories (thus keeping some/all of the pros above), collectively managed under a single umbrella repository:

* the umbrella repository would contain just a manifest file listing the per-customer repositories, their versions and maybe some other relevant info
* changes in one or more of the individual repositories need to be accompanied by corresponding manifest changes in the umbrella repository to advance the overall project
* a small set of wrapper tools would be used to operate at the umbrella repository level but controlling the individual repositories

---
Tags: configuration, monorepo, vcs, cicd
---


# Question
Title: Permission errors with non-root USER when running on Linux
I have been getting some unexpected failures with the execution of my Docker images when running on my Ubuntu 16.04.3 LTS machine. I'm using Docker version 18.02.0-ce (edge), installed from apt. I've narrowed down the problem enough to demonstrate it with this Dockerfile:

```
FROM tomcat:8.5.24-jre8-alpine

RUN adduser -D test

RUN chown -R test "$CATALINA_HOME"

USER test

RUN ls "$CATALINA_HOME"/webapps

```

When I attempt to build this, here's the result:

```
$ docker build -t test test/
Sending build context to Docker daemon  163.4MB
Step 1/5 : FROM tomcat:8.5.24-jre8-alpine
 ---> 134d23a9a14e
Step 2/5 : RUN adduser -D test
 ---> Using cache
 ---> a6f70ba83b8d
Step 3/5 : RUN chown -R test "$CATALINA_HOME"
 ---> Using cache
 ---> 8a0fae3db0a8
Step 4/5 : USER test
 ---> Using cache
 ---> 45fdf333232b
Step 5/5 : RUN ls "$CATALINA_HOME"/webapps
 ---> Running in 48711dfb9569
ls: /usr/local/tomcat/webapps/ROOT: Permission denied
ls: /usr/local/tomcat/webapps/examples: Permission denied
ls: /usr/local/tomcat/webapps/docs: Permission denied
ls: /usr/local/tomcat/webapps/host-manager: Permission denied
ls: /usr/local/tomcat/webapps/manager: Permission denied
The command '/bin/sh -c ls "$CATALINA_HOME"/webapps' returned a non-zero code: 1

```

These "Permission denied" errors are very strange to me, since the `chown` command should have given ownership of the whole /usr/local/tomcat folder, including the subfolders.

I have tried this same scenario with many different versions of Docker on my Ubuntu host, all with the same result. However, when I tried building this image on my Mac OS machine, it worked fine.

One slight variation that I have found which works around the problem is to give the "test" user access to the "root" group, like so:

```
RUN adduser -D test -G root

```

With this simple change, the Dockerfile builds. So it seems obvious to me that somehow group membership permissions are involved, but I can't imagine why that would matter since the user is the owner. Also I can't imagine why building on Linux is different than building on Mac OS.

# Answer
> 2 votes
As described in https://github.com/docker-library/tomcat/issues/35, this is related to the storage driver used (aufs). Changing to use overlay2 solved the issue:

https://docs.docker.com/storage/storagedriver/overlayfs-driver/

---
Tags: docker, linux, permissions, user-namespaces
---


title: Persistent data in MySQL with Docker

text:
# Question
Title: Persistent data in MySQL with Docker
If I startup MySQL server during `docker build` (derived from official MySQL Dokcer Image) in two different statements, would be database files be pertained?

I would say yes, but it seems to be wrong. Why?

Logfile:

```
180228 08:54:50 mysqld_safe Logging to '/var/lib/mysql/c2712433a4da.err'.
180228 08:54:50 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
mysql: [Warning] Using a password on the command line interface can be insecure.
mysql: [Warning] Using a password on the command line interface can be insecure.
Tables_in_FOOBAR
address
category
Removing intermediate container b3ff83bbf273
 ---> 0b366a0d31a9
Step 9/9 : RUN chmod 777 /var/lib/ -R && ./ingest_data.sh
 ---> Running in 149361f7d465
180228 08:55:00 mysqld_safe Logging to '/var/lib/mysql/c2712433a4da.err'.
180228 08:55:00 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
mysql: [Warning] Using a password on the command line interface can be insecure.
Database
information_schema
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 1049 (42000): Unknown database 'FOOBAR'

```

As you can see, in the first RUN statement I can list tables I have created in a new database, but in the second RUN statement the database is gone.

Note. Normally, you would not transport your application data in production containers (immutability principle) but for demo and test purposes this can be nevertheless useful.

# Answer
> 2 votes
That's a known issue .. since image layers are built using containers, data saved to folders does not make it to the next layer or your final container

```
Removing intermediate container b3ff83bbf273

```

If you want data to persist between updated containers you have to manually map data to the host outside the container-specific mount points.

For example, when creating a container use the -v option to map a volume:

```
docker run -d -v /var/container_data/mysql:/var/lib/mysql mysql_image

```

For a detailed documentation https://docs.docker.com/engine/reference/run/#volume-shared-filesystems

---
Tags: docker, mysql
---


# Question
Title: Error when using 'ansible-local' provisioner with Docker container
I'm getting an error when running the `ansible-local` provisioner with a docker builder in Hashicorps Packer.

My template (template.json):

```
{
  "builders": [
        {
            "type": "docker",
            "image": "ubuntu",
            "commit": "true"
        }
    ],
   "provisioners": [
        {
            "type": "shell",
            "inline": ["sudo apt-get update"]
        },
        {
            "type": "shell",
            "script": "setup_ansible.sh"
        },
        {
            "type": "ansible-local",
            "playbook_file": "./kubeadm.yml"
        }
   ]
}

```

The error:

```
docker: Executing Ansible: cd /tmp/packer-provisioner-ansible-local/5a8f3bad-8413-16c3-b255-6a06a9076ef3 && ANSIBLE_FORCE_COLOR=1 PYTHONUNBUFFERED=1 ansible-playbook /tmp/packer-provisioner-ansible-local/5a8f3bad-8413-16c3-b255-6a06a9076ef3/kubeadm.yml --extra-vars \"packer_build_name=docker packer_builder_type=docker packer_http_addr=\"  -c local -i /tmp/packer-provisioner-ansible-local/5a8f3bad-8413-16c3-b255-6a06a9076ef3/packer-provisioner-ansible-local135505282
docker: ERROR! the playbook: packer_builder_type=docker could not be found

```

# Answer
It seems that there is some bug in Hashicorps Packer 1.2.0, consider downgrading or upgrading to v1.2.1, comments in GH issue page confirm that the deployed fix is working.
> 1 votes

---
Tags: docker, ansible, packer
---


# Question
Title: How do I provision a Windows Docker container with Ansible?
I want to provision a Windows Docker container with Ansible, but I am confused about how exactly it will work.

For instance if I use Packer with Linux, I know I can run `ansible` (which is ansible remote) and `ansible-local`. I have to build my Windows Docker containers from a Windows computer and the Windows computer cannot be an Ansible master at this point.

Does any tool, possibly Packer for Windows handle things automatically and spin up a Linux Ansible Master VM to run the playbooks automatically?

How can I easily provision Windows containers with Ansible as part of my image build process?

I am currently using Windows 10 Enterprise and Windows Server 2016 is on the horizon.

# Answer
> 1 votes
I've always had issues with using `ansible` to provision Windows machines with Packer due to the `winrm` connections breaking, example issue.

A workaround which you can do is to use the `shell-local` on your host machine which calls the Ansible playbook with the new host (although you would need to give it the IP address of your Windows Docker machine).

I've added the example which I did to get a test working. This uploads `ip.cmd` to the Windows virtual machine, which gets its IP address, and is then downloaded back to the `destination`. The ansible playbook is then called against that.

```
"provisioners": [
        {
            "type": "file",
            "source": "ip.cmd",
            "destination": "C:/tmp/ip.cmd"
        },
        {
            "type": "windows-shell",
            "inline": [
                "echo [{{ user `ansible_group` }}] > C:/tmp/hosts",
                "C:/tmp/ip.cmd"
            ]
        },
        {
            "type": "file",
            "direction": "download",
            "source": "C:/tmp/hosts",
            "destination": "./ansible/hosts"
        },
        {
            "type": "shell-local",
            "command": "ANSIBLE_CONFIG=./ansible.cfg ansible-playbook -v -i ./ansible/hosts -l \"{{ user `ansible_group` }}\" -e \"ansible_user={{ user `username` }} ansible_password={{ user `password` }} ansible_become_pass={{ user `password` }} ansible_port=5986 ansible_connection=winrm ansible_winrm_server_cert_validation=ignore \" ./ansible/site.yml"
        }

```

where `ip.cmd` gets the IP address of the Windows host:

```
@echo off
FOR /F "tokens=2,3" %%A IN ('ping %computername% -n 1 -4') DO IF "from"== "%%A" set "IP=%%~B"
echo %IP:~0,-1% >> C:/tmp/hosts

```

source

---
Tags: docker, ansible, windows
---


# Question
Title: In ansible how to select specific tasks in a role according to the playbook or a more general parameter?
My understanding of Ansible roles is that they are the unit of reusability when implementing processes using Ansible, similarly to what a class or a package is in some computer languages.

It therefore seems natural to use roles to make all tasks understanding a given data structure in a specific role, but this yields the question of how to select the actual tasks to perform? Is it a good design to use the `main.yml` file in the task as an entrypoint to select actual tasks according to role parameters?

For instance a `framework` role could feature:

* a `defaults/main.yml` file that understands the `framework` variable that could be define by a vars file shared by all playbooks in a project like

framework: java: \[tomcat, mysql\] php: yes

so that after processing the `defaults/main.yml` file the following variables:

```
framework__enable_java_core: yes
framework__enable_java_tomcat: yes
framework__enable_java_mysql: yes
framework__enable_php_core: yes

```

are defined and can be used by the role to actually know which tasks to perform.

* several tasks like `java.yml`, `php.yml` pro framework, which select actions to perform according to a `when: action == 'WHATEVER-ACTION` is to perform; the `main.yml` then leek all the corresponding tasks.
* playbooks that say

for instance

```
roles:
- { role: framework, action: apt-setup }

```

or

```
roles:
- { role: framework, action: install }

```

or

```
roles:
- { role: framework, action: uninstall }

```

This kind of approaches turns out to scale very well when working with Makefiles. It is therefore tempting to use it again when writing complex ansible systems. On the one hand, it seems to fit quite naturally in the language used by ansible. On the other hand, it generates quite a payload of “skipped” tasks in ansible logs, which raises a red sign to me.

Is the approach I describe a sensible way to group data structures with the processes using it? (That is, to implement reutilisation!) If not, what would be a better way?

# Answer
Different approaches can be useful for different setups. You can use *tags* in some cases, or *conditional includes* (as you describe) in other cases.

> On the other hand, it generates quite a payload of “skipped” tasks in ansible logs, which raises a red sign to me.

You should get familiar with difference between static and dynamic includes in Ansible.

So if you use static includes, you will get a lot of skipped tasks, because every task in every file is added to execution queue but most of them are skipped because of `when` statement.

`tasks/main.yml` of your role with actions `install`/`uninstall`:

```
- include: install.yml
  when: action == 'install'
- include: uninstall.yml
  when: action == 'uninstall'

```

But if you use dynamic includes, only required tasks are added to execution queue and `skipped` message is displayed for skipped includes.

`tasks/main.yml`:

```
- include_tasks: install.yml
  when: action == 'install'
- include_tasks: uninstall.yml
  when: action == 'uninstall'

```

More on that, you can avoid `skipped` message altogether like this:

`tasks/main.yml`:

```
- include_tasks: '{{ action }}.yml'

```

The drawback of dynamic includes is that you can't do complete static analysis of your playbook (like syntax checks, etc.) because many things are know only in runtime.
> 8 votes

---
Tags: ansible, architecture
---


# Question
Title: Is there an overview of current resource usage in a GCP region?
When a container cluster was created on GCP in the morning it took less than 10 minutes until it was created. Now, after more than half an hour the cluster is still not created. It turned out that the resources in a certain zone were insufficient. How to check whether there are enough resources in a GCP zone before creating for example a kubernetes cluster?

```
Instance 'X' creation failed: The zone 'projects/Y/zones/Z' does not have
enough resources available to fulfill the request. Try a different zone,
or try again later.

```

After choosing another zone and deploying the cluster took less than 4 minutes to deploy it.

# Answer
> 0 votes
This page indicates something about regions and zones, but does not show any resource usage. I think that the resource usage of a zone is not visible.

# Answer
> 0 votes
My experience is, if there is insufficient resource, it just throws error saying to choose another region but have never taken more than 5min to live cluster.

---
Tags: google-cloud-platform
---


# Question
Title: How to perform logging of Ansible crons in Kibana?
I have set up some crons in Ansible, and I want to log when they start, finish, and if they fail.

How I could do it?

I don’t know so much about Ansible.

# Answer
Ansible is an automation tool, so it can automate things you can do by hands.

But as far as I know no existing cron-like services support sending logs to elasticsearch (Kibana is just a viewer, btw) out of the box.

So at first, you should create some wrapper script that will execute your actual job and send it's result to elasticsearch.

An then schedule this wrapper script via Ansible.

Or choose another route, setup syslog forwarding to logstash/elasticsearch on your target box to collect all logs, then just filter your desired cron jobs.
> 1 votes

# Answer
I don't think there is anything to do with ansible after you add cron. How about writing log of cron on file or send it by otherway maybe through el and view on kibana?
> 0 votes

# Answer
I would suggest checking out ansible callback plugins - they are python modules that implement callback functions that ansible will call during playbook execution. In fact, the messages ansible-playbook logs to your terminal is actually implemented as a callback plugin.

I have written callback plugins for logging to elasticsearch, influxdb, redis - they are quite powerful for hooking the results of your ansible runs up to other tools. Unfortunately I can not share them here as they were implemented during my day job.

Here are the callback plugins bundled with ansible by default: https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/callback \- it's possible you may be able to use one of them to send results to logstash, which would feed them into Elasticsearch for viewing with Kibana.
> 0 votes

---
Tags: ansible, elastic
---


# Question
Title: Minikube - Is there a way to run Windows Containers?
How do I run Windows containers on Minikube?

Is it possible?

# Answer
It is not officially supported (as of Minikube 0.25.0, Kubernetes 1.9, January 2018). But there is Beta support for Windows server containers in Kubernetes.

These articles contain more information:
> 7 votes

---
Tags: containers, windows, minikube
---


# Question
Title: What is the impact of using Helm Deployments instead of StatefulSets for Databases like MongoDB or MySQL?
The definition of a stateless app seems to apply to database servers like MongoDB or MySQL. I found Helm Charts as a kind of template-repository for Kubernetes. However, the stable builds all use deployments. Only in the incubator, which are not provided as stable.

**What is the impact of using Deployments for DB Servers?**

Are there any issues like sync/replication issues, when running multiple instances in a cluster, for example?

Background: I want to run MongoDB in a failover-cluster, so at least 2 instances are running.

# Answer
> 5 votes
Back when Kubernetes announced the new StatefulSet feature with K8s v1.5 (converting it from the old `PetSet` name), they put out a really good blog post walking through an example of its usage: https://kubernetes.io/blog/2016/12/statefulset-run-scale-stateful-applications-in-kubernetes/

The first paragraph has a really good description of differentiating features that always stuck out in my mind:

> ...“at most one pod per index” for deployment of the Pods in the set. Along with ordered deployment, ordered termination, unique network names, and persistent stable storage ...

Break down the important ones:

## Unique network names

In a deployment, it doesn't matter the network name as any pod should be able to pick up the work. However, when you are running things like a stateful cluster, you might have the concept of 'masters' and 'slaves'. This unique network naming will allow you to keep the distinction by referencing a *specific* pod, with a predictable incremental name.

## Ordered Termination

This is important for re-deployments in a stateful cluster as you could have critical machines that need to stay up longer. If you have coordinator or 'leader' nodes, those should be the last to be restarted. StatefulSets allows you to do this. The linked blog post even shows an example of this.

## Persistent stable storage

Arguably, this is the *most* crucial difference. By default, StatefulSets retain their storage. Kubernetes does *not* automatically delete the storage that was attached to a StatefulSet. The same *can* be achieved with deployments, but its not the default.

---
Tags: kubernetes, databases, helm
---


title: trigger command on registry update

text:
# Question
Title: trigger command on registry update
I'm fairly new to CI/CD so this is probably a noob question.

Anyway, here's my current setup: * Bitbucket repo * Registry service on AWS (ECS Registry) * Pipeline on Bitbucket that builds a docker image on master branch update and pushes it to the registry.

So far so good, I'm missing the last part where my server (EC2 instance) listens for registry updates and executes a deployment command like docker-compose pull && etc.

What tools can your recommend me to do this?

# Answer
The key words here are:

> Pipeline on Bitbucket

You already have a pipeline. *This* is where you would stick the deployment command.

AFAIK there is no standard tooling for registries to 'push out' any changes; you'd have to code a custom integrator that continually polls for changes and compares. This would be a lot of work. Same goes for having your servers directly listen for changes on the registry.

It would be far easiest (and is the standard) to simply have your build pipeline issue the deployment command after its successfully pushed a new image to the registry.

There are benefits to this:

* your pipeline is aware of whether the push to the registry failed or passed (and you can handle that failure gracefully with retries)
* your pipeline will be aware of the new registry tag
* your pipeline can control *exactly* which tag gets pushed where, when, and how quickly
* your pipeline has the same insight to whether the deployment passed or failed, and can attempt retries or perform notifications of failure
> 2 votes

---
Tags: docker, continuous-deployment, registry
---


title: How to apply DRY to k8s files?

text:
# Question
Title: How to apply DRY to k8s files?
At the moment there are more than 100 different deployment files. The majority of the config is identical, but there are some differences and therefore every docker image has its own deployment template. If a change has to be made to one of the deployment.yamls, most of the time this has to be copied to other files as well and that is a little bit tedious. How to apply Don't Repeat Yourself (DRY) while constructing k8s files without creating spaghetti?

# Answer
> 5 votes
This is a similar problem I've been facing. And we're not the only ones it seems. GitLab just added a very interesting feature in their recent 10.5 release where you can pull external files into the pipeline.

In their blog post, they had this to say:

> For many of our largest customers, the DevOps team is responsible for providing CI/CD pipelines to a large number of development teams throughout the organization. Previously, this was a painful process to manage. There wasn’t a scalable way to distribute reusable pipeline configuration which meant code needed to be manually copied between multiple .gitlab-ci.yml files in multiple projects. This was a labor-intensive and error-prone process. Additionally, it didn't provide adequate controls to ensure testing and deployment is consistently enforced for each repo.
>
> Starting with 10.5, you can now include external files in CI/CD pipeline definition. Included files can be either local (contained in the same repo) or remote (accessible via HTTP/HTTPS). Including local files allows a large and complex .gitlab-ci.yml to be broken up in to modular chunks that are easier to maintain.

https://about.gitlab.com/2018/02/22/gitlab-10-5-released/#include-external-files-in-cicd-pipeline-definition

This is probably the best way to approach it from a OOP perspective as well: modularize certain aspects of your builds and include it in downstream code.

My peers working at another company do a similar approach with their Jenkins builds. They've essentially build on a 'makefile' for their builds that is based on templates and evaluates certain variables set in the Jenkins UI. Based on those variables, the code will then fetch (curl or git) 'module' files from a central 'builds' repository. It then uses all the modules to compile a single pipeline bash script and runs it.

This allows central changes to be done to a single file, that all the builds inherit.

---
Tags: kubernetes
---


title: Does HA-git exist?

text:
# Question
Title: Does HA-git exist?
For the second time in three months, bitbucket is down

https://status.developer.atlassian.com/

Now I wonder whether HA-git is possible or we should switch. Thanks to this it will be hard to finish the work before the deadline...

What other options do we have for the future? I am considering to start with on premise git as this did not break as often as the Saas solution. It ran without issues for almost 2 years and if there was a issue with the gitlab I configured then I could at least solve it or create a new gitlab and push the git repo as is.

Currently everybody is blocked as neither pull requests could be merged nor code could be pushed or pulled.

I do not want this anymore and I am quite disappointed.

As @Tensibai indicated that it is possible to use multiple remote git's:

My reply: "yes you are right. It is possible to setup multiple remote. What if code is pushed to gitlab and github and a PR is created on github. How to ensure that the code will be updated on gitlab and vice versa without making it tedious for most engineers?"

Based on the advice of @Tensibai, two different git repository will be added as a remote and it will be investigated how it works out.

# Answer
So, I know I’ve praised GitLab in my other answer to your questions before, but it really is awesome. They have a an HA solution called GitLab Geo. It’s primarily meant to speed up GitLab usage for geographically distributed teams, but the other main benefit is that the read-only secondary server can be easily promoted to be the master:

https://about.gitlab.com/features/gitlab-geo/

But to get back to your specific question: no. There is not really a way to get true high availability if you are relying on someone else. You can only ensure true HA if you have control over the infrastructure. This is one of the downfalls of using all SaaS solution. You have to hope thei internal ops teams have a good implementation under the hood.

For that reason, I wouldn’t even recommend using something like GitHub or GitLab.com SaaS paid plans. You should look into self hosted options considering this problem has severely affected your team’s ability to reach deadlines.

As @Tensibai mentioned, this is achievable with multiple remotes, sync scripts, watch processes, and other “glue code”, but it would be a lot of work and a lot to maintain. Another solution would be to self host a Git solution inside a docker container. You could have the file store backed by something like EFS and the database hosted in Amazon RDS. I’ve not tried this setup myself, but the theory is there: make the application hosting ephemeral, keep the files and datastore in a proven high-available system. Amazon’s uptime numbers are really good, though you still have the issue on relying on others here. The difference here is that the *entire* system isn’t out of your control. In the event your application Docker instance goes down, simply spin it up somewhere else and change the DNS.

That all being said, self-hosted GitLab on the other hand *does* have the magic bullet Geo solution that I previously mentioned. I’ve used it and it’s really nice
> 5 votes

# Answer
Take a look at Gitea, https://gitea.io/en-US/, much easier to host yourself than Gitlab and you are in control if things go down. I think that by making the database backend HA you could roll out more Gitea frontends.

Another option to know about at least is Keybase private Git, https://keybase.io/blog/encrypted-git-for-everyone. Have not tried it or know anything about how it works.
> 3 votes

---
Tags: git, high-availability
---


# Question
Title: Microservices terminology i.e. distinguishing between "core " and "other service instances"
In a standard cloud based microservice application, there could be two main categories of microservices:

* `core services` such as service registry & discovery, api-gateway, load-balancer etc.
* `service instances` e.g customers, billing and postage services.

In this illustration, I used `core services` and `service instances` to distinguish the two main groups. Are there standard naming conventions for these two or is my classification/thinking right?

Here are some demo projects about the subject and each of these consists of the mentioned services: Weaveworks Socks Shop, Spring PetClinic and PiggyMetrics. These applications have top ranking stars on GitHub.

# Answer
The last couple of years I am/was part of couple of DevOps teams that are/were creating microservices. Some of the projects failed miserably.

Reasons:

* architect that only read a blog about microservices and claims that he is the expert of microservices
* team members that lack knowledge, but indicate that they know everything about it as the architect told them about it
* lack of DevOps mindset
* no support from management
* thinking that microservices are the holy grail, i.e. Y=\>X instead of X=\>Y

According to me, a DevOps mindset is that one works together and that one focuses on deliverables, i.e. new software live within two weeks. If one wants to create all microservices in advance then a lot of assumptions have to be done and nothing will be put to production, i.e. available to the customer.

This happened in one of my projects. The architects was continuously talking about "what if this would happen and this" and so on. Outcome? After one year, nothing was running in production.

In my opinion, some companies start with microservices because they have heard about it, but do not know exactly what microservices are. Microservices are not the holy grail, e.g. one could create nanoservices, one should master automation.

My point of view about microservices is that one should start with a monolith first, e.g. if you want a webshop just start with it and ensures that it is running in production within two weeks. Secondly, start monitoring it, check if there are problems in certain areas, e.g. more bugs in component X or component Y is extremely slow. If it is clear that component Y is slow, create a plan of extracting it, rewrite it within two weeks, apply autoscaling.

I am in favor of the "cheese slicer method". As the cheese itself is a product that delivers value and is running in production and the X=\>Y approach is used. Start with a problem and check what could solve this problem, instead of thinking about a solution, e.g. microservices (Y) that could solve a (hypothetical) problem.

For your information. I am currently busy with a personal microservices project that consists of two microservices, e.g. an api and a write microservice. I am first solving the impediments and cleaning up my backlog. Off course I have some ideas about next microservices, but I want to put first things first and add value to the customers instead of doing a lot without finishing this microservice project that I have already experienced a couple of times.

---

Note: once the current backlog is complete, an additional paragraph about

> If Microservices terminology i.e. distinguishing between `core` and `other service instances`

will follow.

April 26 2018

At the moment I have a monorepo that consists of 5 microservices, i.e. an API, writer, selector, insert and tableCreator. On one hand it is nice as the shared code could be used directly, but when a code change is pushed then I need to build everything.

The next steps is to create five repos and extract the code from the monorepo to each individual repo. Advantages of this is that a code change to a repo builds the microservice that has been changed, smaller code base (easier to read).

The common code will be moved to separate repos as well and published as artifacts so they could be used by the microservices.

One of the prerequisites is that CI and CD are enabled for each repo. Otherwise it will take a lot of time before everything has been deployed.

Microservices are not the holy grail

Automation is key

"Do one thing and do it well".
> 1 votes

---
Tags: microservices, architecture
---


# Question
Title: How to avoid code duplication in bitbucket-pipelines?
The following bitbucket-pipelines file has been created:

```
options:
  docker: true

pipelines:
  default:
    - step:
        script:
          - echo "Login to the docker registry"
          - docker login some-docker-registry
    - step:
        script:
          - echo "Build some-image"
          - docker build some-docker-registry/some-image .
          - docker push some-docker-registry/some-image
    - step:
        script:
          - echo "Build some-image2"
          - docker build some-docker-registry/some-image2 .
          - docker push some-docker-registry/some-image2
        deployment: production

```

When it is run by bitbucket, it indicates that the authentication to the docker registry fails so now this step has to be added over and over again:

```
options:
  docker: true

pipelines:
  default:
    - step:
        script:
          - echo "Login to the docker registry"
          - docker login some-docker-registry
          - echo "Build some-image"
          - docker build some-docker-registry/some-image .
          - docker push some-docker-registry/some-image
    - step:
        script:
          - echo "Login to the docker registry"
          - docker login some-docker-registry
          - echo "Build some-image2"
          - docker build some-docker-registry/some-image2 .
          - docker push some-docker-registry/some-image2
        deployment: production

```

Is it possible to prevent such duplication? An attempt has been made (see first code snippet), but that does not seem to work, i.e. a docker login is required by every step.

# Answer
> 1 votes
Unlike the declarative pipelines in Jenkins, bitbucket does not seem to have such functionality. In order to prevent code duplication a script was created and the current bitbucket-pipelines looks as follows:

```
options:
  docker: true

pipelines:
  default:
    - step:
        script:
          - ./build-script.sh API true true
    - step:
        script:
          - ./build-script.sh Write true true
        deployment: production

```

---
Tags: bitbucket-pipelines
---


# Question
Title: Is it a common practice to include the version of an API into the rest interface itself?
I was looking for the current version of the github API, but it seems to be omitted.

https://api.github.com/

I am personally considering to add a new field to my own API, but I wonder whether that is a common practice or should be done differently.

# Answer
Many APIs include a version number in the url of the service. Some examples:

However many people on the Internet argue that this is incorrect because it violates REST/HTTP specifications; "embedding of API version into the URI would disrupt the concept of hypermedia as the engine of application state". It’s more appropriate to use the Accept header to request a particular version. For the latter people suggest to use a Vendor MIME type, for example something like `Accept: application/vnd.mycompany-api-v3+json`

Nevertheless, the reason why many API designers do include an API version in the url is because it's easy to implement and debug (clearly visible which version you are using).

*Side note:* in our organization we recently decided to support both a versionless endpoint that always point to the latest stable version of our APIs, as well as an endpoint with a version specification. This allows users to choose if they want to commit to our latest API (which may require making changes when a newer version is released), or to connect to a fixed API version (which may become deprecated over time).
> 9 votes

---
Tags: api
---