# Question
Title: How to remove directories and files in another layer using Docker?
**Why this question?**

The reason for posting this Q&A is that sometimes certain software is required to compile software in a docker image. Once compiled, these packages are superfluous and should be removed in order to reduce the image size. In some cases images that were more than 1.5GB where reduced to less than 300MB. The following use case does not represent a real world scenario, but provides an example of this use case.

**Attempts to solve the issue**

It works to remove a directory when the remove command is defined in the same layer as the creation of the folder:

```
FROM alpine
RUN mkdir dir && cd dir && wget http://google.com && rm -rf dir

```

This results in:

```
user@host$ docker build -t dir .
Sending build context to Docker daemon 1.336 MB
Step 1/2 : FROM alpine
 ---> 4a415e366388
Step 2/2 : RUN mkdir dir && cd dir && wget http://google.com && rm -rf dir
 ---> Using cache
 ---> c283805e687f
Successfully built c283805e687f

```

When the aim is to remove the directory in another layer, this fails:

```
FROM alpine
RUN mkdir dir && cd dir && wget http://google.com
RUN rm -r dir

```

And this fails:

```
FROM alpine
RUN mkdir dir && cd dir && wget http://google.com
RUN rm -rf dir

```

Another attempt was using workdir:

```
FROM alpine

RUN mkdir dir
WORKDIR dir
RUN wget http://google.com
WORKDIR /
RUN ls
RUN rm -r dir

```

Building the image resulted in:

```
Sending build context to Docker daemon 2.048 kB
Step 1/7 : FROM alpine
 ---> 4a415e366388
Step 2/7 : RUN mkdir dir
 ---> Using cache
 ---> aed5c75218cb
Step 3/7 : WORKDIR dir
 ---> Using cache
 ---> 715de09e7e08
Step 4/7 : RUN wget http://google.com
 ---> Using cache
 ---> c07bfb3e1133
Step 5/7 : WORKDIR /
 ---> Using cache
 ---> c6de86d0191a
Step 6/7 : RUN ls
 ---> Using cache
 ---> 9567593cce39
Step 7/7 : RUN rm -r dir
 ---> Running in 7fe7b28294bf
rm: can't remove 'dir': Directory not empty
The command '/bin/sh -c rm -r dir' returned a non-zero code: 1

```

# Answer
> 1 votes
A solution is to remove the `not empty directory` using `find`:

**Dockerfile**

```
FROM alpine

RUN mkdir dir && cd dir && wget http://google.com && cd / && echo -e "BEFORE\n" && ls && find /dir -delete && echo -e "\nAFTER\n" && ls

```

**Outcome**

```
Sending build context to Docker daemon  2.048kB
Step 1/2 : FROM alpine
 ---> 4a415e366388
Step 2/2 : RUN mkdir dir && cd dir && wget http://google.com && cd / && echo -e "BEFORE\n" && ls && find /dir -delete && echo -e "\nAFTER\n" && ls
 ---> Running in ba93a1742a76
Connecting to google.com (172.217.20.110:80)
Connecting to www.google.nl (172.217.20.99:80)
index.html           100% |*******************************| 11092   0:00:00 ETA

BEFORE

bin
dev
dir
etc
home
lib
media
mnt
proc
root
run
sbin
srv
sys
tmp
usr
var

AFTER

bin
dev
etc
home
lib
media
mnt
proc
root
run
sbin
srv
sys
tmp
usr
var
 ---> 2e3109285a2d
Removing intermediate container ba93a1742a76
Successfully built 2e3109285a2d

```

# Answer
> 19 votes
`mkdir dir && cd dir && wget http://google.com && rm -rf dir` doesn't do what you think it does.

Let's break it down:

1. `mkdir dir` creates `dir`.
2. `cd dir` changes directory into it.
3. `wget http://google.com` downloads google.com inside `dir/`.
4. `rm -rf dir` attempts to delete a directory called `dir` that's in the current directory.

The problem is that there is not `dir` in the current directory, because you're already `cd`ed into it. And since you've added the `-f` flag, the command doesn't produce an error. This is one of the reasons why you shouldn't use force flags without being very sure you need them.

Since you didn't use the `WORKDIR` command, the directory change is discarded at the end of the `RUN`. Thus, the next line executes in the original working directory, where it tries to `mkdir dir` but, as we've previously discussed, that directory still exists with the wgetted contents inside it.

# Answer
> 7 votes
Recent Dockerfile spec allows for multiple FROM statements.

https://docs.docker.com/engine/userguide/eng-image/multistage-build/

Using this approach, you can retain your compilation results to a pure runtime environment.

# Answer
> 6 votes
Building on Xiong Chiamiov's answer, which correctly identified the root cause of the problem - the `dir` reference by relative path when attempting to empty or delete that directory depends on the working directory at the time, which was not correctly set in the cases mentioned in the OP.

So there are 2 solutions available:

* set the proper working dir prior to executing the `dir` removal:

```
  RUN mkdir dir && cd dir && wget http://google.com && cd .. && rm -rf dir

```
* use a `/dir` full path reference to that directory instead. But with care (adjust if needed) if used in cases just similar, but not identical to this one, since this approach assumes the initial working dir was `/`:

```
  RUN mkdir dir && cd dir && wget http://google.com && rm -rf /dir

```

---
Tags: dockerfile
---


# Question
Title: Had to point localhost to remote ip for docker-compose to work. why?
I *had* my docker env variables set like this:

```
set -gx DOCKER_TLS_VERIFY "1";
set -gx DOCKER_HOST "tcp://xxx.xxx.xxx.xxx:2376";
set -gx DOCKER_CERT_PATH "/home/me/.docker/machine/machines/myhost";
set -gx DOCKER_MACHINE_NAME "myhost";

```

When I tried running `docker-compose up`, I got this error:

```
backports.ssl_match_hostname.CertificateError: hostname 'xxx.xxx.xxx.xxx' doesn't match 'localhost'

```

But then I happened upon this blog post that recommended that I set my local dns to point `localhost` to `xxx.xxx.xxx.xxx`. And it worked.

Why did this solution work? And more importantly, what can I do to not use it again?

# Answer
It looks like you are using instead of .

https://docs.docker.com/machine/get-started/#create-a-machine

> Get the environment commands for your new VM.
>
> As noted in the output of the docker-machine create command, you need to tell Docker to talk to the new machine. You can do this with the docker-machine env command.
>
> ```
>  $ docker-machine env default
>  export DOCKER_TLS_VERIFY="1"
>  export DOCKER_HOST="tcp://172.16.62.130:2376"
>  export DOCKER_CERT_PATH="/Users/<yourusername>/.docker/machine/machines/default"
>  export DOCKER_MACHINE_NAME="default"
>  # Run this command to configure your shell:
>  # eval "$(docker-machine env default)"
>
> ```

Why did this solution work?

According to this documentation the variables have to be set over and over again.

> Unset environment variables in the current shell
>
> You might want to use the current shell to connect to a different Docker Engine. This would be the case if, for example, you are running Docker for Mac concurrent with Docker Toolbox and want to talk to two different Docker Engines, or running swarms on Docker Cloud and want to switch between managing the swarm and using Docker hosts. In both scenarios, you have the option to switch the environment for the current shell to talk to different Docker engines.
>
> ```
> Run env|grep DOCKER to check whether DOCKER environment variables are set.
>
> $ env | grep DOCKER
> DOCKER_HOST=tcp://192.168.99.100:2376
> DOCKER_MACHINE_NAME=default
> DOCKER_TLS_VERIFY=1
> DOCKER_CERT_PATH=/Users/victoriabialas/.docker/machine/machines/default
>
> If it returns output (as shown in the example), you can unset the DOCKER environment variables.
>
> Use one of two methods to unset DOCKER environment variables in the current shell.
>
>     Run the unset command on the following DOCKER environment variables.
>
>     unset DOCKER_TLS_VERIFY
>     unset DOCKER_CERT_PATH
>     unset DOCKER_MACHINE_NAME
>     unset DOCKER_HOST
>
>     Alternatively, run a shortcut command docker-machine env -u to show the command you need to run to unset all DOCKER variables:
>
>     $ docker-machine env -u
>     unset DOCKER_TLS_VERIFY
>     unset DOCKER_HOST
>     unset DOCKER_CERT_PATH
>     unset DOCKER_MACHINE_NAME
>     # Run this command to configure your shell:
>     # eval $(docker-machine env -u)
>
>     Run eval $(docker-machine env -u) to unset all DOCKER variables in the current shell.
>
> Now, after running either of the above commands, this command should return no output.
>
>  $ env | grep DOCKER
>
> If you are running Docker for Mac, you can run Docker commands to talk to the Docker Engine installed with that app.
>
> If you are running swarms on Docker Cloud, you can re-run the export command you used to connect to the swarm.
>
> Since Docker for Windows is incompatible with Toolbox, this scenario isn’t applicable because Docker for Windows uses the Docker
>
> ```
>
> Engine and Docker Machine that come with it.
> 1 votes

---
Tags: docker-machine
---


# Question
Title: Can you replicate Active Directory from a Corporate Intranet to an AWS VPC where there is an outbound-only link?
In our work environment we have a standard Corporate Intranet with Active Directory.

We've been granted limited access to an AWS VPC. Our connection allows outbound (from the Intranet to the VPC) but not inbound.

That is - if we run a webserver in the AWS VPC, then a client in the Corporate Intranet can connect and browse to it. But a client in the AWS VPC cannot connect to a webserver in the Corporate intranet. Note that this 'outbound connections only principle' applies to all ports, not just http port 80.

An associate has suggested we need to replicate our Active Directory down to the AWS VPC AD. I think it is not possible to do a one-way replication.

My question is: **Can you replicate Active Directory from a Corporate Intranet to an AWS VPC where there is an outbound-only link?**

# Answer
Yes, you just need a Read Only Domaine Controler (search for RODC) in the vpc and you'll be able to have a one way replication as the read only controler won't need to send datas back.

This bring limitation for clients in the vpc, as they won't be able to reach a writable DC they can't change passwords and you'll need a careful site architecture in your forest.
> 1 votes

---
Tags: amazon-web-services, aws-vpc, active-directory
---


# Question
Title: What are best practices to implement Gatling in CI?
Gatling creates an HTML report:

```
Reports generated in 0s.
Please open the following file: /path/to/target/gatling/recordedsimulation-1496161958749/index.html
[info] Simulation RecordedSimulation successful.
[info] Simulation(s) execution ended.
[success] Total time: 31 s, completed May 30, 2017 6:32:57 PM

```

What are best practices to read the output and check whether it is acceptable, i.e. if a certain value is larger and smaller than X? Should one create a custom script or does Gatling itself provide such a functionality?

# Answer
https://gatling.io/docs/2.3/extensions/jenkins\_plugin/

https://wiki.jenkins.io/display/JENKINS/Gatling+Plugin

> Configuration
>
> ```
> Install Gatling Plugin (via Manage Jenkins -> Manage Plugins)
> Configure your project to execute Gatling simulations, for example using the Maven plugin (see Maven plugin documentation)
> Configure your job :
>     For a maven job: add "Track a Gatling load simulation" as a new post-build action. Beware that neither your Jenkin's path nor your
>
> ```
>
> job's name should contain any space. For a pipeline job: add a line to your pipeline script: gatlingArchive()
>
> Optional configuration: You can publish Gatling results with the Jenkins JUnit plugin. Your Gatling simulation needs to have some assertions, as each Junit testcase is a different Gatling assertion.
>
> ```
> Configure the Jenkins plugin with the following Test report XMLs: target/gatling/assertions-*.xml
> Execute Gatling with the following option: -Dgatling.useOldJenkinsJUnitSupport=true, for example: mvn gatling:execute -Dgatling.useOldJenkinsJUnitSupport=true
>
> ```
> 1 votes

---
Tags: continuous-integration, gatling
---


# Question
Title: Rename VM in VMware Fusion via command line
Is it possible to rename a VM in VMWare Fusion via command line? (ie not clone and then name the new VM and delete the old VM).

For example, Parallels has the `prlctl` tool, and the process would be something like:

```
prlctl stop oldname
prlctl set oldname --name newname
prlctl start newname

```

# Answer
Well, there is no rename, it is a multi-step process, which involves renaming a bunch of entries inside the vmx file, using vmware-vdiskmanager to rename to vmdk and in general doing a bunch of potentially problematic actions if you don't know what you are doing.

You could probably write a quick Perl script to do all that using this guide.

Another option would be to clone it and delete the original, but it is resource intensive and does way too much work for what it is worth. The code below is untested:

```
vmrun -T fusion stop "/Volumes/vms/old/old.vmx"
vmrun -T fusion clone "/Volumes/vms/old/old.vmx" "/Volumes/vms/new/new.vmx" full --cloneName=new
vmrun -T fusion deleteVM "/Volumes/vms/old/old.vmx"
vmrun -T fusion start "/Volumes/vms/new/new.vmx"

```
> 3 votes

---
Tags: vmware, vmware-fusion
---


title: Use Vault SSH OTP backend with Jenkins

text:
# Question
Title: Use Vault SSH OTP backend with Jenkins
I'm setting up Vault to store secrets. SSH backend with OTP auth looks like a good way to stop managing SSH keys for access.

I have SSH backend working, and am looking at hooking up Jenkins with the SSH-OTP auth so that I can remove the deployment keys too.

I looked at Jenkins plugins for vault, but they only work for fetching secrets from Vault.

Are there plugins or any Jenkins hacks for this purpose, or if not, is it possible to write one of my own?

**Note**: Right now, Jenkins uses SSH keys. I want to eliminate the use of keys, and configure Jenkins to get an SSH OTP from Vault every time it needs to SSH into some host for a deploy.

# Answer
> I looked at Jenkins plugins for vault, but they only work for fetching secrets from Vault.

It depends how the current configuration looks like that is used to deploy apps.

https://github.com/jenkinsci/hashicorp-vault-plugin

If one uses Jenkins pipelines, then one could replace the keyId with the one that is defined in Hashicorp vault.

> ```
> node {
>   // define the secrets and the env variables
>   def secrets = [
>       [$class: 'VaultSecret', path: 'secret/testing', secretValues: [
>           [$class: 'VaultSecretValue', envVar: 'testing', vaultKey: 'value_one'],
>           [$class: 'VaultSecretValue', envVar: 'testing_again', vaultKey: 'value_two']]],
>       [$class: 'VaultSecret', path: 'secret/another_test', secretValues: [
>           [$class: 'VaultSecretValue', envVar: 'another_test', vaultKey: 'value']]]
>   ]
>
>   // optional configuration, if you do not provide this the next higher configuration
>   // (e.g. folder or global) will be used
>   def configuration = [$class: 'VaultConfiguration',
>                        vaultUrl: 'http://my-very-other-vault-url.com',
>                        vaultCredentialId: 'my-vault-cred-id']
>
>   // inside this block your credentials will be available as env variables
>   wrap([$class: 'VaultBuildWrapper', configuration: configuration, vaultSecrets: secrets]) {
>       sh 'echo $testing'
>       sh 'echo $testing_again'
>       sh 'echo $another_test'
>   }
> }
>
> ```
> 1 votes

---
Tags: jenkins, hashicorp-vault
---


# Question
Title: Reuse of docker swarm containers for batch jobs
Docker compose allows for static stack description. But what if I have a set of containers based on same image but different by subtle environment configuration? Moreover, I don't need them all at once, they can be launched and disposed of sequentially.

A quick shot would be to copy-paste the service, but a couple of dozens of them is a waste of resources.

Is there sort of elegant orchestration loop in this case?

UPD After some more googling, interesting fact came out that apparently what I do is batch job processing which has currently no conceptual (!) support in Docker compose because services are not batch jobs. While there is a proposed feature for that I will try out one of the community workarounds, Jobs As A Service (JaaS) but my question is still - maybe there is an elegant way to do the following:

```
version: '3.1'

services:

  task1:
    image: myjob
    environment:
      PARAM: foo

  task2:
    image: myjob
    environment:
      PARAM: bar

```

You may have asked yourself "why not just take a series of `docker run/exec/rm`", but the rest of the environment like database runs in the stack.

# Answer
> 2 votes
Correct me if I am wrong, but that is one of the reasons why docker orchestration tools exists like docker-swarm to be able to deploy multiple docker containers by using one command line.

> Create a service with 5 replica tasks (–replicas)
>
> Use the --replicas flag to set the number of replica tasks for a replicated service. The following command creates a redis service with 5 replica tasks:
>
> ```
> $ docker service create --name redis --replicas=5 redis:3.0.6
>
> 4cdgfyky7ozwh3htjfw0d12qv
>
> ```

If for example 1000 identical containers that use a different environment variable have to be deployed, multiple `docker service create` could be added to a script, instead of copy-paste multiple times in one docker-compose file.

---
Tags: docker
---


# Question
Title: Google Cloud Storage vs Redis / ElasticSearch
We're building an application that reads data from an API and displays it in a frontend. Due to API limits, we have to cache the API calls meaning that we save the result of the API calls (both user information (first name, country, email etc.) and big sets of activity log data) in a database and only request new data from the API if the stored data is older than x hours.

This application should also work for multiple users that can log in and see their relevant data. For each user, the application connects to a different API endpoint (e.g. "user1.api.com/data/whatever" and "user2.api.com/data/whatever").

So, in the end there will be two databases. One for caching the API calls of an external tool and another one for the application's user management. However, we are not sure what would be best to use.

Is it generally a good idea to use cloud databases for either caching or user management? As the application will run on Google Container Engine, we could probably use Google Cloud Storage (for caching) and Firebase (for user management). Or should we rather use something like Redis / ElasticSearch (for caching) and something different for the different user data?

# Answer
I don't know much about Redis/ElasticSearch, but GCS is not really a database-like solution, it is closer to a file storage solution.

If you're looking for database-like storage Google Cloud offers:

If you expect a modest app traffic or alternating high/low traffic it *might* be more cost-effective (and simpler) to have a Google App Engine app instead of a GCE-based solution.
> 4 votes

# Answer
We had the similar requirement as there needs to be a set of data cached in a server for serving the application (fast processing) and also by end of the day, we would need to sync the data from the cache server to the origin server.

In the longer run, I would suggest to go for Redis as you can have Redis as an intermediate database (caching) and it will help you in scaling.

You can write scripts to run at an interval or at specific time of the day so that you can pull/push the data from the origin server (external application).

Your requirement wouldn't require Elastic Search at any point of time as it will help you in speeding up search results based on caching, it is just a search engine.
> 2 votes

---
Tags: databases, google-cloud-platform, google-cloud-storage
---


# Question
Title: Continuous Integration and Isolated Feature Testing
I've been given the task at work to move everyone from using TFS Source Control to Git as the Development Team has been lead to believe that Git may well be the silver bullet that will fix alot of the teams issues. I carried out the migration a month back and fortunately Git has fixed one thing for the team - code reviews are now alot more structured as pull requests in Git give the developer and reviewers a auditible platform to merge code into the master branch. There is however an issue of merge conflicts still, some of which taking up to 2 or 3 hours to resolve. The team are questioning if the tool (Git) is to blame whereas under analysis, I believe there are many other fundamental processes and pratices at fault within the team. To name but a few that don't sit well with me:

* All products (Visual Studio Projects) maintained by the company sit under one Visual Studio Solution. This is around 40+ VS Projects ranging from Class Librarys, Services and Database tiers. If an update needs to be made to Production for one product, the whole Solution (along with the other products) needs to be released.
* There a 4 Scrum Teams working on the same VS Solution looking after their own Products. The Sprints all start on different days in a two-week cycle and therefore at any point in time, each team's Sprint is at a different state. Due to the issue raised in the first bullet point however, complexity ensues when it comes to the **monthly** release to Production as code for each product can be at separate stages.

Anyway, that paints part of the scene to help pose my question. We're now using Git using the following structure:

* Feature branch - Used for each User Story being worked upon in a Sprint
* Develop branch - All Feature branches are merged into here once the Pull Request has been approved and the change **tested in isolation** on a test server
* Master branch - Successfully tested features are cherry-picked into the Master branch

My question is, how does DevOps / CI handle testing newly coded features in isolation? I've never worked in a place where we've tested new changes in total isolation therefore I can't get my head around why it's so important to my new Dev Team. In my opinion, the sooner we can test new changes with the rest of the codebase, the better.

# Answer
First of all, git is certainly not to blame for merges taking a long time. Clean merging is *the* hallmark of git, unless it is used in some broken way. So I encourage you to look at what exactly seems to be the underlying problem. If your devs regularly make sweeping changes to the same files, then it's likely more a problem of cohesion/coupling than of the underlying merge tool.

Second; I like to do isolated feature-testing, if by that you mean that your test suite is ran against the feature branch (with only the branch changes) in a clean environment. In fact, I like to test *all* branches in this way. How could the `develop` or `master` branch become green if one of the features going into it are not? It goes without saying that `develop` / `master` need to be tested as well.

You could even go so far as to do a pre-merge test by doing a merge of `feature` into `temp_develop_with_feature` (or whatever you may call it), that is, simulate the merging of a branch into `develop` (under a different name) and test *that*.

Obviously, all of this assumes that sufficient testing ressources (VMs etc.) are available and fast enough, and that everything is 100% automated, both of which should be rather easy to achieve with something like Jenkins, Gitlab or whatever tool you use (if not, then it's maybe time to look at another one...).
> 6 votes

# Answer
> There is however an issue of merge conflicts still, some of which taking up to 2 or 3 hours to resolve. The team are questioning if the tool (Git) is to blame whereas under analysis.

**What is causing the merge hassle?**

* Why does it take up to two or three hours to resolve merge conflicts?
* What workflow do the developers use?

In summary, what is the cause of this merging conflict hassle?

**Is git causing the issue?**

This has nothing to do with git, but looks like a workflow issue.

My favorite flow is the GitHub flow. When a feature has to be created, a new branch will be created from the master, when development has been completed, a PR will be created, when the review is completed the branch could be merged into master.

There are other flows, but what I do not like about gitflow is that I have seen in multiple companies that developers were applying a kind of "through it over the wall" approach as they merge multiple features to the develop branch without releasing. When after a couple of months software has to be released from the master then a lot of issues could occur. I personally like the following quote from this answer.

> don't use git-flow. It looks nice in that diagram because of the lovely coloured dots and nicely laid out lines, but in reality it looks like an insane messy web of coloured lines and dots.

**Release software more often**

When the github flow will be applied then less merge conflicts will occur as when one creates a new feature then one has to create a branch from the master. As features are merged into the master, there is a bigger change that the commits in the feature branch will be behind master.

**Discipline**

Although workflows could be applied, the most important thing is discipline. Developers should ensure that if they create a Pull Request that their commits that reside in a feature branch are ahead of the master. Other developers should enforce that features will not be too big as these will be harder to merge and are harder to review.
> 4 votes

---
Tags: continuous-integration, git, testing
---


# Question
Title: how is software defined infrastructure related to devops?
I'm a new developer who is picking up python. I'm currently writing code using python's asyncio or twisted framework to automate the setup of some raspberry pi and servers I got 2nd hand (xen, hyper-v) using SSH, I tried ansible but I prefer writing my own code for learning.

does this fit into software defined infrastructure? if I setup automate deployment of code, is this considered devops?

additionally, I hope to enquire if my intended workflow is considered "devops"?

* gitlabs
* automated code testing (tox/travis ci)
* automated deployment of code to raspberry pi

# Answer
> 6 votes
> if I setup automate deployment of code, is this considered devops?

Sure, automating everything is a big part of DevOps.

> I tried ansible but I prefer writing my own code for learning

Now, this is laudable, but using Ansible (or Puppet or Chef or ...) would be "learning" as well. The benefit of using established tools over DIY is that if/when you meet other developers, you have a common toolset to work with. Doing it yourself is nice sometimes as well, but basing a whole software development process on 100% self-written tools (from scratch) is not a good idea, unless you have deeply evaluated the existing tools and consider them all unfit for the purpose.

When your team grows and you encounter a new developer - is it more likely that they already know about Ansible, or that they know about the code you wrote?

Also, the fact that the tools mentioned above do not do their stuff in the "code" directly, but that they have text files which configure them, is important as well. It abstracts the configuration stuff away from actual code, which *will* be hard to maintain a few years down the road, no matter how good you are.

---
Tags: configuration-management
---


# Question
Title: How to send sms to a phone number using aws sns cli?
I am using the following command. Getting Authorization error.

```
aws sns publish --phone-number +91XXXXXXXXXX --message hello

An error occurred (AuthorizationError) when calling the Publish operation: User: arn:aws:XXX::XXXXXXXXXXXX:user/ABC-DEF-SNS is not authorized to perform: SNS:Publish on resource: +91XXXXXXXXXX

```

Other related sns commands are working. I can also able to send sms through a topic to a subscribed number.

# Answer
> 1 votes
Most probably your user `arn:aws:XXX::XXXXXXXXXXXX:user/ABC-DEF-SNS` does not have the required permissions to the `sns:Publish` API.

You can check using the IAM Policy Simulator at https://policysim.aws.amazon.com/

using policy simulator is explained in depth here https://docs.aws.amazon.com/IAM/latest/UserGuide/access\_policies\_testing-policies.html

---
Tags: aws-cli
---


title: Custom Layer 4 Load Balancer

text:
# Question
Title: Custom Layer 4 Load Balancer
What are my best options for deploying a load balancer on AWS with the following requirments:

Add auto scaling that will start my own EC2 instances. Deploy 3rd party SW on the load balancer. Have a readable shared memory (can be S3).

Found the tutorial "Create an External Load Balancer" about Kubernetes but not sure about it.

EDIT:

deploy 3rd party \[software\] on the load balancer" -- to do what: Want to start new instances with parameters and for routing rules to the instances that were created with the auto scaling.

how is "readable shared memory" used, and for what purpose? shared memory between the load balancer and the instances. I can do it with sockets.

Why is the balancer layer 4 as opposed to a different layer? Need TCP parametrs, layer 7 info can also be useful

Where is TLS handled? I havn't thought about this.

# Answer
Historically, F5's Application Delivery Controller has been the enterprise-grade industry solution for doing this. With this solution you can:
> 2 votes

---
Tags: amazon-web-services, autoscaling, load-balance
---


title: Networking between Docker containers

text:
# Question
Title: Networking between Docker containers
I have a docker container 'A' on a host 10.11.12.13 listening on port 0.0.0.0:8443-\>8443/tcp I have another container 'B' on same host how wants to talk to 'A' on 8443 via host. Thus when I run command curl https://10.11.12.13:8443 inside B it gives me error as 'No route to host' but I can ping 10.11.12.13 successfully. I am not sure what else I am missing?

Can someone help regarding the issue?

# Answer
Docker actually provides DNS support for networking between containers. This means that if you define your database connection as:

```
http://redis:6379

```

This will resolve `redis` to the correct network address, given that you fulfill the following:

* All containers are on the same network
* The containers have been named

This can either be done manually by starting both instances via the cli or automated with a docker compose script, which is *my recommendation*. In the compose script, from version 3, you would define the both services (aka the containers):

```
version: "3"
services:
  web:
    build: ./web
    container_name: web
    networks:
      - backend
    command: bash -c "./start.sh"

  redis:
    image: redis:3.2
    container_name: redis
    networks:
      - backend

```

And at the end of the file, define the network both instances should be connected to:

```
networks:
  backend:

```
> 4 votes

---
Tags: docker, containers, networking
---


# Question
Title: What should I use for configuration management on AWS?
I am trying to find a solution for configuration management using AWS OpsWorks. What I can see is AWS offers three services for OpsWorks

1. Chef Automate
2. Puppet
3. AWS stacks

I have read basics of all three of them but unable to compare between three of them. I am unable to understand when to use which solution.

I want to implemnet a solution for my multiple EC2 instances, using which I can deliver updates to all my instances from a central repository(github). And, rollback changes if needed.

So following are my queries:

1. Which of the three solutions is best for this use case?
2. What should I use if my instances are in different regions?

I am unable to find anything useful on these topics so that I can make my decision. It would be great if I can get links to some useful articles as well.

Thanks in advance.

# Answer
> 6 votes
A good way to find comparison information about things like this is googling for "X vs Y", e.g. "Chef vs AWS stacks", "Chef vs Puppet" or something like that. That *does* turn out subjective information, and while it is nothing like having hands-on experience, you still get a few nice nuggets here or there.

For example, Chef gives you the full Ruby language while Puppet has a ruby-based DSL. Does that matter for you? Only you can tell.

From my experience with Puppet/Ansible (and I don't know Chef yet) it's all mostly down to taste. For example, I really like the fact that Ansible needs only a working ssh key and nothing else. And I like that there is no central repository. Others may like the fact that Puppet has a repository. Also, I like the dead simple configuration file format. But that is all subjective.

So, to cut to the chase, my advice would be:

* Use AWS Stacks if you are fully committed to AWS, will never change to another provider, and like the higher integration with AWS features. It uses Chef internally anyways, and exposes the Chef files to you.
* Use Chef directly if you like to learn something new and want to stay a bit vendor independent. If you later decide to commit to AWS fully, then you should be able to do so.
* Use Puppet (which AWS starts to support as well, seemingly) if you, well, like Puppet more, or if you can't stand Chef.

# Answer
> 3 votes
One consideration might be lock-in. For example, Amazon's cloud can become expensive very quickly - either because compute ends up being more than you estimated or there is always the possibility that Amazon decides to raise the cost of their compute price. This could cause you to want to change cloud providers to a cheaper one or perhaps you scale to a point at which it makes more financial sense to insource.

Another scenario could be that Amazon's cloud somehow becomes breached. For example, the Spectre and Meltdown exploits are particularly nasty on hypervisors and offer some possibility and prospect of breaking out of a guest into the hypervisor or gaining sensitive information from another tenant on the hypervisor.

These hypothetical and potential scenarios might or might not become realized and may or may not be realistic but nevertheless by choosing something that is easily exported to another cloud allows you to remain agile and easily deploy into another cloud or to insource. This guards against these above scenarios or any others that could arise.

There is nothing more important in this regard than your configuration management system - especially if you are using an \[immutable architecture\] and/or a cattle-not-pets philosophy as this allows you to simply begin spinning up new compute instances in the new cloud or cluster immediately. You needn't migrate virtual machines. (and hopefully you don't have a large amount of data to export).

So that is to say: Perhaps it would be wise to avoid AWS stacks so you aren't locked into this particular vendor.

---
Tags: amazon-web-services, amazon-ec2, configuration-management, continuous-deployment
---


# Question
Title: Is it better to perform delayed jobs using queues or at app level?
I'm trying to scale up my push notification infrastructure. We currently use laravel app running on 2 ec2 instances horizontally scaled behind a load balancer, and each instance has a beanstalk queue engine that processes jobs created by laravel (including push notification jobs).

the problem is that this solution is not performant. Sometimes we send a marketing push to all our clients.. numbering in the thousands, and the time range difference between the first push notification and the last can be many hours.

I was considering what infrastructure would be good options. I considered AWS SQS, but it turns out that it's max delay job capacity is 15 minutes. In my business, customers can make pre-orders 20 hours before. So what we currently do is scheduled a delayed job 20 hours from now.

But then some people suggested that the delayed part should only happen at the application level (ie `delay(work, timedelta(hours=20))`), and that would allow me to use SQS anyways. Their reasoning is that delaying jobs is a business logic thing, and it's best handled at an application level and not an infrastructure level.

I'm not sure I agree.. and that's why I'm picking AWS elastic cache with redis. What do you guys think?

# Answer
> 2 votes
I would recommend using a scheduler to schedule job execution, rather than building logic to delay execution. Schedulers give you the added advantage of status updates. Consider using Simple Workflow Service from AWS.

Instead of using beanstalk queue engine, could you use Lambda to process your marketing jobs? Simple Workflow Service works with Lambda. You don't have to worry about scaling with Lambda.

# Answer
> 2 votes
This sounds on the fence with opinion based but I think there's an important thing that narrow enough the problem to be answered.

It is still an opinion but, as far as you're horizontally scaling, instances may appear and disappear at will, keeping the delay in memory is thus not an option, so at this point you'll have to workaround this and bake something like a notifications queue/table to keep what is to be sent at which time and periodically check what is to be sent.

Once that is said, you'll need a central place somewhere to keep the needed notifications out of the app instance themselves and as such you'll need a place to store them, it could be a redis service, a database table or whatever, you'll be coding it anyway.

When working on AWS, the services provided are already redundant and on high availability without much effort needed on your part, as such I'd go with an application code storing the notification somewhere with the time to send it and a lambda checking periodically which notification has to be sent, the notification can then be sent through SQS to take advantage of the retries in case of delivery failure or you'll have to handle it in the notification firing system.

The store part being elastic cache or something else is a matter of taste and team knowledge in my opinion, take whatever you're more comfortable with.
For the part firing the notifications I'd use something separated from the app itself, another app or a lambda checking periodically the 'cache' (DB or redis or whatever) for notifications to be sent.

This allows the app to be sure the notification once declared and acknowledged by the cache is properly registered and will be fired when needed and retried in case of problem. A last case is the impossibility to deliver the notification and you may have to keep track of this in the application to avoid sending too much notifications ending as dead letter but that's sliding out of scope for this question.

---
Tags: amazon-sqs
---


title: DynamoDB vs RDS for a counter store?

text:
# Question
Title: DynamoDB vs RDS for a counter store?
I have multiple lambda functions, which a file passes through for processing, and I want to store the status of each computation in a counter, for better logging and monitoring.

Current plan is to log it like this:

```
file-name  Lambda1  Lambda2 Lambda3
Name1      Done     Done    In-process

```

And, I'll be updating the values as `Received`, `In-process` and `Done` under the respective Lambda functions for the file.

So, two doubts:

1. **Is this a good practice?** If not, then is there a better alternative for the same?
2. **If it is, then what would make a better/cheaper store? RDS or DynamoDB?**

# Answer
> 4 votes
Why not something like Redis? (can use it through ElastiCache). Hard to answer this without knowing the operational requirements of this project. Is this data strictly for logging/research, or is it operational (i.e. this "table" is consulted in order to know which lambda to invoke).

Dynamo vs. RDS - depends how spiky your traffic is. Dynamo is very scalable and will give you a guaranteed throughput. But if you traffic can grow 10x over hours (spiky) Dynamo might be too expensive, since you'll need the maximum throughput provisioned or else you'll get throttled.

RDS is basically managed RDBMS on an instance type that you choose, so it can handle larger spikes, but ultimately not as scalable as Dynamo.

Again, I would consider Redis, for it's simplicity (seems like you just need a KV/counter store), and incredible speed/efficiency. If it's just for logging, it's probably good enough.

# Answer
> 2 votes
For question #1)

I think this could perhaps be better answered on SoftwareEngineering SE.

Nonetheless, I'll risk an answer: for this kind of information (states) coming from that kind of architecture (distributed), I recommend event sourcing. It removes the complexity and many headaches that come with the nature of distributed systems if someone try to use traditional database paradigms.

For question #2: both would work. To know what is better and cheaper we'll need to know more about your use case.

# Answer
> 1 votes
The cost will depend on the number of messages you write per second. Most often DDB is cheaper. If you have a large throughput, 10k requests per second, then your DDB costs will need to be carefully considered vs. RDS.

In this case however, I would actually reccommend using S3 to output your log messages to a file. I think technically this will be a good enough solutin, at a fraction of the cost to setup and maintain compared to either DDB or RDS.

---
Tags: aws-lambda, amazon-rds, databases
---


title: List encrypted RDS snapshots using CLI

text:
# Question
Title: List encrypted RDS snapshots using CLI
I have automated the RDS snapshot copy across the region, but the Snapshot which is encrypted gets fail to copy. I have figured a way to copy an encrypted snapshot using

```
$  aws rds copy-db-snapshot   --source-db-snapshot-identifier arn:aws:rds:eu-west-1:XXXX:snapshot:XXX-2017-01-31-04-30      --target-db-snapshot-identifier mydbsnapshotcopy   --kms-key-id XXX --region eu-central-1 --source-region eu-west-1

```

I want to filter/test the snapshots if they are encrypted or not, if they are encrypted I will perform the above-defined operation. I tried this which is not working.

```
$ aws rds describe-db-snapshots  --filter "Name=encrypted,Values=true"

```

How do I list/filter all the encrypted snapshots and move it to a file?

# Answer
> 3 votes
Either use the `--query` attribute

```
aws rds describe-db-snapshots --query "DBSnapshots[?Encrypted].DBSnapshotIdentifier"

```

If the `[?Encrypted]` does not work, try `[?Encrypted == 'true']` as the first is from JMESPath improved filters. The quotes might need to be backticks in some case. It all depends.

Second way:

```
aws rds describe-db-snapshots --output json | jq '.DBSnapshots | map(select(has("Encrypted"))) | .[].DBSnapshotIndetifier'

```

One of them should work. Might need a bit messing around as I don't have your output to tune the `jq` command. Feel free to edit and fix the answer.

# Answer
> 1 votes
You will require a **script** that interacts with the **AWS cli** to accomplish your goal.

To list encrypted drives I use the following query when describing the DB snapshots `--query 'DBSnapshots[*].[DBSnapshotArn,Encrypted]'`

I have put together the following script which will **look for all encrypted DBS snapshots and then copy all of them to a different region**. It can also be modified to copy unencrypted snapshots to a different region and then encrypt them.

> I have explained the script in the comments. It is quite **dirty**... so feel free to **improve** it!

```
#!/usr/bin/env bash
#
# Copy encrypted or unencrypted DBSnapshots to an encrypted DBSnapshot in a different region
#

aws_profile=""
aws_source_region=""
aws_dest_region=""
aws_kms_key_id="" #destination kms key ID
num=1

aws --profile "${aws_profile}" --region "${aws_source_region}" rds describe-db-snapshots \
  --query 'DBSnapshots[*].[DBSnapshotArn,Encrypted]' \
  --output text > rds-snapshots-list.txt

echo "Encrypted snapshots"
awk '{IGNORECASE=1}{if ($2 == "True") print}' rds-snapshots-list.txt #Show list of encrypted snapshots
echo "Unencrypted snapshots"
awk '{IGNORECASE=1}{if ($2 == "False") print}' rds-snapshots-list.txt #Show list of unencrypted snapshots

IFS=$'\n' read -d '' -r -a rds_list_lines < rds-snapshots-list.txt #Move txt file content into an array

echo "Copying encrypted snapshots from ${aws_source_region} to ${aws_dest_region}"
for i in "${rds_list_lines[@]}"; do #Loop through array
if [[ "${i}" == *"True"* ]]; then #Check for encrypted rds snapshot & if true copy snapshot
    source_snapshot_ident="$( echo "${i}" | sed 's/[[:blank:]]True.*//')" #Remove all spaces tabs and the word true
    target_snapshot_ident="$( echo "$source_snapshot_ident" | sed 's/.*:snapshot:rds:.//')snapshotcopy$num" #Name of target snapshot with incrementing number
    ((num++))

    aws --profile "${aws_profile}" --region "${aws_dest_region}" rds copy-db-snapshot \
      --source-db-snapshot-identifier "${source_snapshot_ident}" \
      --target-db-snapshot-identifier "${target_snapshot_ident}" \
      --source-region "${aws_source_region}" \
      --kms-key-id "${aws_kms_key_id}"
fi
done

```

---
Tags: amazon-web-services, aws-cli, amazon-rds
---


# Question
Title: How to organize infrastructure as code projects?
I have completed an Infrastructure-as-Code project using Terraform and Ansible. Currently, I organize my project as follows:

```
Project1
    Terraform
      ec2.tf
      rds.tf
      …
      variables.tf
      scripts\<some script files>
      templetes\<some tfl files as templates>
    Ansible
      Playbooks<some playbook yml files>
         playbook1.yml
         playbook2.yml
         ……
      ansible_hosts
      vars\<some variable files used by vars_file>
      files\<files to be copied to remote hosts>
      templates\<jinja2 templated files>
      roles\<some roles such as webserver, dbserver, etc.>

```

Note I did not layout Ansible code using the following links yet:

e.g. I did not put `vars/`, `templates/`, `files/` to individual role folder since Most of my roles did not have vars and templates. Besides, I want to manage `vars`, `templates`, `files` in one file instead of multiple roles, so I moved `vars/`, `templates/` and `files/` out of `roles/`.

For my next project, I may build an Infrastructure-as-code project using CloudFormation/Terraform and Ansible/Chef. Next, I may work on Jenkins, code pipeline, and so on.

In general, should I organize my DevOps projects based on projects, in this way:

```
Project1:
  Terraform
  Ansible
Project2:
  Terraform
  Ansible
  ......

```

Or should I organize my DevOps projects based on tools, in this way:

```
Terraform
  Project1
  Project2
Ansible
  Project1
  Project2
Cloudformation
  Project3
  Project4

```

I believe separate with projects makes more sense. but then how do we reuse code? for example, what if project1 and project2 use similar terraform code and some ansible roles. and I want to reuse the code of project1 to project2?

or is there any typical samples online I can refer?

# Answer
> 4 votes
The layout of your repository depends in many ways upon the context you are developing the automation in. If, for example, you are building out the infrastructure for a product as part of a product team, then it would make sense to tie the infrastructure to the product - i.e. keep the infrastructure in the same repository as the software source code.

If however, you are building out common infrastructure components to stand-up basic infrastructure for DNS, File Storage, Email, etc. then it probably makes more sense to have a single `common-infrastructure` repository. Some of this will be tied to the tool you are using, Terraform for example leads you down the path of having a single repository for all of your environments to support the `terraform.tfstate` construct.

# Reusability

You quite rightly called out reusability and code sharing as a problem you are going to encounter sooner rather than later. This is an important principle in Software Development called **Don't Repeat Yourse - or DRY** for short. Most all DevOps tools allow you to modularize your code in such a way that you don't need to copy/paste code:

# Conceptual Consistency

I **advise** that you don't use too many tools, try and standardise on a single tool rather than changing tools for each project, typically you see sets of tools being used across an organization:

* Jenkins, Terraform and Ansible
* TeamCity, Octopus Deploy and Desired State Configuration

This makes it easier to share code across projects, document what has been done so far and handover the code to other or future developers.

---
Tags: ansible, terraform, infrastructure-as-code, repository
---


title: Passing in a file with configuration

text:
# Question
Title: Passing in a file with configuration
One of the steps in our deployment process requires a configuration file that we don't want to commit to the repository and rather have it passed in by the Bamboo itself. Is there some sort of task/way to conveniently add it? It can be either a task that creates a file in work directory or assigns the file content to a variable.

What we want to avoid is to have the file on the disk and passing in the path to the file - I'd rather have a way of pasting the file directly into a task.

Currently, the deployment step calls a script that expects the content of said config file to be passed in (shortened for brevity):

`deploy.ps1 --settings=<file_content>`

edit: The settings file is a lengthy json file, with several arrays on few levels. I've been considering just assigning an inlined content of that file to the environment variable, but I was wondering if there was a better way to pass it as well as allow maintainers edit the values, if necessary.

Splitting the config into several Bamboo environment variables is not really feasable, especially since one of our use cases requires the ability to run the deployment script without Bamboo, in case of server failure, for example.

# Answer
> 5 votes
It is convenient to use Environment Variables to provide runtime configuration for such scripts. Especially when it contains secrets that you don't want to hold in any persistent storage or in version control.

When you do actually have secrets, it is even better to use some kind of secret management system that will provide these secrets on request. One such tool that I highly recommend is HashiCorp's Vault.

Depending on how you are executing the jobs on the Bamboo Server, another option might be to store a local file on the server and reference that during builds. This is not really recommended since the server then becomes a special snowflake that only a few people know how to actually set up. It is much better to allow malfunctions with the server, or even the loss of all server data, to not affect your ability to deploy code. Thus having special files on the server itself is not so great.

---
Tags: deployment, bamboo
---


# Question
Title: Vagrant box take long time to access shared /vagrant folder
I have a vagrant box running on windows 10 hyper-v

config.vm.box = "kmm/ubuntu-xenial64" #A standard Ubuntu 16.04 LTS (x64).
config.vm.provider "hyperv"

Just after provisioning it, it seems to be running o.k. but then if I exit and ssh back to it then trivial actions such as pressing tab after typing `cd /vag` would take tons of time (e.g 30 sec or more) and after that a `cd /vagrant` worked o.k. then ls -la worked o.k. and another `ls -la` took something like 3 minutes or so.

During that time hyper-v is showing the virtual machine at 0% CPU.

Any suggestions on where I should look to try and figure out why is this happening?

# Answer
> 1 votes
It seems ther reason for this was that the SMB connections would disconnect. using

```
net config server /autodisconnect:-1

```

in a "run as administrator" command window fixed it.

as explained here: https://www.vagrantup.com/docs/synced-folders/smb.html#preventing-idle-disconnects

---
Tags: vagrant
---


# Question
Title: Chef (well, Test Kitchen) hanging when using local cookbook
I am new to Chef, trying to put together a POC for my company. So I've got a project that has two cookbooks under the cookbook folder. One cookbook is a "base" cookbook called 'hydra-base', the other (called 'automator') is supposed to extend it. Right now all I have is a test in the extending cookbook, and a recipe invocation.

My problem is when I attempt to verify it with Kitchen, everything uploads ok until it gets to trying to apply the recipe, then I get the following line: `Recipe: hydra-base::default` and then kitchen just hangs. It has been idle for about 45 minutes now. If I break out of the tool (ctrl-c) then rerun kitchen converge, it gives me this error then hangs:

`[2018-01-09T17:35:23+00:00] WARN: Chef client 1514 is running, will wait for it to finish and then run.`

So it appears that chef still thinks its trying to update something. Here are the relevant files:

```
# automator/recipes/default.rb
include_recipe 'hydra-base::default'

# automator/Berksfile
source 'https://supermarket.chef.io'

cookbook 'hydra-base', path: '../hydra-base'
metadata

# automator/metadata.rb
depends 'hydra-base'

```

calling `kitchen verify -l debug` does not seem to provide any additional information after the line that hangs.

Can someone please give me a hint on how to fix this or how to get more information out of kitchen? Thanks

# Answer
It turns out that AWS was just excruciatingly slow and apt-get (applied in the base) was timing out. I would have known this had I gone back earlier and tried testing the base cookbook.
> 1 votes

---
Tags: chef, testing
---


# Question
Title: Debug Docker networking and Windows 10 firewall
**Given:**

1. Windows 10 / 64 bit
2. Docker edge 17.11-ce-win40

**Experiment:**

```
docker run -it --rm -p 8888:8080 tomcat:8.0

```

Expected behaviour:

* If all network settings are okay, Tomcat must be reachable locally and from outside on the host port 8888.

**Observed behaviour:**

* `curl localhost:8080` inside container proves Tomcat to be up and running
* ERR\_EMPTY\_RESPONSE locally/from outside
* Flipping Windows Firewall leads to the question whether vpnkit should be allowed, allowing leads to no change
* nmap from outside reveals port open
* telnet is possible as well but no further interaction
* `docker inspect -f '{{ .NetworkSettings.Networks.nat.IPAddress }} <CONTAINER_ID>` outputs `<no value>`
* find `ipconfig /all` output below

**Further debugging**

```
ipconfig /all [excerpt]

   Hostname  . . . . . . . . . . . . : foo
   Primäres DNS-Suffix . . . . . . . : foo.com
   Knotentyp . . . . . . . . . . . . : Hybrid
   IP-Routing aktiviert  . . . . . . : No
   WINS-Proxy aktiviert  . . . . . . : No
   DNS-Suffixsuchliste . . . . . . . : foo.com

Ethernet-Adapter vEthernet (DockerNAT):

   Verbindungsspezifisches DNS-Suffix:
   Beschreibung. . . . . . . . . . . : Hyper-V Virtual Ethernet Adapter
   Physische Adresse . . . . . . . . : 00-15-5D-48-8F-A4
   DHCP aktiviert. . . . . . . . . . : No
   Autokonfiguration aktiviert . . . : Yes
   Verbindungslokale IPv6-Adresse  . : fe80::e93b:535c:4ea5:84cf%18
   IPv4-Adresse  . . . . . . . . . . : 10.0.75.1
   Subnetzmaske  . . . . . . . . . . : 255.255.255.0
   Standardgateway . . . . . . . . . :
   DNS-Server  . . . . . . . . . . . : fec0:0:0:ffff::1%1
                                       fec0:0:0:ffff::2%1
                                       fec0:0:0:ffff::3%1
   NetBIOS über TCP/IP . . . . . . . : Active

Ethernet-Adapter Ethernet:

   Verbindungsspezifisches DNS-Suffix: foo.com
   Beschreibung. . . . . . . . . . . : Realtek USB GbE Family Controller
   Physische Adresse . . . . . . . . : D4-81-D7-57-AC-C6
   DHCP aktiviert. . . . . . . . . . : Yes
   Autokonfiguration aktiviert . . . : Yes

```

# Answer
Since Windows use a tiny VM called MobyLinux, the port mapping will happen with tiny VM and not with Windows Host. If you are not reachable from Windows Host to Docker MobyVM, check windows events for possible error. For more networking related information, please see -

https://docs.docker.com/engine/userguide/networking/
> 2 votes

# Answer
The tomcat is reachable on port 8080 instead of 8888 when the following is defined in the question

> ```
> docker run -it --rm -p 8888:8080 tomcat:8.0
>
> ```

Documentation

> Publish or expose port (-p, –expose)
>
> ```
> $ docker run -p 127.0.0.1:80:8080 ubuntu bash
>
> ```
>
> This binds port 8080 of the container to port 80 on 127.0.0.1 of the host machine. The Docker User Guide explains in detail how to manipulate ports in Docker.
>
> ```
> $ docker run --expose 80 ubuntu bash
>
> ```
>
> This exposes port 80 of the container without publishing the port to the host system’s interfaces.

Based on the documentation, the first port is the internal port, i.e. 8080 and the second one, i.e. 8888 the external or exposed one.
> 1 votes

---
Tags: docker, windows, firewall
---


# Question
Title: How do I automate deployments with Kubernetes?
Assuming I have Kubernetes deployed via Rancher and Jenkins is building new images and pushing them to DockerHub upon check-in of new code to GitHub, how do I automate getting new images deployed?

Another way to ask the question might be, “I used to use Octopus to manage my deployments. Is there something similar built in to Kubernetes or Rancher?” Ultimately, it’s this last gap that I’m struggling with.

# Answer
As of you are using Rancher the easiest way would be to register custom Rancher Catalog and create an item for each stack/service you want to deploy.

Rancher Catalog is a Git repository with the following structure

Then in Jenkins you can create a job that would call Rancher REST API to deploy/update stack/service.

In its turn Rancher would pull latest version of the Docker image for this service and deploy it corresponding to Docker Compose file from the catalog.

Pros:

* generic approach that might be used for almost every app
* Jenkins itself might be deployed into Rancher environment, agents might be created in Kubernetes cluster

Cons:

* development team should follow solid release strategy to be able to use generic builds
* storage drivers are still under the question in alpha release of Rancher 2.0
> 4 votes

# Answer
Terraform with Kubernetes provider.
> -2 votes

---
Tags: docker, deployment, kubernetes, rancher
---


title: Is icmp blocked at travis-ci?

text:
# Question
Title: Is icmp blocked at travis-ci?
I have a

```
ping -c 1 -q n.n.n.n command

```

that fails only at Travis CI.

Is ping blocked there?

# Answer
Found the answer to my own question, straight from travis support

> Hi Name,
>
> Thanks for writing in. Yes indeed, on our container based infrastructure, ICMP is not possible. Could you speficy `sudo:
>  required` in the .travis.yml and see if it brings improvement?
>
> Thanks!
>
> Joep
> 1 votes

---
Tags: travis-ci
---


# Question
Title: What is the major difference between continuous delivery and continuous deployment
I am a beginner in devops.I am getting confused with the answers on google so posted here.

# Answer
"Continous delivery" is the general principle of a development process (including development, testing, etc.) producing software in a way where you could deploy often, ideally at any time. In contrast to a classical waterfall model where you would have fixed deployment dates which would be months or even years apart and planned with huge feature sets. These days, this means that there is some CI/CD pipeline which may or may not include the final step of "deployment".

"Continuous deployment" is a specific aspect which means that the final step of the process, the actual deployment, is automated as well, and happens very frequently, ideally on every commit (after running through a rigorous CI/CD pipeline which may or may not include any human interaction).

There is no *versus* here. Continuous Deployment is an optional sub-part of Continous Delivery.
> 2 votes

# Answer
Continuous delivery: test the application automatically when the code changes

Continuous deployment: deploy the application to production automatically when the code changes
> 0 votes

---
Tags: continuous-delivery, continuous-deployment
---


# Question
Title: How should we automatically rebuild immutable infrastructure when new packages are available?
We're going to be using Terraform to automate our infrastructure deployment and Packer to create the machine images deployed by Terraform. By following immutable infrastructure design principles, we will implement patching by creating a new image with the patch applied and then redeploy our infrastructure.

With this setup, are there any additional tools we can use to automatically detect when a package or the OS itself in our base image needs updating and trigger the build pipeline?

Chef Automate seems close to what I'm looking for, however, it seems to scan running nodes for compliance rather than analyze the image manifest itself.

Thanks!

# Answer
> 6 votes
Part of adopting the Immutable Infrastructure Pattern is decomposing your system into small manageable pieces that can move through CI/CD Pipeline very quickly, this means that OS patches can be done quickly and in a controlled manner. I often see clients ending up with a halfway house where infrastructure is mostly immutable.

However, there are a few approaches to this which I have used in large-scale deployments of cloud architecture, typically I implement more than one as part of a Defense in Depth strategy:

* **Security Information and Event Management (SIEM)**: These products, for example, LogRhythm Security Intelligence Platform and more generic products such as ElasticStack take feeds from the operating system, which includes the output from a frequent check for updates. The trick here is to get the information about what to patch quickly and automatically to inform decisions about when to roll those through your pipeline.
* **Vulnerability Management** systems are more tailored than SIEM in that they are only focusing on vulnerabilities across the system so could catch issues with libraries installed as part of the software deployed to the system but not managed by the Operating System. This might highlight vulnerabilities for which there is no patch (yet... hopefully).
* **Dependency Check** tools form part of your pipeline and can be configured to fail the build if check-ins add vulnerabilities, this also works if a new vulnerability is added to the tool since the last check-in.

# Answer
> 1 votes
You could do a vulnerability scan with AWS Inspector and look for CVE vulnerabilities. Inspector has a CloudWatch metric to cause an action when there are findings. Would be tricky to determine WHICH packages need to be updated, but you could just update everything. You don’t have to use Inspector, any vulnerability scanning tool you can hook into would work. Vulnerability scanning is a best practice in general.

Another idea would be to setup a cron job that does apt-get update && apt-get upgrade —dry-run \> stuff-that-should-be-updated.txt and parse the text file for a list of packages to be updated and feed that back into your build process.

# Answer
> 0 votes
This approach looks to solve your purpose using Chef audit but it has got it's own problems. https://joshuakugler.com/using-chef-zero-audit-mode-and-packer.html

A custom solution - If you want to use a Compliance Policy ( for example , CIS, HIPPA ), create a job which will run audit parser on running package images and notify a update task upon non-complaint status. There are OVAL parser and OVAL definition available for free for above policies. For custom aduit policies you can write your own parser and audit the image.

---
Tags: chef, terraform, infrastructure-as-code, packer, immutable-servers
---


# Question
Title: What is difference between release and deployment?
What is the basic difference between software release and deploy? What is the relation between software release management and continuous deployment?

# Answer
> 3 votes
From Wikipedia:

> Release management is the process of managing, planning, scheduling and controlling a software build through different stages and environments; including testing and deploying software releases.

Depending on the context the term `release` is used to reference either of:

* the version of the software being released (from a version control system prospective)
* the artifacts actually being delivered corresponding to that version (which can include physical media, printed documentation, etc)
* the process leading to making those artifacts available and, in the case where the release includes deployment - deploying them in the respective environment (which may or may not be the production one)

Continuous Deployment (together with Continuous Integration and Continuous Delivery) is the modern DevOps methodology aiming to releasing better quality software as fast as possible.

So you can see Continuous Deployment being part of the release management in some cases, but you can also see release management without it - for example in the waterfall development model.

Related: How does continuous integration relate to continuous delivery / deployment?

---
Tags: terminology, deployment, continuous-deployment, release-management, release
---


# Question
Title: create resource name based on terraform.workspace
I want to have different s3 names based on different environments using terraform.

If I define the name in terraform.tfvar, it won't work:

```
s3_bucket_name          = "tf-${terraform.workspace}"

```

If I hardcode s3 bucket name in s3.tf, it works:

```
resource "aws_s3_bucket" "s3" {
   bucket        = "tf-${terraform.workspace}"
   ....
}

```

how to define the workspace related name from terraform.tfvar? I do not want to hard code it in tf file.

# Answer
Variables are static, literal inputs to your configuration that come from either the command line or from `tfvars` files. It is not possible to use interpolations here, since the variable values are processed before interpolation begins in order to make them available to other expressions.

---

If the goal is to define the S3 bucket name in one place and re-use it several times in the same configuration, the *Local Values* feature can do this if you are using Terraform v0.10.4 or newer. You can declare the named value in one of your `.tf` files:

```
locals {
  s3_bucket_name = "tf-${terraform.workspace}"
}

```

You can then interpolate this value in various places in your configuration using the name `local.s3_bucket_name`, like this:

```
resource "aws_s3_bucket" "s3" {
  bucket = "${local.s3_bucket_name}"
  ...
}

```

Local values are conceptually similar to module outputs but they are exposed within the same module they are defined in, rather than passing a value up to the parent module.

---

If the goal is to allow this generated default name to be overriden optionally by a variable, then that can be achieved with some conditional logic:

```
variable "override_s3_bucket_name" {
  default = ""
}

locals {
  s3_bucket_name = "${var.override_s3_bucket_name != "" ? var.override_s3_bucket_name : "tf-${terraform.workspace}"}"
}

resource "aws_s3_bucket" "s3" {
  bucket = "${local.s3_bucket_name}"
  ...
}

```

The above will generate a default name based on the selected workspace if `override_s3_bucket_name` is not set, but if that variable is set then its value will be used instead.
> 4 votes

---
Tags: terraform
---


# Question
Title: What's missing from this simple recipe that creates a user?
I've spun many boxes in the cloud before, but was growing tired of going through the same dance; and as I'm trying to get familiar with Chef, I figured, that's a good use case to do a little learning.

Ah, that feeling when you think you know how it'll work - but it doesn't. Since I'm new to Chef, I suspect I'm not doing something right in this recipe, which is like my 2nd ever. NOTE: I understand there probably are 'cookbooks' out there that do something like that, but at this point I'm trying to build some ver basic knowledge with the simplest approach.

```
user 'foox' do
  comment 'test me'
  uid '6711'
  home '/home/foox'
  shell '/bin/bash'
  password '<password_shadow_here>'
end

directory "/home/foox" do
  owner "foox"
  group "foox"
  mode 00755
  action :create
end

directory "/home/foox/.ssh" do
  owner "foox"
  group "foox"
  mode 00600
  action :create
end

file '/home/foox/.ssh/authorized_keys' do
  owner "foox"
  group "foox"
  mode "00600"
  content IO.read('/root/.ssh/authorized_keys')
  action :create
end

```

Now, this does create the user with a password and home folder with .ssh, and copies the public key from root's own file. But this system ends up failing to let me log in -- `Permission denied (publickey)` -- when using the same private key as what I have as my root's. But, well, root user wasn't created by my recipe. And if I create my own user manually, it's working fine.

I guess there's something missing in my recipe, then?

# Answer
> 7 votes
I'll put in an answer although I'm not 100% sure. I think that you got the permissions wrong on `~/.ssh` and it should be `0700` instead of `0600`, otherwise the user cannot access it.

Another tip which is unrelated to your problem, the `user` resource has a property `manage_home` (https://docs.chef.io/resource\_user.html#properties) which will create the home directory for you and will you save a redundant resource.

---
Tags: chef, linux
---


# Question
Title: Ensure a line is uncommented in an XML file using ansible
How can I use ansible to ensure that a line is uncommented in an XML file? In `/etc/tomcat8/context.xml` I want to ensure that the section:

```
<!-- Uncomment this to disable session persistence across Tomcat restarts -->
<!--
<Manager pathname="" />
-->

```

Is changed to:

```
<!-- Disable session persistence across Tomcat restarts -->
<Manager pathname="" />

```
* The lineinfile module doesn't appear to be able to help because this is a multi-line section
* The xml module might be able to ensure that the `<Manager...` is in place, but it doesn't appear to be able to delete the commented out version.
* I could write a script to do this with `sed` if I have to, but I find it is better to use ansible modules when possible.

# Answer
> 3 votes
As mentioned in the question and the comments on it, this isn't an area in which Ansible excels. The "best" solution depends a bit on the type of file you're dealing with, and the structure of it.

The cleanest way to deal with these sorts of changes in Ansible is to templatize the entire thing; this allows you to see the entire file at once, rather than piecing it together through many places, and is much less fragile than pattern- or line-based approaches. When you need to add configuration in multiple places (for instance, different roles), that's when dynamic-loading directives become very useful, like Apache httpd's `Include`:

```
Include /etc/httpd/conf.d/*.conf

```

This then allows you to write individual configuration files from different places, and let the client software concatenate them. However, as far as I can tell from a quick search, Tomcat doesn't support this, so you would need to operate with a single template file.

`lineinfile` is a hacky solution that's very fragile, and `template` is almost always a better option. However, in this case, the node that you're adding is a self-closing one, so as long as it is top-level, it can go anywhere and `lineinfile` will work (I know you say it won't, but I'll get back to that). `xml` is a similar option that's a bit less fragile.

Those two you've ruled out because they can't delete the comment you have in the file. However, I'll argue that that's an improper requirement (in most cases, and probably yours).

When using a configuration management tool, the source of truth shifts from the files themselves on servers to the configuration that's checked in to master. If someone wants to edit the file, they'll do so by editing the configuration management scripts, not by editing the file directly on the server. Therefore, there is no need for comments in the resultant file that lives on the server, except perhaps one at the top that says "this is auto-generated, go look at source control instead". The comment should be *in your templating code* \- and as long as the result compiles and works correctly, you shouldn't care at all what it looks like.

---
Tags: ansible, configuration
---


# Question
Title: Discovering extra args needed for running ansible playbooks
I've got a number of ansible playbooks that I come back to from time to time, some have required and optional arguments that I forget about from time to time.

I know there's a list-tags, list-hosts and list-tasks option baked into the ansible-playbook command, there's no list-args, which would be super useful, but maybe it means I'm using ansible wrong?

So what strategies do devops engineers use to manage "api's" for ansible playbooks? Maybe just a recursive grep command to list non-standard args that aren't registered variables like this

```
grep -or "{{[a-zA-Z_ ]\+}}" roles/.  | grep -v "ansible" | grep -v "item"

```

only better.

---

For instance, I've got a program I want to put on a device called "showstopper", but I specify the version with extra\_args and if I don't specify the version, it doesn't get installed.

```
 - name: download showstopper from artifactory
   local_action: get_url url=https://example.net/artifactory/generic-release-local/com/cdw/mans/silo/showstopper/showstopper-{{ showstopper_version }}.noarch.rpm dest=/tmp validate_certs=no
   run_once: true
   when: showstopper_version is defined

 - name: copy showstopper to silo
   copy: src=/tmp/showstopper-{{ showstopper_version }}.noarch.rpm dest=/tmp
   when: showstopper_version is defined

 - name: Install showstopper
   yum: name=/tmp/showstopper-{{ showstopper_version }}.noarch.rpm state=present
   notify:
    - start cron
   when: showstopper_version is defined

```

This works good when I run it the way I intended to run it, but in 2 months when I want to run it on another machine, I forgot to specify showstopper\_version and the last few machines I've built went out without that configured on it.

# Answer
> 4 votes
If an extra variable is truly required, you can add `assert` statements at the top of the role to stop it from executing if they're not defined:

```
┌─[jamesph@geror] - [~/temp] - [Sat Jan 13, 09:40]
└─[$]> cat assert.yml
- hosts: localhost
  vars:
    foo: False
  tasks:
    - assert:
        that:
          - "foo is defined"
          - "bar is defined"
┌─[jamesph@geror] - [~/temp] - [Sat Jan 13, 09:40]
└─[$]> ansible-playbook assert.yml
 [WARNING]: Unable to parse /etc/ansible/hosts as an inventory source

 [WARNING]: No inventory was parsed, only implicit localhost is available

 [WARNING]: Could not match supplied host pattern, ignoring: all

 [WARNING]: provided hosts list is empty, only localhost is available

PLAY [localhost] *************************************************************************************

TASK [Gathering Facts] *******************************************************************************
ok: [localhost]

TASK [assert] ****************************************************************************************
fatal: [localhost]: FAILED! => {
    "assertion": "bar is defined",
    "changed": false,
    "evaluated_to": false
}
    to retry, use: --limit @/Users/jamesph/temp/assert.retry

PLAY RECAP *******************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=1

```

# Answer
> 2 votes
> some have required and optional arguments that I forget about from time to time

http://docs.ansible.com/ansible/latest/playbooks\_variables.html#variables-defined-in-inventory

> Ok, so if you are writing a redistributable role with reasonable defaults, put those in the roles/x/defaults/main.yml file. This means the role will bring along a default value but ANYTHING in Ansible will override it. It’s just a default. That’s why it says “defaults” :) See Roles for more info about this:
>
> ```
> ---
> # file: roles/x/defaults/main.yml
> # if not overridden in inventory or as a parameter, this is the value that will be used
> http_port: 80
>
> ```
>
> If you are writing a role and want to ensure the value in the role is absolutely used in that role, and is not going to be overridden by inventory, you should put it in roles/x/vars/main.yml like so, and inventory values cannot override it. -e however, still will:

In summary, always set reasonable defaults, e.g. a software version that exists instead of using a non-existing one or empty variable and if there is a constant, put it in the vars directory.

There is a number of galaxy playbooks that could be checked:

---
Tags: ansible
---


# Question
Title: Execute multiple Ansible tasks with the same list of items
I am using an ansible playbook to configure Apache for a list of sites. The playbook has to copy the virtual host configuration template for each site into place, and then enable each site using `a2ensite`:

```
- name: Install apache site conf
  template: src=apache-sites-{{item}}-conf.j2 dest=/etc/apache2/sites-available/{{item}}.conf mode=0644
  with_items:
  - sitea
  - siteb
  - sitec
  - sited
- name: Enable site apache conf
  command: a2ensite {{item}}
  args:
    creates: /etc/apache2/sites-enabled/{{item}}.conf
  with_items:
  - sitea
  - siteb
  - sitec
  - sited

```

I don't like having to repeat the same list for each task. How to I configure the playbook to execute both tasks with the same list of items?

# Answer
I found a solution using file globs. Since I have a configuration file for each site, I can simply use the list of those files to iterate over all of them. That way I don't have the list of sites in my task file even once, let alone twice. All I need to do to add a site is add a file.

To make things a little easier I created a directory for the templates:

* `roles/webserver/templates/apache-sites/sitea.conf.j2`
* `roles/webserver/templates/apache-sites/siteb.conf.j2`
* `roles/webserver/templates/apache-sites/sitec.conf.j2`
* `roles/webserver/templates/apache-sites/sited.conf.j2`

Then in `roles/webserver/tasks/main.yml` I can use that list of files and some regular expressions:

```
---
- block:
  - name: Install apache site conf
    template: src={{item}} dest=/etc/apache2/sites-available/{{item|regex_replace(".*/","")|regex_replace("\.j2$","")}} mode=0644
    with_fileglob:
    - "roles/webserver/templates/apache-sites/*"
  - name: Enable site apache conf
    command: a2ensite {{item|regex_replace(".*/","")|regex_replace("\.conf\.j2$","")}}
    args:
      creates: /etc/apache2/sites-enabled/{{item|regex_replace(".*/","")|regex_replace("\.j2$","")}}
    with_fileglob:
    - "roles/webserver/templates/apache-sites/*"
  become: yes

```

This technique could even be used with empty dummy files to create a list for other applications.
> 2 votes

# Answer
Make separate tasks file `make_site.yml`:

```
---
- name: Install apache site conf
  template:
    src: apache-sites-{{ site }}-conf.j2
    dest: /etc/apache2/sites-available/{{ site }}.conf
    mode: 0644

- name: Enable site apache conf
  command: a2ensite {{ site }}
  args:
    creates: /etc/apache2/sites-enabled/{{ site }}.conf

```

And in your playbook:

```
- include_tasks: make_site.yml
  with_items:
    - sitea
    - siteb
    - sitec
    - sited
  loop_control:
    loop_var: site

```
> 21 votes

# Answer
Konstantin gave a good answer; here is an additional flavor on it.

I will commonly define the lists as variables, and just write two separate loops over the same variable:

```
┌─[jamesph@geror] - [~/temp] - [Sat Jan 13, 10:06]
└─[$]> cat loops-1.yml
- hosts: localhost
  gather_facts: no
  vars:
    menu:
      - Egg and Spam
      - Spam, bacon, sausage and Spam
      - Spam, Spam, Spam, Spam, Spam, Spam, baked beans, Spam, Spam, Spam and Spam
  tasks:
    - debug:
        msg: "We have {{ item }}"
      with_items: "{{ menu }}"
    - debug:
        msg: "I love {{ item }}!"
      with_items: "{{ menu }}"
┌─[jamesph@geror] - [~/temp] - [Sat Jan 13, 10:06]
└─[$]> ansible-playbook loops-1.yml
 [WARNING]: Unable to parse /etc/ansible/hosts as an inventory source

 [WARNING]: No inventory was parsed, only implicit localhost is available

 [WARNING]: Could not match supplied host pattern, ignoring: all

 [WARNING]: provided hosts list is empty, only localhost is available

PLAY [localhost] **********************************************************************************************************************

TASK [debug] **************************************************************************************************************************
ok: [localhost] => (item=Egg and Spam) => {
    "changed": false,
    "item": "Egg and Spam",
    "msg": "We have Egg and Spam"
}
ok: [localhost] => (item=Spam, bacon, sausage and Spam) => {
    "changed": false,
    "item": "Spam, bacon, sausage and Spam",
    "msg": "We have Spam, bacon, sausage and Spam"
}
ok: [localhost] => (item=Spam, Spam, Spam, Spam, Spam, Spam, baked beans, Spam, Spam, Spam and Spam) => {
    "changed": false,
    "item": "Spam, Spam, Spam, Spam, Spam, Spam, baked beans, Spam, Spam, Spam and Spam",
    "msg": "We have Spam, Spam, Spam, Spam, Spam, Spam, baked beans, Spam, Spam, Spam and Spam"
}

TASK [debug] **************************************************************************************************************************
ok: [localhost] => (item=Egg and Spam) => {
    "changed": false,
    "item": "Egg and Spam",
    "msg": "I love Egg and Spam!"
}
ok: [localhost] => (item=Spam, bacon, sausage and Spam) => {
    "changed": false,
    "item": "Spam, bacon, sausage and Spam",
    "msg": "I love Spam, bacon, sausage and Spam!"
}
ok: [localhost] => (item=Spam, Spam, Spam, Spam, Spam, Spam, baked beans, Spam, Spam, Spam and Spam) => {
    "changed": false,
    "item": "Spam, Spam, Spam, Spam, Spam, Spam, baked beans, Spam, Spam, Spam and Spam",
    "msg": "I love Spam, Spam, Spam, Spam, Spam, Spam, baked beans, Spam, Spam, Spam and Spam!"
}

PLAY RECAP ****************************************************************************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0

```

This works nicely with variable precedence, for instance by defining different sets of servers per environment. It also works when you need to perform various other non-looped tasks in-between the two loops.
> 2 votes

---
Tags: ansible
---


# Question
Title: Apache container cant access php files mounted in /var/www/ - 403 error
I want to run my php website in a container with apache and php7 I am using this docker image : https://hub.docker.com/r/nimmis/apache-php7/

I run Fedora so I enter the example command given in docker hub in my terminal.

```
docker run -d -p 8080:80 -v {my_php_app_path}:/var/www/html nimmis/apache-php7

```

The container pull and run fine. But once i try to access the asset (on localhost:8080) I mounted to it, I receive a 403 error. So now i try to connect to the container to see what happens.

I use the command adviced by the author:

```
docker exec -ti <container id> /bin/bash

```

Once in it, I cd to /var/www/html all the file are presents but I cant read or ls in the folder. the root of the container cant do anythig with it. Here is what I get :

```
>>> root@a1f13dc4c29d:/var/www/html# ls
>>> ls: cannot open directory '.': Permission denied

```

Why is it that way ? Am I missing a step ?

Any help would be loved :D

# Answer
SELinux prevent the docker filesystem to work in the intended way, to fix this issue just add the :Z flag at the end of the volume declaration. It goes something like this:

```
docker run -d -p 8080:80 -v {my_php_app_path}:/var/www/html:Z nimmis/apache-php7

```
> 1 votes

---
Tags: docker, linux, php, apache-httpd
---


# Question
Title: Should worker processes of web applications be placed in separate containers?
My setup is based on a Flask app that has a PostgreSQL and Redis database. In addition, a worker process is started to handle asynchronous tasks such as sending emails. I use Gunicorn to create multiple app processes. Passing of jobs from the app processes to the worker process is done via Redis.

The question is, should the worker process be started in a separate container despite being based on the same code? Currently I use a start script with the relevant bit:

```
gunicorn --bind=0.0.0.0:8000 --workers=3 manage:app &
python -u manage.py run_worker

```

Also if they are to be separated, how can I ensure that they reuse the common file system container? Additionally, when scaling either Gunicorn with the main app or the worker processes, should additional instances of the container be created or the number of processes within a container be increased?

Related: What are the advantages of dockerizing nginx and php in different containers?

# Answer
I can think of only one case where overhead of running workers in separate containers is justified: if your setup uses docker swarm for clustered deployment. This way you will get all of the HA benefits.

Otherwise I don't see a reason to complicate such tasks, especially if they must use strictly the same codebase (which leads me to believe that they share one domain of concerns, so to say). It might seem elegant, but I just don't see enough benefits

p.s. any split of such kind will force you to manage interconnection and complicate setup for local development
> 6 votes

---
Tags: docker, containers
---


# Question
Title: Migrate Rancher host from server to server
I have 4 Rancher servers: one for each environment: dev, sandbox, staging & prod.

3 of them are of an older Rancher 1.4 version (sandbox, staging, prod) & are hosted on OVH hosting, the one left (dev) is on thr latest 1.6.14 & AWS.

I need to move 3 hosts from OVH to AWS & attach all of them to the only one aws Rancher server which already has 1 dev host.

If I stop rancher-agent on one of OVH rancher hosts & reconfigure it to point to aws, then it appears there in the console, but the config for stacks & etc is lost, as it’s, probably, stored in mysql of the original rancher server.

If I deploy a host from each of the OVH rancher servers to AWS, then I’m stuck with the same problem, after I switch the new hosts agents to the only aws rancher server the configurations for stacks & etc will be lost.

Please advise on the strategy to merge all that into single aws rancher server + 4 hosts: one for each environment. I’m trying not to recreate all 3 environments on aws manually, to avoid downtime & a lot of manual work.

Thanks!

# Answer
First of all you need to support HA mode. You will need:

1. external MySQL DB (e.g. AWS RDS Aurora)
2. external load balancer (e.g. AWS ELB)
3. 2 additional Rancher Server Nodes (to support quorum of 3)

Once you will be ready with external DB Rancher would ask you to backup & restore his database in it. After that you'll be asked to redeploy Rancher Server with additional params for external DB connection. Then you'll be able to run two more Rancher Server Nodes (e.g. using AWS ELB).

When previous steps have finished you would be able to add more Rancher Agent Nodes in AWS directly from Rancher Hosts tab. After that you can just switch off OVH nodes - payload would be transferred to AWS automagically.

**Make sure to backup & restore all databases or data-sensitive instances on OVH hosts.**

More information such as HA Requirements etc. can be found in documentation.
> 1 votes

---
Tags: docker, rancher, aws-vpc
---


# Question
Title: availability calculation of a azure service fabric stateful application
I have looked at the question and answer in How do you calculate the compound Service Level Agreement (SLA) for cloud services?

I have a service fabric application which has a stateful service which stores the data in the disks of the VMs themselves. Service fabric relies on underlying Virtual machine scale set's availability, which is 99.95 % The azure storage VHD has a SLA of 99%

Going by the answers in the question above, is my applications availability 99.95 % * 99% = 98.9505 % ?

This seems a little strange that i use a stateful service in azure service fabric which is supposed to be a distributed service mesh framework and the availability of the services built is so low.

I am sure I am missing something. Can anyone tell me if my calculation is correct ? Do I need to consider the number of replicas in my calculations in which case I have as many VHDs as replicas ??

# Answer
> 3 votes
Virtual Machine Scale Sets (VMSSs) don't have a SLA of their own:

> Virtual Machine Scale Sets is a free service, therefore, it does not have a financially backed SLA itself. However, if the Virtual Machine Scale Sets includes Virtual Machines in at least 2 Fault Domains, the availability of the underlying Virtual Machines SLA for two or more instances applies. If the scale set contains a single Virtual Machine, the availability for a Single Instance Virtual Machine applies. See the Virtual Machines SLA for more details.

The above statement points to the fact that VMSSs are just an orchestration layer over the top of Virtual Machines and as you quite rightly pointed out results in an availability for a VMSS of **99.95%**.

In terms of the storage calculation, there are several parameters here:

1. Whether you are using Hot or Cold storage, the availability for Hot storage is 10 times that of cold storage.
2. Whether your application is read or write heavy, what you need to consider here is that there are separate SLAs for reading and writing.

Let's make some assumptions:

1. Storage is, in general, cheep. Therefore, only ever use the Hot Storage Tier.
2. Managed Disks are, in general, easier to use. Therefore, you are limited to only using Locally Redundant Storage.

With the above two assumptions your storage SLA is **99.9%**.

Multiplying these two figures together you get, **99.985%**, this is still kind of low, but higher than your 98% figure above. You can increase this number by hosting Service Fabric across multiple regions:

This would give you system availability close to four nines, i.e. **99.9898%**

---
Tags: cloud, azure, architecture, infrastructure, sla
---


# Question
Title: How to track (non ECS) container costs on EC2?
I've been evaluating Netflix Ice \- as a billing tool - but it appears to only work at the Machine instance level - not at the container level.

I'm looking for a tool to help generate billing reports for using docker containers across a cluster of docker servers on EC2. (Non ECS)

My question is: **How to track (non ECS) container costs on EC2?**

EDIT: I have different groups each running their container on the same host. So I want a way to split instance costs by container usage.

# Answer
Since you are not asking about ECS containers, this means that you must have a bunch of EC2 servers that you manage in some way (ECS/Kubernetes/Nomad/etc...). These servers are most probably included in an Auto Scaling Group.

Once you run containers on these servers, AWS does not charge you for the container. AWS only charges you per second of EC2 instance time while the instance was in a `running` mode.

So in fact you could say that containers are "free", you don't pay for containers, you pay for the infrastructure underneath (EC2 servers).

Netflix ICE is a great tool to track EC2 instance costs, as well as the AWS Billing dashboard.
> 2 votes

---
Tags: amazon-ec2, billing
---


title: difference between packer and matchbox

text:
# Question
Title: difference between packer and matchbox
I have this setup to standup a baremetal machine:

```
Terraform-> Matchbox
         \-> Chef-> Docker

```

My colleague has a setup to standup VMs in AWS:

```
Terraform-> Packer-> Chef-> Docker

```

Is Packer to AWS as Matchbox is to Baremetal, in terms of image provisioning? I know there not 1:1 in features since packer can provision chef and stuff (it seems, and so can terraform) but for the general purpose of standing up a machine can this be considered.

If not what is the difference between the two. I am having trouble understanding exactly what packer is for (compared to matchbox), if it doesn't fit that relation.

# Answer
> 5 votes
You may use matchbox on a VM as you do on a baremetal machine, you won't be able to use packer on a baremetal machine ont he other hand as it doesn't handle any PXE boot option.

That said, leveraging vSphere/AWS/ API/cli to create a new machine from a template is usually quicker and more effective than using the API/cli to create the VM and then make it boot in PXE.

Packer will create machines, matchbox won't, it will allow you to match a profile with a machine but it won't rack your machine nor plug the networks and power cables, it won't do the machine creation neither for a VM, that not its goal.

So the main difference is the "before provisioning (chef)" step, as the machine creation, Packer will handle the machine creation whereas matchbox won't, matchbox will handle booting a blank machine and push the OS where packer usually won't.

---
Tags: amazon-web-services, chef, terraform, packer, provisioning
---


title: Docker on Development Workstations

text:
# Question
Title: Docker on Development Workstations
I recently had an interesting discussion with a colleague. We have different perspectives on what is required to develop Docker containers. I won't say who believes what, to keep from prejudicing the responses. Here are two competing assertions:

**Assertion 1**:
It is *not* necessary to install Docker on development workstations. Container build is a CI task that can be performed by a CI tool, then containers can be troubleshot/debugged in a DEV environent.

**Assertion 2**
It is *critical* for developers to run Docker locally so they can smoke test and execute containerized processes *before* they are committed to SCM or CI. Containerization is a core developer skill, which is specialized by development platform.

**Is it critical, nice-to-have, or not necessary for Docker to be installed on developer workstations where containerized workloads are developed?**

# Answer
> 3 votes
Someone said that to each question, there are always *three* answers. So here's your third answer.

Install docker on the dev machine, and build images with it.

*But:*

The images should *not* contain your source code or *anything* that changes during the usual edit-build-test-run cycle. The images only need to contain the basic stuff that *does* the build/test/run parts.

That is, if you were to develop a Maven application, your image/container would contain the `mvn` command with all its depdendencies. A second image/container would contain your `wildfly` or whatever you are using as an application server.

If you were to develop in ruby on rails, you would have one image/container which contains everything to be able to run `bundle exec rake cucumber`, another image/container which contains everything to be able to run `rails s` or `rails c`.

The images will be recreated seldomly, for example if you want to upgrade the application server, and they will certainly not be `docker build`'t on each run by the dev. Both images would *not* contain anything that belongs to your application. No source code, no `.war` file, no precompiled assets or application-specific gems for the ruby application. These would be mounted as volume when starting the containers, and point directly to whatever local directory the dev is currently working with.

This way, it is guaranteed that your dev uses the exact same base environment as your CI or production. New devs can start building/testing/running instantly and never need to install anything to their host machine. They can use the editor of their choice (if they wish to use something like Eclipse which has lots of magic and wants to compile the stuff itself, then they are free to do so of course - as long as they do at least a cursory check whether everything still works in the docker variant, before pushing).

And if you wish to deploy your application as a fully self-contained docker image, finally, then you are free to do so. Your CI server can build those images. It should use the smaller images used by your devs as base image, obviously.

This gives you a nice medium ground which should cover everything.

# Answer
> 2 votes
Assertion 1:

* Pro: The build environment is guaranteed to be what's expected
* Cons: It slow down validation of patches.

Assertion 2: Exactly the opposites pro and cons.

I'd advocate to let developers have docker on their system to iterate quickly on fixes or advancement and keeping regression and integration tests in the CI pipeline. That's not required, every dev should be able to choose if they just code and never run anything locally or if they want to run whole test suite on their workstation.

So both assertions are valid, I see no valid reasons to enforce one or the other

# Answer
> 2 votes
Being a fan of The Twelve-Factor App, I'll recommend installing Docker on every workstation, following rule *X. "Dev/prod parity*: Keep development, staging, and production as similar as possible".

I encourage that every developer is responsible to produce production-ready software, so you need to test it locally before pushing it. CI is for integration as its name suggests, not to check any individual contribution, but to check any conflict between the contribution of several developers.

# Answer
> 0 votes
IMHO it is at least a nice-to-have if not critical. But if critical it's not for the reason stated in assertion #2.

The ability to exactly replicate the containerized build from the CI can be critical to reproduce and debug some issues found in the CI verifications, then develop and test fixes for them.

Such ability can also be useful to *optionally* run parts of or the entire CI verification prior to committing the code for potentially high-risk changes.

And, in general, docker on the dev workstations can contribute overall to reducing the differences between the development and QA/staging/production environments.

But running CI-class verifications prior to commit can also be done on shared resources or on resources temporarily borrowed from the CI system, not necessarily on the developer's workstation, which in some cases can even be prohibitive - some CI builds can require much more resources than those available on the typical developer workstations.

But when having such ability it can be very tempting to actually *require* developers to perform such verifications in an attempt to completely eliminate regressions in the real CI executions, which is, I believe, implied in assertion #2. But that would IMHO be sub-optimal - such verifications are still done in isolation and thus cannot completely eliminate CI failures (see How can successfully pre-verified changes cause regressions which should have been caught?), while being expensive and chewing developer time for stuff that can be entirely automated.

If eliminating regressions in the real CI executions is desired then the precommit verifications have to be coordinated, ideally centralized, automated and immediately followed by automated commits.

---
Tags: docker, continuous-integration
---