# Question
Title: VM Hosts: Single Large Host or Few Smaller Hosts?
First, I apologize if this is the wrong forum to ask this question. I was wondering if this is a ServerStack or DevOps question. In either case here it goes:

I am building a small software shop and I am trying to figure out the infrastructure and machines required for it.

I bought MSDN Pro Subscription ($600/year) as well as JetBrains All Product Subscription ($600) for the development tools, desktop and server OS licenses, etc etc...

At the moment I have a budget of roughly $1500 (Walmart Credit Card) and I am entertaining the idea of buying 3x refurbished machines (each with Core i7-3770 cpu and 16GB of RAM) at $452 a pop or buying a large server at $1120 and maxing out it's RAM capacity for another $300-$400.

The background info on this is that I want to develop a product and have a full blown development environment (Source Control, Build/Deploy/Test and Release Management, Dev, Beta, and Prod environments). Cloud services (the cheapest) will run me about $200-$300 a month, and over period of 6 months it will become a recurring liability rather than an asset (which I pay out of pocket btw).

If I buy the hardware - it will have an initial cost of about $1500 but it will last me for the duration of the development process which is at least a year or so. Beyond that, if I outgrow it's capacity, I can move to docker containers and maybe a cloud solution for the Dev, Beta, and Prod environments.

So the question is - which is better - multiple dedicated cpu's or a single large server?

**1x Large VM Host (64gb RAM) to have the following VMS (from here) (4GB used by Hyper-V 2016 Server):**

* 1x - 8GB RAM - TFS - (Git, Active Directory, SQL2016, Sharepoint, IIS)
* 1x - 4GB RAM - TFS - Build Controller
* 1x - 4GB RAM - TFS - Release Agent, Test Agent
* 1x - 4GB RAM - DEV - Linux Mint (Postgre SQL)
* 1x - 4GB RAM - DEV - Linux Mint (Web Server/Apache)
* 1x - 4GB RAM - BETA - Linux Mint (Postgre SQL)
* 1x - 4GB RAM - BETA - Linux Mint (Web Server/Apache)
* 1x - 4GB RAM - PROD - Linux Mint (Postgre SQL)
* 1x - 4GB RAM - PROD - Linux Mint (Web Server/Apache)

**Or having multiple smaller hosts:(from here)**

1x Small Host - 16GB RAM (4GB used by Hyper-V 2016 Server):

* 1x - 6GB RAM - TFS - (Git, Active Directory, SQL2016, Sharepoint, IIS)
* 1x - 3GB RAM - TFS - Build Controller
* 1x - 3GB RAM - TFS - Release Agent, Test Agent

1x Small Host - 16GB RAM (4GB used by Hyper-V 2016 Server):

* 1x - 4GB RAM - DEV - Linux Mint (Postgre SQL)
* 1x - 4GB RAM - BETA - Linux Mint (Postgre SQL)
* 1x - 4GB RAM - PROD - Linux Mint (Postgre SQL)

1x Small Host - 16GB RAM (4GB used by Hyper-V 2016 Server):

* 1x - 4GB RAM - DEV - Linux Mint (Web Server/Apache)
* 1x - 4GB RAM - BETA - Linux Mint (Web Server/Apache)
* 1x - 4GB RAM - PROD - Linux Mint (Web Server/Apache)

# Answer
> 3 votes
A good rule of thumb is to try and avoid using one bit of logical kit (be it a single VM or a single physical host) for more than one thing.

If a physical server is going to be a VM host then that's all it should be. If a VM or a physical host is going to be a source code repository then that's all it should be etc. You can squeeze a bit extra out of your infrastructure if you consider everything to be a VM host and you then partition up those VMs to be single purpose hosts themselves.

The rationale behind this is that

1. A single host going down shouldn't impact your entire stack. Trust me this will hurt you badly as soon as you need downtime for an upgrade.
2. The cost to scale any individual component increases drastically if you want to try and scale on multiple similar servers (which will save you management dollars by reducing complexity) or it will cost you to scale out adhoc hardware as needed because you then need to deal with many different-sized hosts that are a pain to manage and require anyone managing them to know the platform intimately.

**recommendation:** split your infrastructure into as small a unit as is logical. If you can only afford to buy a single large bit of kit then make sure you split it into logically separate VMs or containers so that if you need to pop a bit of your infrastructure onto another server, or you need to scale stuff at different rates you can easily expand out but investing in a smaller upgrade.

Obviously the realities of software licencing, especially for windows OS's might make this cost prohibitive but, while I haven't caught up on windows licensing models in a while, I'm sure there are development centric licensing models that can suit your specific needs.

# Answer
> 1 votes
As I see it, the basic problem you have is that you aren't looking at the Total Cost of Ownership (TCO). While you may eventually save on the *hardware* costs, this is costing you in man-hours - either yours or your employees - a cost which you *also* pay out of pocket until you begin generating revenue.

The reason for this is this: Instead of using a ready to go system, you have to set up, install and configure the following:

* Hardware and RAID
* Version and source control
* DNS
* Firewall
* Operating system
* Database(es)
* Web server
* Build controller / test agent
* Backups

Then you have to deal with the operational maintenance of that as well. While *some* operational maintenance is required with in-house equipment, There is less required if you host. Consider:

Instead of sharing enterprise-grade infrastructure, you are buying white box home desktops from Walmart. This means you are accepting all of the liability and reliability issues that goes with that - drive failures, motherboards that fry, CPUs that overheat and toast, PSUs that give out and so forth - making your backups (which you don't even have to deal with if you host) that much more important. Odds are at those prices, these machines are built off of "seconds" - refurbished parts that have *already* failed once. There is a reason my organization purchases proper servers from Dell or HP - it's reliability.

This early in your software shop's life it is important to have *focus* on a *single* strategic vision - to produce great software (whatever software that is). In order to do that most effectively, you need to minimize distractions. You don't want to be in the hosting business too. It will distract you from making great software and that setup and management will cost you a great deal more than it would cost to simply have it hosted.

Figure out an estimate of how long it would take for you to set up and build what your hosting provider already has using the components you are looking at. Then multiply times your (or your employee's) hourly rate and factor that into your cost. At 300 a month, you are nearly halfway through your year. If your figured set up costs are greater than 1500 to 2000 dollars, then it is actually cheaper to host - plus less headache *and* you got a true year of development time instead of losing 2 weeks to a month (or more!) to setting up your environment.

After that year, you can still move in-house on the hardware - you were planning on possibly needing to migrate anyway.

---

But finally, to answer your question, use the three smaller hosts. You are clearly virtualizing anyway and this provides you redundancy. If your single large hosts crashes or reboots, it affects your entire system. If you have a problem with one instance, it affects 1/3rd of your environment and you might even be able to seamlessly migrate your VM to another host if your system is properly configured.

---
Tags: environments, servers
---


# Question
Title: Docker Sign an Image with an existing Certificate
With respect to docker's security system.

Is it possible to somehow get images signed by some entity that has trust from a company's CA-cert and then be able to find out who signed the image?

It seems like Docker Trusted Registry (DTR), and Notary in general, have a binary mode of trust: either an image is signed or it is not, but it is not qualified by who performed the signing.

# Answer
> 1 votes
This documentation indicates that it is possible to add collaborators by trusting their key and that it is also possible to revoke access.

Having said that, the answer to:

> either an image is signed or it is not, but it is not qualified by who performed the signing

that it is possible to check who signed the image if every collaborator creates their own key and each key is trusted individually.

As notary and dtr were already mentioned, an alternative is Portus. Portus is able to restrict access to docker images, e.g. some users are only allowed to view images, while others may pull them. It is possible to revoke access as well.

---
Tags: docker, security
---


# Question
Title: Jenkins DSL - Pipeline Job (Unexpected token '$')
I have the following DSL code and I´m getting error on line 13, `$class`. I guess than everything inside `script("""...""")` is parsed as plain text to the pipeline definition but I don´t know what´s wrong or how to escape the `$` character.

My code:

```
pipelineJob('DSL-Test') {

parameters {
    choiceParam ('version', ['1.2.1', '1.2.0'], 'Docker Version')
}

definition {
    cps {
        sandbox(boolean sandbox = true)
        script("""
            node ('master'){
                stage('Git Checkout'){
                    checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'mycrednetial', url: 'myurl']]])
                }
            }
        """.stripIndent())
         }
    }
}

```

This is the error I get:

> Processing provided DSL script ERROR: startup failed: script: 13: unexpected token: class @ line 13, column 24. checkout(\[$class: 'GitSCM'\]) ^
>
> 1 error
>
> Started calculate disk usage of build Finished Calculation of disk usage of build in 0 seconds Started calculate disk usage of workspace Finished Calculation of disk usage of workspace in 0 seconds Finished: FAILURE

Does anyone know how to make that character not display error?

# Answer
I have just managed how to escape characters.

The solution is to add: "\\" before the character.

```
checkout([\$class:....

```
> 0 votes

---
Tags: jenkins, job-dsl
---


# Question
Title: How to get memory usage per process with sar, sysstat?
Can I get memory usage per process with Linux? we monitor our servers with `sysstat`/`sar`. But besides seeing that memory went off the roof at some point, we can't pinpoint which process was getting bigger and bigger. is there a way with `sar` (or other tools) to get memory usage per process? and look at it, later on?

# Answer
As Tensibai mentioned, you can extract this info from the `/proc` filesystem, but in most cases you need to determine the trending yourself. There are several places which could be of interest:

* **/proc/\[pid\]/statm**

> ```
>           Provides information about memory usage, measured in pages.
>           The columns are:
>
>               size       (1) total program size
>                          (same as VmSize in /proc/[pid]/status)
>               resident   (2) resident set size
>                          (same as VmRSS in /proc/[pid]/status)
>               shared     (3) number of resident shared pages (i.e., backed by a file)
>                          (same as RssFile+RssShmem in /proc/[pid]/status)
>               text       (4) text (code)
>               lib        (5) library (unused since Linux 2.6; always 0)
>               data       (6) data + stack
>               dt         (7) dirty pages (unused since Linux 2.6; always 0)
>
> ```

```
cat /proc/31520/statm
1217567 835883 84912 29 0 955887 0

```
* memory-related fields in **/proc/\[pid\]/status** (notably `Vm*` and `Rss*`), might be preferable if you also collect other info from this file

> ```
>           * VmPeak: Peak virtual memory size.
>
>           * VmSize: Virtual memory size.
>
>           * VmLck: Locked memory size (see mlock(3)).
>
>           * VmPin: Pinned memory size (since Linux 3.2).  These are
>             pages that can't be moved because something needs to
>             directly access physical memory.
>
>           * VmHWM: Peak resident set size ("high water mark").
>
>           * VmRSS: Resident set size.  Note that the value here is the
>             sum of RssAnon, RssFile, and RssShmem.
>
>           * RssAnon: Size of resident anonymous memory.  (since Linux
>             4.5).
>
>           * RssFile: Size of resident file mappings.  (since Linux 4.5).
>
>           * RssShmem: Size of resident shared memory (includes System V
>             shared memory, mappings from tmpfs(5), and shared anonymous
>             mappings).  (since Linux 4.5).
>
>           * VmData, VmStk, VmExe: Size of data, stack, and text
>             segments.
>
>           * VmLib: Shared library code size.
>
>           * VmPTE: Page table entries size (since Linux 2.6.10).
>
>           * VmPMD: Size of second-level page tables (since Linux 4.0).
>
>           * VmSwap: Swapped-out virtual memory size by anonymous private
>             pages; shmem swap usage is not included (since Linux
>             2.6.34).
>
> ```

```
server:/> egrep '^(Vm|Rss)' /proc/31520/status
VmPeak:  6315376 kB
VmSize:  4870332 kB
VmLck:         0 kB
VmPin:         0 kB
VmHWM:   5009608 kB
VmRSS:   3344300 kB
VmData:  3822572 kB
VmStk:      1040 kB
VmExe:       116 kB
VmLib:    146736 kB
VmPTE:      8952 kB
VmSwap:        0 kB

```

Some processes can, through their behaviour and not through their actual memory footprint, contribute to the overall system memory starvation and eventual demise. So it might also be of interest to look at the OOM Killer related information, which already takes into account some trending information:

* **/proc/\[pid\]/oom\_score**

> ```
>           This file displays the current score that the kernel gives to
>           this process for the purpose of selecting a process for the
>           OOM-killer.  A higher score means that the process is more
>           likely to be selected by the OOM-killer.  The basis for this
>           score is the amount of memory used by the process, with
>           increases (+) or decreases (-) for factors including:
>
>           * whether the process creates a lot of children using fork(2)
>             (+);
>
>           * whether the process has been running a long time, or has
>             used a lot of CPU time (-);
>
>           * whether the process has a low nice value (i.e., > 0) (+);
>
>           * whether the process is privileged (-); and
>
>           * whether the process is making direct hardware access (-).
>
>           The oom_score also reflects the adjustment specified by the
>           oom_score_adj or oom_adj setting for the process.
>
> ```

```
server:/> cat proc/31520/oom_score
103

```
* **/proc/\[pid\]/oom\_score\_adj** (or its deprecated predecessor **/proc/\[pid\]/oom\_adj**, if need be)

> ```
>           This file can be used to adjust the badness heuristic used to
>           select which process gets killed in out-of-memory conditions.
>
>           The badness heuristic assigns a value to each candidate task
>           ranging from 0 (never kill) to 1000 (always kill) to determine
>           which process is targeted.  The units are roughly a proportion
>           along that range of allowed memory the process may allocate
>           from, based on an estimation of its current memory and swap
>           use.  For example, if a task is using all allowed memory, its
>           badness score will be 1000.  If it is using half of its
>           allowed memory, its score will be 500.
>
>           There is an additional factor included in the badness score:
>           root processes are given 3% extra memory over other tasks.
>
>           The amount of "allowed" memory depends on the context in which
>           the OOM-killer was called.  If it is due to the memory
>           assigned to the allocating task's cpuset being exhausted, the
>           allowed memory represents the set of mems assigned to that
>           cpuset (see cpuset(7)).  If it is due to a mempolicy's node(s)
>           being exhausted, the allowed memory represents the set of
>           mempolicy nodes.  If it is due to a memory limit (or swap
>           limit) being reached, the allowed memory is that configured
>           limit.  Finally, if it is due to the entire system being out
>           of memory, the allowed memory represents all allocatable
>           resources.
>
>           The value of oom_score_adj is added to the badness score
>           before it is used to determine which task to kill.  Acceptable
>           values range from -1000 (OOM_SCORE_ADJ_MIN) to +1000
>           (OOM_SCORE_ADJ_MAX).  This allows user space to control the
>           preference for OOM-killing, ranging from always preferring a
>           certain task or completely disabling it from OOM killing.  The
>           lowest possible value, -1000, is equivalent to disabling OOM-
>           killing entirely for that task, since it will always report a
>           badness score of 0.
>
>           Consequently, it is very simple for user space to define the
>           amount of memory to consider for each task.  Setting an
>           oom_score_adj value of +500, for example, is roughly
>           equivalent to allowing the remainder of tasks sharing the same
>           system, cpuset, mempolicy, or memory controller resources to
>           use at least 50% more memory.  A value of -500, on the other
>           hand, would be roughly equivalent to discounting 50% of the
>           task's allowed memory from being considered as scoring against
>           the task.
>
>           For backward compatibility with previous kernels,
>           /proc/[pid]/oom_adj can still be used to tune the badness
>           score.  Its value is scaled linearly with oom_score_adj.
>
>           Writing to /proc/[pid]/oom_score_adj or /proc/[pid]/oom_adj
>           will change the other with its scaled value.
>
> ```

```
server:/> cat proc/31520/oom_score_adj
0

```
> 5 votes

# Answer
Is it a hard requirement to use only `sar` and `sysstat`? If not you might want to look at collectl or collectd. These will enable you to study memory usage over time on a per-process granularity. It's really not worth wasting your time writing your own scripts to parse `/proc` as the other answer suggests; this is a solved problem.
> 4 votes

---
Tags: monitoring, linux, logging
---


# Question
Title: Why are books so prevalent in the DevOps community?
I have seen quite a few of the blogs I follow recommending more and more books over time.

I enjoy reading fiction and have no aversion to books but where a blogpost can be updated/rewritten when tech moves on these books which are normally ~£20-30 cannot.

Is there a particular quality to DevOps related titles which is missing from the online world or is everyone but me nuts?

# Answer
> 16 votes
In most cases, the recommended books are not about technology. While technology changes, the fundamental principles behind organizations like system thinking, leadership, common sense, etc... do not change as often.

Books such as *The Goal*, and even *The DevOps Handbook* do not mention much technology on their pages but rather ways to manage work being performed by people.

Many problems are technology related, topics such as Microservices, Architecting large-scale systems, Infrastructure as Code, etc... these don't talk about a specific tool and/or technology but rather about an architectural topic. A field of knowledge that people who build large systems need to know in order to build the system correctly. This knowledge is rare, and its great the books are written about these subjects - just disregard the tools mentioned, or translate into their new-er reincarnation.

One of the better books about creating quality software (imho) is *Agile Software Development, Principles, Patterns, and Practices*. And while the language used in this book (Java) has moved along quite a bit, the examples provided in the book are timeless and can be easily translated into any other language of choice.

Some of the problems the DevOps movement tries to solve are involved with common ways work is managed in organizations that just doesn't make any sense. As Eliyahu Goldratt often said (author of *The Goal*) "Common sense is not very common".

These books teach principles of thinking correctly about problems and human relationships in a system setting so that the whole system is improved. The lessons are old, and unfortunately, only rarely there are people who work in the field who actually learned them.

Naturally, there are also authors who wrote books about such and such fizz-bang tech tool that is new and relevant to the field, like AWS or Docker or Jenkins or whatever and just want to push their book sales ... but I try and exclude these kinds of blog posts from my answer.

# Answer
> 4 votes
This is a sign of the growing maturity of infrastructure engineering as a field or a profession. If you consider any of the more traditional forms of engineering such as mechanical, civil or electrical, the bulk of the knowledge is is paper book form, that is how it is taught, practicing engineers consult reference books. That is because once the underlying principles are understood and codified, the details of implementation are specific only to a particular application or installation. You can consider any engineering artefact - a skyscraper or bridge, a jet engine, an aircraft carrier. Enormously sophisticated, requiring great skill to construct, but built using general principles that now they are understood, change only over the course of decades, and would be readily understandable to an engineer from decades ago.

Making it more DevOps specific - it really doesn't matter if you implement configuration management with CFEngine, Chef, Puppet or anything else, the principles of configuration management are well understood enough now that they can be written down and applied to any actual tool.

---
Tags: culture
---


# Question
Title: How to start a service in fedora container without using systemd?
I want to start a service in my fedora docker container. But since systemd is not working in the fedora container. I am unable to start the service. I'm getting following error.

> Failed to connect to bus: No such file or directory

In Ubuntu container, I am able to start any service using /etc/init.d/. I want to know any other way to do the same for my fedora container.

# Answer
Running systemd inside the container should not be required as docker run eventually does the same thing. Make changes in your run scripts to make them independant from systemd. If you still insist on using systemd and do not care about how docker works, there is a way to let systemd work inside the container. Follow this link:

https://developers.redhat.com/blog/2014/05/05/running-systemd-within-docker-container/
> 1 votes

---
Tags: docker, dockerfile, containers
---


title: Running the docker daemon without root

text:
# Question
Title: Running the docker daemon without root
Inspired by What are best and comprehensive practices to consider when running docker in production? , I stumbled over Why we don't let non-root users run Docker...

They came up with `docker run -ti --privileged -v /:/host fedora chroot /host` , which puts your unprivileged (but `docker`-grouped) user in a root shell on the host (not in the container).

No need to mention, that is rather bad news for production systems.

What can be done to avoid this or similar things? I assume the docker daemon cannot be run as a non-root user (or else that would likely be the default way to start it)? One solution that comes to mind is not putting unprivileged users in the `docker`group and only allowing specific docker command lines via `sudoers`. But is that enough?

Do we know how hard it is to break out of the container and gain a host `root` shell like in the example given, if the image is ran as usual (i.e. `docker run --rm -it myimage`)? I am not even talking about some attack via the service running in the container, but by, say, a specially prepared image.

EDIT: to clarify, I am specifically interested in this attack scenario:

* Untrusted software developer delivers a ready-to-run image.
* Trusted user runs that image on a production system in a normal fashion (i.e., `docker --rm -it myimage`), without `--privileged`.
    + Possibly with sub-scenarios of `-u unprivileged_u` or not.
* Does the untrusted person that created the image have a reasonable chance to break out of the docker container runtime? I.e., more than with, say, classical `chroot` or classical permission elevation?

EDIT2: My research shows only a single real breakout incident which was fixed with Docker Engine 0.12, and then again in 1.0.1. Also, all things I found that are critical about this mention the erstwhile TCP/IP interface of the docker daemon, which was changed to the Unix socket for a long time now. Is there anything else, or would you say that there is no problem as long as the person typing/running the actual `docker` command is good?

# Answer
Compared to just adding the user to the docker group the second links solution is not any more secure.

Note how that page still can launch with the `--privileged` flag, and that it is running unconfined.

`unconfined_u:unconfined_r:unconfined_t`

This means that the container can access all resources including the hosts disks and hardware. It is security through obscurity, which is not security. Anyone who knows how to use `mknod` or walk the /sys and /proc trees could easily compromise the host and all containers with zero logging.

The reality is that docker shifts both complexity and the security boundaries. All users who can launch container or hit the API need to be restricted to trusted users in that security context.

`--privileged` disables all apparmor and selinux policies which is actually far less secure than a native package.

Namespaces are not a security function and depend on apparmor and selinux to enforce reasonable constraints.

Docker notes this on this page.

https://docs.docker.com/engine/security/security/#control-groups

'First of all, only trusted users should be allowed to control your Docker daemon.'

The security functions of docker/kube are not administrative boundaries like classical unix permissions, they are tools to prevent non-privileged containers from breaking out.

In a docker world the administrative boundary becomes the host, and the selection and segmentation required to isolate applications or users within that context needs to be applied at that boundary.

The benefits of this shift in complexity and responsibility generally outweigh the risks if implemented with those changes in mind.

TLDR

Docker API user == Sudo ALL user

Running a container with the --privlaged flag == running a web service as suid or as the root user.

**Edited per the OP's request for additional information**

The referenced issue with breakout int he OP's edit was an non **uid0** privilege escalation.

Unfortunately, due to the need to perform root only actions Docker needs to enable some capabilities so that apt/dnf can install packages etc...

This need does pose a risk if production workloads are run in this default configuration and one should adopt the security principle of least privilege for production workloads.

`--privileged` disables *apparmor/selinux* and opens up *capabilities*

I am using ubuntu but it may be useful to work through the following steps. First start a default container with *docker run -i --rm -t debian bash*

From the parent host find the PID for bash using *ps* and note that the process is owned *root*. If you look in `/proc/$PID/status` you will see the contexts it is running under.

```
# egrep '^Cap(Prm|Inh|Eff)' /proc/16026/status
 CapInh:    00000000a80425fb
 CapPrm:    00000000a80425fb
 CapEff:    00000000a80425fb

```

You will want to refer to *man 7 man capabilities* and */usr/include/linux/capability.h* for better info but a summer is

CapInh = The Inherited capabilities (what docker provided) CapPrm = The capabilities due to permissions (inside the container OS) CapEff = The effective capabilities

You can decode these to human readable form by running:

```
$ capsh --decode=00000000a80425fb

```

Now do the same with `$ docker run -i --rm --privileged -t debian bash` and you will find that the effective capabilities are `0000003fffffffff`

Also just do a `dir /dev` in both VM's and you will see just how much access a privileged container has.

By looking at `/etc/apparmor.d/docker` for apparmor systems or the lables in SElinux you will see the implications.

Going back to the principle of least privileges, I would ensure that my docker container is running process as a unprivileged user and with as few enabled capabilities as possible.

As an example, you can test this by running it first by dropping all caps.

```
$ docker run -i --rm -t --cap-drop=all -t debian bash

```

This can be validated through the `/proc/$PID/status` method above.

```
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000

```

Note it is all zeros, from CapInh. Then if I had to enable features for the application I would use `--cap-add` *after* the `--cap-drop=all`

Example, `--cap-drop=all` will break ping:

```
# ping -c 1 www.google.com
ping: Lacking privilege for raw socket.

```

So we can add the `cap_net_raw` cap. Docker expects the arguments with the `cap_` prefix removed so the command is `docker run -i --rm -t --cap-drop=all --cap-add=net_raw -t debian bash`

```
# ping -c 1 www.google.com
PING www.google.com (216.58.216.164): 56 data bytes
64 bytes from 216.58.216.164: icmp_seq=0 ttl=54 time=4.779 ms

```

But be very careful adding them as an example.

cap\_sys\_module will allow a container to add or remove kernel modules from the parent host.

cap\_sys\_rawio will open memory and *all* block devices to attack

cap\_sys\_admin is super dangerous.

So in this case I would see if you can make things run in this context.

$ docker run -i --rm -t --cap-drop=all -u nobody:nogroup -t debian bash

Hopefully the robustness of apparmor and selinux profiles improves over time, but if that is not enough for your security needs you can look in there too.

Really, avoiding the *--privileged* flag and using principle of least privilege will make the most impact. Especially if you take advantage of the ephemeral nature of containers to keep packages up to date.

If you need more Red Hat covers some basic seccomp options here.

https://access.redhat.com/documentation/en-us/red\_hat\_enterprise\_linux\_atomic\_host/7/html/container\_security\_guide/linux\_capabilities\_and\_seccomp
> 7 votes

---
Tags: security
---


# Question
Title: Does Spinnaker's rosco use HashiCorp's packer or is it another tool?
I am confused. Does Spinnaker use HashiCorp's Packer for baking or is it another tool? The reason for asking this that a folder called packer resides in the github repo of rosco https://github.com/spinnaker/rosco/blob/master/rosco-web/config/packer/docker.json

# Answer
> 2 votes
From spinnaker own documentation

> Rosco is the bakery. It is used to produce machine images (e.g. GCE images, AWS AMIs, Azure VM images). It presently wraps packer, but will shortly be expanded to support additional mechanisms for producing images.

So yes, rosco uses HashiCorp's packer.

The main reason to use packer is that you just need to change the provider used to create a docker image or an AWS ami or an Azure image and keep your building step identical.

---
Tags: cicd
---


# Question
Title: What are reasons for using HashiCorp's packer to build docker images instead of using docker build?
I have read https://www.packer.io/docs/builders/docker.html, but I do not see the advantage of using Packer over docker build or docker-compose up --build for building docker images

# Answer
Mainly the reason is to keep your image building steps intact if you get to move from docker to another image building system.

Packer does support a bunch of providers (builders in packer terminology), and changing the target "container" is just a matter of changing the builder (or using multiple builders in the same packer file), the build steps (provisionner step) are kept intact and will be the same if you build a docker image or an AWS ami for example (you can even build both at the same time).
> 9 votes

---
Tags: docker, packer
---


# Question
Title: When to put all containers in a pod and when to deploy them individually?
One of the reasons for asking this question is that it seems to be impossible to access a container using kubectl exec when these are deployed in a pod.

# Answer
Let's start from the beginning.

First, I'll answer the original question, the one in the title.

## When to put all containers in a pod and when to deploy them individually?

This depends on what your app does. For instance (this is an actual example from the company I work for), for a web application, you may have a pod which runs three containers:

* app: the actual app with its code
* nginx: serves all the requests
* heka: a log service which proxies the logs to wherever you want them

The reason for grouping this three containers within a single pod is simple: a pod assures you the containers run in the same Kubernetes node, this is to have the least possible latency with requests being interchanged along the three containers.

That being said, this is only an example, you could have different and multiple reasons for grouping containers in a pod, but I think this is the most common.

With that being clarified, I follow to the question that you present in the body of your post.

## How do you access a single container within a pod?

This is simply using kubectl, I use the command like this:

`kubectl exec -it <pod-name> -c=<container-name> -- /bin/bash`

Suppose we want to access the `nginx` container inside our `my-app` pod; we would type this:

`kubectl exec -it my-app -c=nginx -- /bin/bash`

If you don't specify the container with the `-c` flag, it defaults to the first container that is in the Kubernetes `yaml` (I think). So when you say it was impossible to access a container, you actually ***did*** access a container (the default one), because a pod cannot be accessed, it is merely a collection of containers and its configurations.

Hope I've been helpful.
> 2 votes

# Answer
You can still exec containers when multiple are running in a pod, you specify the container to execute in using the "-c" parameter.
> 1 votes

---
Tags: kubernetes
---


# Question
Title: How to Versionize Ansible (Tower) with GitHUB
I have configured Ansible Tower with Centos 7.2 and I am able to see my Tower dashboard. My major concern is how to do versioninzation in Ansible Tower as given in this link with Github.

Since I have written the playbook for configuration changes in my Centos server and push my changes to GITHUB in a branch as `SAAS_1` now that I have to integrate this changes to Ansible Tower, how can I see this versioning? How can I visually see all of this content in Ansible Tower?

Like production -1.1 UFT-2.1 and continuation based on changes enabled

# Answer
one strategy is to put your playbook repo(s) thru a regular build/deploy pipeline, where you separate between "nonrelease" and "release", for example by linking the master branch to "release" (most CI tools support this out of the box). Then you'd have two "streams" of artifacts you could deploy, one for release and one for non-release (or release candidate or whatever you'd like to call it). These can be auto-deployed to different folders inside Ansible tower and then picked up as different "projects". That way youd have two "instances" of every playbook folder (or "project" as Tower calls them).
> 2 votes

---
Tags: ansible, github
---


# Question
Title: What is Titus and what issues does it solve?
I read an article from Medium Titled "The Evolution of Container Usage at Netflix" and watched a couple of videos on the topic, but it is unclear to me why Titus (referenced in these) should be used. Is it different than Kubernetes or should it be used in concert with Kubernetes?

# Answer
It's different to Kubernetes in that it seeks to leverage existing AWS services instead of bolting on top. Kubernetes supports a number of cloud providers and contains a number of components that exist in AWS already.

Although we're starting to see better kubernetes AWS integration and with AWS joining the cloud native foundation this is only going to get stronger.

Titus seems to be an alternative to Kubernetes but its not available yet. My bet is AWS will strive to better integrate K8 and likely Titus won't take off even when it is released.
> 4 votes

# Answer
Titus is been kept in the dark, for now, I guess mostly because we are kind of used to see these cool tools from Netflix in the OSS scene, not the case for Titus, yet. If you want to know more about what Netflix's plan is for Titus, I think this is worth reading.
> 0 votes

---
Tags: terminology, titus
---


# Question
Title: Mastering CI infrastructure changes with GitLab
With Bamboo, a best practice for infrastructure management of agents seems to be using Docker: any time you need a new version of compiler or test tool, the path to deploy a modified version of an agent is very fast.

How could this work with GitLab?

# Answer
> 3 votes
Gitlab CI supports running all of your builds inside Docker containers so you have a clean build environment every time. I'd recommend starting with the basic Docker executor but there is also an opportunity to autoscale your runners using Docker Machine or to execute the runners on a Kubernetes pod.

If you need to use Docker as part of your build (such as building, tagging and pushing images) then you can either use Docker in Docker or, preferably, bind mount the Docker socket of the host into your runner allowing it to run Docker containers as a sibling rather than child process. Note that the original author of DIND recommends using bind mounting instead of DIND, especially for CI.

To bind mount the Docker socket register your runner with the following command:

```
sudo gitlab-runner register -n \
  --url https://gitlab.com/ \
  --registration-token REGISTRATION_TOKEN \
  --executor docker \
  --description "My Docker Runner" \
  --docker-image "docker:latest" \
  --docker-volumes /var/run/docker.sock:/var/run/docker.sock

```

And then in your `.gitlab-ci.yml` file you can define your job as so:

```
image: docker:latest

before_script:
- docker info

build:
  stage: build
  script:
  - docker build -t my-docker-image .
  - docker run my-docker-image /script/to/run/tests

```

Using just the basic `docker` image rather than the `docker:dind` image

# Answer
> 1 votes
I prefer DIND as every run start with a clean sheet. Otherwise the docker image will be built on the gitlab host itself.

* installation of gitlab runner
* registration of runner

> ```
> sudo gitlab-ci-multi-runner register -n \
>   --url https://gitlab.com/ \
>   --registration-token REGISTRATION_TOKEN \
>   --executor docker \
>   --description "My Docker Runner" \
>   --docker-image "docker:latest" \
>   --docker-privileged
>
> ```

* commit and push .gitlab-ci.yml

> ```
> image: docker:latest
>
> # When using dind, it's wise to use the overlayfs driver for
> # improved performance.
> variables:
>   DOCKER_DRIVER: overlay2
>
> services:
> - docker:dind
>
> before_script:
> - docker info
>
> build:
>   stage: build
>   script:
>   - docker build -t my-docker-image .
>   - docker run my-docker-image /script/to/run/tests
>
> ```

---
Tags: docker, continuous-integration, gitlab
---


# Question
Title: Docker daemon error on joining swarm node - could not find local IP address - unknown port
Consider the following setup:

Three hosts in the same network:

* Host A - SLES
* Host B - Ubuntu
* Host C - Ubuntu

Then, the following has worked:

* Init swarm on A
* Join swarm at B, make B a manager

The following is however not working:

* Join swarm at C as worker with token issued from A
* Join swarm at C as worker with token issued from B

In both cases we get the same error:

```
Error response from daemon: could not find local IP address:
dial udp: address udp/2377: unknown port

```

Honestly, I have even no idea how to interpret this message.

Any ideas?..

UPD. nmap results for ports 2377,4789,7946 (hint from Felipe in a comment below) - no idea what are they good for but now it is clear that something is wrong with host C.

**Host A** (Primary manager)

```
PORT     STATE  SERVICE
2377/tcp open   unknown
4789/tcp closed unknown
7946/tcp open   unknown

```

**Host B** (Secondary manager)

```
PORT     STATE  SERVICE
2377/tcp open   unknown
4789/tcp closed unknown
7946/tcp open   unknown

```

**Host C** (should become a worker)

```
PORT     STATE  SERVICE
2377/tcp closed unknown
4789/tcp closed unknown
7946/tcp closed unknown

```

**Host D** (successfully added as a worker)

```
PORT     STATE  SERVICE
2377/tcp closed unknown
4789/tcp closed unknown
7946/tcp open   unknown

```

Finally, with host B there is the actual problem.

* swarm init =\> OK; port 2377 is open
* swarm leave =\> port is closed again
* swarm join =\> error "port not found"

# Answer
> 1 votes
Could you please try the commands above between all hosts ?

INSIDE HOST A:

```
nc -zv HOST_B_IP_ADDRESS 2377,4789,7946
nc -zv HOST_C_IP_ADDRESS 2377,4789,7946

```

INSIDE HOST B:

```
nc -zv HOST_A_IP_ADDRESS 2377,4789,7946
nc -zv HOST_C_IP_ADDRESS 2377,4789,7946

```

INSIDE HOST C:

```
nc -zv HOST_A_IP_ADDRESS 2377,4789,7946
nc -zv HOST_B_IP_ADDRESS 2377,4789,7946

```

As explained in https://docs.docker.com/engine/swarm/swarm-tutorial/#open-protocols-and-ports-between-the-hosts, some ports must be available between hosts in order for Docker Swarm to work correctly.

The error reported sounds like a port related problem. Maybe is there another application in one of these hosts already using one of the ports needed by Docker ? Maybe a firewall configuration ?

---
Tags: docker, docker-swarm, networking
---


title: How to create ansible host file on mac?

text:
# Question
Title: How to create ansible host file on mac?
When I try to create ansible host file as follows:

```
sudo vi /usr/local/etc/ansible/hosts

```

When I try to save the file I get following error:

```
"/usr/local/etc/ansible/hosts" E212: Can't open file for writing

```

# Answer
Run:

```
$ sudo touch /usr/local/etc/ansible/hosts
$ sudo vi /usr/local/etc/ansible/hosts

```

To create a file and then open it.
> 3 votes

# Answer
The issue was that the host file did not exist. So, first I created the host file at the location: /usr/local/etc/ansible/hosts Then after that i could add the hosts to this file.
> 0 votes

---
Tags: ansible, amazon-ec2
---


# Question
Title: Start dev with purely docker, or something like vagrant (in a to-be-dockerized application)?
I have a Ruby on Rails application that has been around for a while, and implemented completely classical, so far. Everything is fine; but since the number of environments we are deploying to (testing, QA, several production envs) have grown, I decided to dockerize the thing.

A side effect should obviously be that new developers get on board quicker.

At the moment, I have made a Dockerfile to build the application and run it; I am happy, everything is fine. I am now pondering how to move on:

* Either make it so that the tests can be run by a simple docker command as well - so that the daily development loop simply consists of edit-rebuild-run-tests (or, for that loop, we'd probably rather just mount/bind the sources that are actually being edited by the dev outside of the container, so we save the explicit "rebuild". Nothing needs to be precompiled etc. for running the tests anyways.).
* Or use something like Vagrant to streamline the development environments, and don't really use docker for the actual edit-test cycle.

I am strongly tending to the first option (i.e., KISS), but I have little experience working with this. I can't really speak on the second as I have no real experience with any of those tools like Vagrant that focus on the developers perspective.

The other developers have no particular experience with any of these tools, so whatever I pick/suggest should better be pretty non-invasive. Everyone has their happy, well-configured VM to develop in, so far. We don't really encountered any problems with developer VMs diverging, or something like that (this being Ruby, after all).

**Questions:** is there anything wrong with going for my first scenario? Are there *huge* benefits introducing something else aside of docker at this very early stage of our team digging into DevOps tools?

# Answer
Vagrant, in this case, would simply add a lot of unnecessary overhead into your testing environment. You can create containers that quickly test specific groups of functionalities and paste the results somewhere. Your team won't spend much time learning how to start these containers as it is usually a simply command. Most of changes are going to be done inside the code, not into the testing environment. But if you need to change the test environment, the Dockerfiles must be in an easy to access place. You could also automate the build process of containers linking the Dockerfile (e.g. from a GitHub repository) into the DockerHub or Store. I can't see a reason here for not going into Docker. This knowledge will be more usefull for your team than Vagrant skills.

You can read more about it here: https://stackoverflow.com/questions/16647069/should-i-use-vagrant-or-docker-for-creating-an-isolated-environment
> 1 votes

---
Tags: docker, vagrant, learning
---


title: What is Chef Automate

text:
# Question
Title: What is Chef Automate
I've noticed that now most of the Supermarket's recipes are using Chef Delivery (aka Chef Automate). There is little information about it on their site, and it looks for me like it is a "pig in a poke": first you pay and they you know what you've paid for.

1. Does Chef Automate require you any additional server, besides Chef Server (we already have it). I guess yes.
2. Is it required at all to write cookbooks now?
3. Will it replace Foodcritic, ServerSpec and Kitchen eventually? Also, I see they now offer and "InSpec" which seems to be similar to ServerSpec.

# Answer
For the whole points and to try things there's https://learn.chef.io which allow you to test automate also.

Mainly chef automate is the next iteration bringing together 4 commercial products from Chef: chef manage (UI), chef reporting, chef compliance and delivery (CI/CD) with the addition of push jobs which was open sourced a little before.

Inspec is another new tool for compliance, it is based on serverspec but not extending it not using its code base.

Automate is not mandatory to author cookbooks at all, it is just a convenient way to display back status of nodes either on configuration point of view or compliance point of view (see the audit cookbook for inspec usage within a run), you can still does everything on your nodes on the free open source model, you'll have to handle the feedback loop yourself, either through a handler or any other way of your choice as it was already the case.

The CI/CD part of Automate doesn't replace any tool, it's more another approach to CI/CD pipelines, so you may use it to do the usual linting (foodcritic/cookstyle), test (chefspec), and then upload to your chef-server for use. It can handle deploys through the push jobs feature and after review and approval of changes.

Presentation of automate and the related documentation (workflow is the first concept, others are accessible from the menu on the left)
> 7 votes

---
Tags: automation, chef
---


title: Organizing Chef cookbooks

text:
# Question
Title: Organizing Chef cookbooks
Is it 100% required to store all the cookbooks under `chef-repo`? This seems a little bit inconvenient for me, I'd prefer to store solution cookbooks in the application codebase, or just as a standalone git repos.

I know that tools like `knife` rely on configuration files to be present in `.chef`, and it frustrates me a little bit too.

# Answer
No, it is not, the chef-repo is the legacy way back to chef 10.

Knife allows working on multiples directory with the -o option. Berks work from current directory.

You can also work around the default .chef directory by using the `KNIFE_HOME` environment variable too.

The current recomendations is indeed a repo per cookbook (or alongside another app)
> 5 votes

---
Tags: chef, repository
---


title: Defined start sequence of containers

text:
# Question
Title: Defined start sequence of containers
AFAIK you cannot influence the order how Docker Swarm starts containers for services.

Is that possible with:

* Mesos?
* K8s (Kubernetes)?
* Rancher?
* Other?

This question is not about specific orchestration of dependent services but the problem that if you run 10 services each with one JVM this creates a load peak. The system slows down and you need complex loops to wait for other services. Are there any mechanisms known in other orchestration concepts to manage the startup somehow?

For example, in completely other domain, in JMeter you can tell a timeframe in which a thread group will start. So could I say "try to start this service with some offset"?

# Answer
https://forums.rancher.com/t/start-order-of-stack-containers/3106/9

> We do not support depends\_on, and neither does Docker in Swarm mode. It is not a real solution to the problem anyway and leaves you with unhandled pointy-edge cases when failures occur and containers are being replaced. Your services should know how to either wait for their dependencies or exit and be rescheduled to retry.

I agree with this comment. Services should always be able to start. Imagine that one service goes down then another service will not work anymore.
> 4 votes

---
Tags: containers, cloud, orchestration
---


# Question
Title: If I set up a Travis CI agent, will it be independent of Travis's existing infrastructure?
Is this correct that if I decide to setup an own TravisCI agent, I will be enabled to be independent of the existing distributed Travis computation network?

# Answer
> 1 votes
Travis CI doesn't offer running your own workers, you use their infrastructure unless you mean the on-prem Travis Enterprise. You might be thinking of GitLab CI?

# Answer
> 0 votes
Travis CI is build as a cloud solution focusing upon github repositories. It is possible to setup a Travis CI in-house as a self-hosted solution, however the open-source project is not build with this in mind and therefore it is likely that you will hit limitations. This is why Travis CI has an Enterprise solution: https://enterprise.travis-ci.com/. I think you would be better off looking at similar CI products or buying the enterprise edition of Travis which are focused upon you owning both the master server and the build agents. However, it depends on your specific need.

---
Tags: continuous-integration, travis-ci
---


# Question
Title: How to start the latest version of k8s using minikube?
When a k8s cluster is created using minikube, the output indicates that an outdated version of k8s is deployed:

```
user@host ~ $ minikube start
Starting local Kubernetes v1.6.4 cluster...
Starting VM...

```

# Answer
> 1 votes
https://rominirani.com/tutorial-getting-started-with-kubernetes-on-your-windows-laptop-with-minikube-3269b54a226

```
user@host ~ $ minikube start --kubernetes-version="v1.7.5"
Starting local Kubernetes v1.7.5 cluster...
Starting VM...

```

---
Tags: kubernetes, minikube
---


# Question
Title: How to pass dynamic properties to an MSbuild task?
Is it possible to pass a dynamic property group to an MSbuild task? So if I have the following properties and values:

```
<MyPropertyGroup>
   <Foo>bar</Foo>
   <Foo1>bar1</Foo1>
   <Foo2>bar2</Foo2>
</MyPropertyGroup>

```

I could call MSBuild with a short property list:

`<MSBuild Projects="$(SolutionFile)" Targets="Build" Properties="MyPropertyGroup" />`

But it would be the same as calling the task like this:

```
<MSBuild Projects="$(SolutionFile)" Targets="Build" Properties="Foo=bar;Foo1=bar1;Foo2=bar2" />

```

This is helpful if there's a large property list, and allows for only 1 place needs to be maintained.

# Answer
This question was kindly answered on StackOverflow. For anyone else looking for an answer:

```
<PropertyGroup>
  <BuildParameters>
    Configuration=Debug;
    Platform=Any CPU;
        SomeOtherProperty=Foo
  </BuildParameters>
</PropertyGroup>
…
<PropertyGroup>
  <!-- this property can even be extended afterwards, e.g. when a condition is needed -->
  <BuildParameters Condition=" '$(ShallAppendThings)' == 'true' ">
    $(BuildParameters);
    AnotherProperty=SomeValue
  </BuildParameters>
</PropertyGroup>
…

<MSBuild Projects="$(SolutionFile)" Targets="Build" Properties="$(BuildProperties)" />

```

source: https://stackoverflow.com/questions/46226120/how-to-pass-dynamic-properties-to-an-msbuild-task
> 0 votes

---
Tags: automation, builds
---


# Question
Title: Deploying VMs using roles and hosts in Ansible based on operating system
I'm having issues working out how to best apply hosts/roles to different operating systems in Ansible.

I'm currently using Ansible to generate Windows/Mac/Linux VMs, and each of the VMs have different roles which need to be applied to them.

I've read the best practices and have tried to follow them, especially to use the grouping method, but I'm still unsure as they mostly mention webservers and dbservers.

---

site.yml

```
- name: Apply common configuration to all nodes
  hosts: all
  roles:
    - common

- name: Configure and deploy test node 1
  hosts: build-test-node
  roles:
    - build-machine-test-1

- name: Configure and deploy test node 2
  hosts: build-test-node-2
  roles:
    - build-machine-test-2

```

---

environments/test/groups

```
[win_build_nodes]
win_build_machine_1
win_build_machine_2

[mac_build_nodes]
mac_build_machine_1

```

---

environments/test/hosts

```
win_build_machine_1   ansible_host=......
win_build_machine_2   ansible_host=......
mac_build_machine_1   ansible_host=......

```

---

environments/test/meta

```
[mac_build_nodes]
[win_build_nodes]

[win:children]
win_build_nodes

[mac:children]
mac_build_nodes

[linux:children]

[build_nodes:children]
win_build_nodes
mac_build_nodes

```

The `environment/test/meta` file I believe needs to be changed with the `build_nodes:children`, but I'm not sure what to. Previously I was just having the VMs have all the roles built to it with the following:

Old site.yml

```
- name: Apply everything
  hosts: build_nodes
  roles:
    - common
    - build-machine-test-1
    - build-machine-test-2

```

But then realised I didn't want everything being built on the same machine. Note, `build-machine-test-1` and `2` both have separate Windows/Mac/Linux builds as they detect what to include based on the operating system.

Can anyone suggest a better way of separating the VMs based on operating systems?

# Answer
So I managed to keep the same structure roughly, and managed to separate how the machines are generated, however still is a bit fuzzy and can probably be improved to be more efficient!

---

**site.yml**

```
- name: Apply common configuration to all nodes
  hosts: all
  roles:
    - common

- name: Configure and deploy test node 1
  hosts: build-test-node
  roles:
    - build-machine-test-1

- name: Configure and deploy test node 2
  hosts: build-test-node-2
  roles:
    - build-machine-test-2

```

---

**environments/test/groups**

```
[win_test_1_nodes]
win_build_machine_1

[mac_test_1_nodes]
mac_build_machine_1

[win_test_2_nodes]
win_build_machine_2

[mac_test_2_nodes]
mac_build_machine_2

```

---

**environments/test/hosts**

```
win_build_machine_1   ansible_host=......
win_build_machine_2   ansible_host=......
mac_build_machine_1   ansible_host=......
mac_build_machine_2   ansible_host=......

```

---

**environments/test/meta**

```
[win_test_1_nodes]
[mac_test_1_nodes]
[win_test_2_nodes]
[mac_test_2_nodes]

[win:children]
win_test_1_nodes
win_test_2_nodes

[mac:children]
mac_test_1_nodes
mac_test_2_nodes

[build-machine-test-1:children]
win_test_1_nodes
mac_test_1_nodes

[build-machine-test-2:children]
win_test_2_nodes
mac_test_2_nodes

```

---

This solution allows more test nodes to be generated with separate roles added to them, and can just be added to the correct group for the machine to be provisioned.
> 6 votes

# Answer
https://github.com/030/ansible-certbot

main.yml

```
- include: Debian.yml
when: ansible_os_family == 'Debian'

```

put everything that is required on debian like apt in Debian.yml and yum in Centos.yml.

In summary, it is possible to let all OS types use the same role if the OS specific things are put in separate files.

This is a more comprehensive example https://github.com/geerlingguy/ansible-role-mysql/blob/master/tasks/main.yml
> 1 votes

---
Tags: ansible, environments
---


title: Crash course in Dev for Ops?

text:
# Question
Title: Crash course in Dev for Ops?
I schooled in CompSci where we were primarily taught Java, but what I *learned* there is that my passion is systems, so I've always worked the ops side. I'm handy with scripting, so I'm not looking for a site to teach me Ruby, but something to explain more in-depth what you devs do all day. I want to understand the culture better and how you digest the sheer number of files in your projects -- the intangibles.

If I learned today I was being moved to a development team on Monday, what would I want to read this weekend?

# Answer
Since you've tagged this question as "culture", I'm assuming you're not interested in a specific application, but in the broader questions of workflow and management.

I'd probably start with "The DevOps Handbook"; it's a good overview of different things to consider, without diving too deep.

"Continuous Delivery" by Jez Humble is also often referenced; I haven't read much of it yet, but it covers the concepts of source control & automation of builds.

If you're starting to get into applications at scale (this may be too much of an assumption), another good book is "The Practice of Cloud System Administration" by Limoncelli et al.
> 8 votes

# Answer
This is not about DevOps, but straight software development, I assume.

> I want to understand the culture better

Well, the big thing in straight development (without the "DevOps" angle) is certainly "agile", i.e. for the most part SCRUM. You might do worse than to sit down and read the Agile Manifesto or a primer on SCRUM, or Kanban for the more day-to-day, bugfixing, maintenance jobs.

Aside from that, speaking of "culture" at all is, coming from the dev side, mostly a DevOps specific thing. Yes, we have our evangelists as well, specifically for newer stuff like ruby or golang, but not as extreme as in the DevOps/Cloud world, where there are actual paradigm shifts going on.

> and how you digest the sheer number of files in your projects

Having worked on nontrivial ruby applications myself, that's no biggy. See, those files are not just strewn around nilly-willy, but there's a hierarchy, conventions and all that. You never actually need to have all those files in your head at a single point in time, for a well-designed project. If you work at a specific area, it is usually pretty clear where the relevant files are, and you can zoom in on them pretty easily. Same *should* go for other modern programming environments.

In bad applications, that is different, but then the developer won't actually "digest" anything, but just stumble around in a frenzy all day until he quits. ;)
> 7 votes

---
Tags: culture, builds, methodology, software-engineering
---


# Question
Title: Connecting to a Windows machine with Ansible
I'm trying to simply connect to a machine and run the win\_ping module.

There are several tutorials online that outline a few key steps:

* Create a directory `./windows`
* Create a file `hosts` in `./windows`
* Create a directory `./windows/group_vars`
* Create a file `windows.yml` in `./windows/group_vars`
* Insert several variables with your Windows credentials into `windows.yml`

The problem is that none of these actually specify where to put these relative to the '.'. I've tried placing this structure in `/etc/ansible/` and unfortunately it's looking like Ansible isn't picking this up at all. I've also tried moving the structure around but no luck.

How do I get Ansible to reference the directory I've created? I believe `ansible.cfg` would make sense but I can only see how to point it to a default inventory file. Interestingly enough - even adding a group called 'windows' in the base hosts file and calling ansible with the group there - it says no hosts found. I have also uncommented the inventory location in ansible.cfg to point to `/etc/ansible/hosts` and it still says no hosts found when calling `ansible windows -m ping` (or `win_ping` for that matter).

# Answer
There are two places where Ansible looks for `group_vars` subdirectory:

* the playbook directory - the one containing a playbook you run
* the inventory directory - the one specified with `-i` option at run time, or a default one (usually `/etc/ansible`, or `/usr/local/etc/ansible`)

In most typical scenario you might place `group_vars` along with the `playbook.yml`.

The example you posted uses the second method, so to reference the files you need to add `-i` to the command:

```
ansible windows -m ping -i ./windows

```
> 5 votes

# Answer
There are a few GitHub repos regarding Ansible best practices, and I've found enginyoyen/ansible-best-practices helpful not only in your case but also to understand the playbook/inventory/role/task differences in a full-blown Ansible project.

Some highlights: (`/` refers to the **project's** home directory)

* Your variables do indeed go in `/group_vars/`
    + `/group_vars/windows.yml` will be automatically read by hosts you assign to the windows **group**
    + `/group_vars/db.yml` will be automatically read by hosts you assign to the db **group** and so on
* You may also include files with names like `/host_vars/win_host_1.yml` which will be read only by that host
* The groups from `/group_vars/` are defined in your inventory file**s** (plural)

```
[windows]
win_host1.example.com
win_host_2.example.com

```
* Inventory files have names like `/production.yml` and `/staging.yml` to help ensure code in different states of readiness is deployed to (ideally) identical environments.

Other highlights not directly answering your question

* A *role* is a file made of the *tasks* required to do something, like install `nginx`. They have names like `/roles/install_nginx.yml`
* Playbooks have names like `/playbooks/win_hosts.yml` which do nothing but match the *groups* from your inventory files with the *roles* you want hosts in those groups to run.
* **When you run `ansible-playbook` your `$HOME` should be `/playbooks/`** so that the different inventory files are referred as `ansible-playbook -i ../production.yml win_hosts.yml`
* This is also the place to put a custom ansible config file, `/playbooks/ansible.cfg`
* There's also a way to manage *external* roles from Ansible Galaxy or GitHub and keep them separate from custom roles you built, but still in your project directory instead of `/etc/ansible/roles/` where `ansible-galaxy` puts them by default.
* A nice little bash script to install Ansible
> 0 votes

---
Tags: ansible
---


# Question
Title: How to shut down a stateless app gracefully?
Quote from CF training on twelve-factor app’s processes are disposable:

> Apps should also shut down gracefully (but can also die unexpectedly). Cloud Foundry stops routing traffic nearly instantaneously to failed or shutdown instances.

Cloud Foundry apps are stateless. I don't understand how to shut down gracefully a stateless app. A stateless app can always handle `kill -9`?

# Answer
> 3 votes
> A stateless app can always handle kill -9?

Yes. But even a stateless app handles connections from the outside (or it would do nothing, really!). That is the issue here. If your app, say, some HTTP server that only serves static files, and is thus stateless, were to be routinely killed with -9, then ever so often, you would kill it with an active connection. So your customer would get some HTTP error code. In the best case, it is an unimportant small image and he doesn't even notice; in the worst case, it is your main `.js` or `.css` file and he *will* indeed notice badly.

So, by using a weaker signal, you give your stateless app the chance to finish the current request before shutting down.

Obviously, you can finish it off with -9 if it does not go away after a second or two (or however long you wish to give it time to end its business).

# Answer
> 6 votes
`kill -9` is used by the kernel to terminate stateful programs that are no longer responding (including to shutdown requests). To gracefully shut down a stateful app you would use `kill -15` which instructs the application to call it's shutdown functions and do the requisite cleanup. If you app is truly stateless, there should be no difference in the results of `kill -9` and `kill -15`, but technically, a graceful shutdown of this program would still be `kill -15`

---
Tags: cloud, data
---


# Question
Title: Accessing attributes from within recipes
Let us say I do `knife node edit mynode` and within the `normal` section I edit the JSON to add `"myattribute": "myvalue",`. I save this and verify that it has taken effect by issuing `knife search node -a myattribute mynode`, and I see it there. How would I then use this value from within a recipe? I would have expected to just be able to say `node['myattribute']` when running the recipe on `mynode` but that just returns nothing. The usual Ohai attributes such as `node['hostname']` obviously work as expected.

# Answer
> 1 votes
I think that according to chef documentation you should use `normal` with node data.

> A normal attribute is a setting that persists in the node object. A normal attribute has a higher attribute precedence than a default attribute.

you can read about it here:

https://docs.chef.io/attributes.html

https://docs.chef.io/knife\_node.html

some example:

```
  "normal": {
    "ebs": {
      "devices": [
        {
          "mount_device": "/dev/xvdf",
          "encrypt": true,
          "volume_mapping": [
            {
              "device": "/dev/xvdf"
            }
          ]
        }
      ]
    },

```

In you recipe:

```
if node.ebs.devices[0]['mount_device'] == '/dev/xvdf' and node.ebs.devices[0]['encrypt'] ...

```

Another example with `node['myattribute']`

```
   "normal": {
        "myattribute": "myvalue"
        },

```

---
Tags: chef, configuration, configuration-management, infrastructure-as-code
---


# Question
Title: Re-using Python virtual environment vs Re-building it
Currently, we build our application on a build server where we create a virtual environment using `virtualenv` command, install all the Python dependencies into it, then "patch" it using the following bash commands to make it *relocatable*:

```
# Adopt virtualenv to the target system
echo "* Patching virtual envirinment for target system"
for f in "${VENV_PATH}/bin"/* ; do
    if [ -n "$(file "${f}" | grep "text")" ] ; then
    echo "  + ${f}"
        sed -i "s;"${VENV_PATH}";/path/to/the/target/directory/venv;" "${f}"
    fi
done
rm -rf "${VENV_PATH}/lib64"
ln -s "/path/to/the/target/directory/venv/lib" "${VENV_PATH}/lib64"

```

Then, we pack it into an RPM package including some other parts of the application, deploy to multiple target servers and install it via `yum`.

The main motivation behind making the virtual environment relocatable is to "build once, deploy multiple times" - in other words, to *avoid installing the same dependencies on all the target servers*.

The problem is that, this patching script looks really fragile and, from what I understand, this is quite tied to the current Python version and also brings the requirement for the build server to be the same operating system as the target servers (or, at least, to have all the python related paths match).

Is my understanding correct? Would it be better to rebuild the virtual environment on every single target? (we can accelerate `pip` installation commands with, for example, `pip-accel` or caching) What are the general ways to tackle this problem?

# Answer
> What are the general ways to tackle this problem?

The ideal way to resolve your problem would be to dockerise your application, as it would remove any worries you have about compatibility between your build and running servers. Installing dependencies using pip in a Dockerfile is trivial, and as long as you're deploying the same container image you never need to worry about inconsistency.

Edit: as luck would have it I actually found myself wanting to have a venv that worked both inside and outside Docker, so I came up with the following:

```
python3.6 -m venv --copies venv
sed -i '43s/.*/VIRTUAL_ENV="$(cd "$(dirname "$(dirname "${BASH_SOURCE[0]}" )")" \&\& pwd)"/' venv/bin/activate
sed -i '1s/.*/#!\/usr\/bin\/env python/' venv/bin/pip*

```
> 4 votes

---
Tags: deployment, python, performance, virtualenv
---


# Question
Title: Check a process is running with Salt Stack?
Is it possible to check if a specific process is running via a Salt State? Looking through Salt Stack documentation and scouring forums I haven't found any way of simply checking if a service is running and reporting the output.

I have however found this:

```
httpd-absent:
  process.absent:
    - name: apache2

```

The opposite of this would be ideal for my needs e.g:

```
httpd-present:
  process.present:
    - name: apache2

```

Is anyone aware of a method in Salt which would check if a defined process is running based on a name or pattern (regex) and simply report back with a true/false or similar?

# Answer
It's a bit hacky, but you can always do something like:

```
check_process:
  cmd.run:
    - name: ps aux | grep '[f]oobar'

```

The exit code will be non-zero and the state will fail if `foobar` doesn't exist.
> 7 votes

---
Tags: salt, process
---


# Question
Title: Ansible - private git repositories - SSH agent forwarding vs copying private SSH key
I recently started playing around with Ansible and it seems very nice. I don't have much experience in DevOps stuff and never really had to handle any complex scenarios. I started creating my Ansible playbook to replace my current deployment tool - Deployer PHP. I'm stuck at cloning git repository unfortunately. Now, I know I need a public key added to enable access to git repository and here comes my question.

Should I be using SSH agent forwarding (this way I can use my local SSH keys) or should I store private SSH key (encrypted, added to source control) within my ansible project and copy it using Ansible to my target node? I know the question may be very broad, so what interests me is security implications of both approaches.

# Answer
> 4 votes
You can also consider to encrypt the file containing your private SSH key using Ansible Vault.

Some links:

http://docs.ansible.com/ansible/latest/playbooks\_vault.html

https://therealmarv.com/ansible-vault-file-handling/

https://www.calazan.com/how-to-deploy-encrypted-copies-of-your-ssl-keys-and-other-files-with-ansible-and-openssl/

# Answer
> 4 votes
If your git repos is a private github, you can use github methods (deploy keys is very powerful).

If your repos are not hosted on github or they don't provide read-only deploy key feature, The two options you mentioned are both valuable.

## Single DevOps guy

If you are the only person working on the project and Your machine/machines are the only you will run ansible from (no CD tools like jenkins and friends) you're good to go with the agent forwarding option: is likely less possible that you forget your private key unencrypted around.

## Small Team

If you are a team of people working on the same Ansible playbook or multiple people that run the playbook then you have both options:

* keep the agent-forwarding option; every teammate run the script from his machine.
* create a deployer key-pair, upload the pub-key and encrypt the private with vault as suggested by Berlin. The key is encrypted and the scope of the private key is restricted to only 1 or few repos. The problem with this approach is that you have to rotate the vault password every time a teammate leave the team.

I would never put my personal private key, which has a wide scope, in any possibly unsafe place. Vault encryption is as good as the people that uses it and there's a lot of keys/passwords in cleartext out there :)

# Answer
> 2 votes
> Should I be using SSH agent forwarding (this way I can use my local SSH keys)

Yes, that could be an option

> should I store private SSH key (encrypted, added to source control) within my ansible project and copy it using Ansible to my target node?

no

https://serverfault.com/questions/823956/ansible-security-best-practices

> For servers, don't allow root access via ssh at all. Many audits scoff at this.
>
> For ansible, let every admin use their own personal account on each target server, and let them sudo with passwords. This way no password is shared between two people. You can check who did what on each server. It's up to you if personal accounts allow login on password, ssh key only, or require both.

In summary, use an ssh key with password. Let every user use their own ssh keys. Never allow sudo access via ssh.

# Answer
> 0 votes
You can story your private key in one circumstance: if you configure your repository so that it only allows read access for that particular key, and you don't bother about the repository being publicly readable. In this case, you can easily store your private key - obviously make a throwaway key without passphrase. But as I said, only if you are fine with public access to that repository.

If that is not the case, then you are probably stuck with the SSH agent. It is not ideal, as `root` can take the socket, but if you are in a kind of benevolent environment (i.e., no attackable webserver anywhere near), then it should be OK.

---
Tags: ansible, git, security
---


# Question
Title: What is the best way to share maven repository using Docker build (dynamic) image with maven?
We have created a Docker build container to be run by Jenkins for Java based projects. We are using Maven for building our software. The problem we are facing is that, the builds are taking way to long to finish and the main reason for that is that maven is re-downloading all the dependencies for every single build, maven is run from inside the build container and not from the pipeline step.

We thought about using a shared folder but we have dynamic build agents so no shared folder. Is there anyone who is using the same approach and is there a better approach to this?

# Answer
> 3 votes
You might use docker builder pattern. Briefly you need to create Dockerfile.build which adds `pom.xml` and run `mvn dependency:resolve`:

```
FROM maven:latest
ADD ./pom.xml /src/pom.xml
WORKDIR /src
RUN mvn dependency:resolve

```

Rebuild that image every time prior build. Docker will use cached image if ./pom.xml has not changed.

Docker 17.05 introduced 'multi-stage builds' feature which implements this pattern. More information is available here: https://docs.docker.com/engine/userguide/eng-image/multistage-build/

---
Tags: docker, jenkins, jenkins-pipeline, maven
---


# Question
Title: 1 second system engineering work cycles: educated workflow setup for Docker-driven development
Maybe the whole setup is wrong but imagine your task is to model a system, or set of systems using Docker, where exact system configuraton can be tricky.

The workflow resulting from it seems like series of making small changes, waiting to analysing logs, changes, waiting... It seems like during an hour of work you could manage several iterations, but waiting is mind-boggling, though it already takes place on top of base images and inside-container work is done as well to try out things. Additionally, often this work can block many others.

**Question**: how to speed up this process? Do you guys use powerful appliances to reduce the cycle periods which is in term of lean manufacturing a clear *waste* given cost of systems compared to human time (**imagine any change would be landing in the target environment always in just one second**)? Or is the workflow wrong?

# Answer
In some cases, depending on the actual configurations being targeted, it *might* be possible to try the effect of various changes *live*, using other automation tools, and only update the docker images (which acounts for a big chunk of that waiting time) after the bulk of the config changes are eliminated.

Stabilizing first the common/base docker image used across most/all of the candidate variants and sharing it would also help.
> 1 votes

# Answer
Indeed that there can be a compound set of measures to reduce build times.

For example:

* Consider using RAM volumes for builds to reduce filesystem I/O latency
* Consider defining more build targets (Vakilian et al. 2015) and if possible build them in parallel with more hardware build agents
* Consider parallel execution if supported by software build agent e.g. Maven multi-thread mode however it might be not thread safe and corrupt builds in an unexpected way
* As suggested above, evaluate which part of environment can be reused as Docker base image i.e. save time for unneeded repeated tasks (however on long term you still need complete rebuilds to validate integrity of external dependencies)
> 1 votes

---
Tags: systems, workflow, checklist
---


# Question
Title: Is it possible to use multiple if statements in a salt state?
I have a Salt state which checks multiple files exist on our webservers, we have 4 webservers and 2 environments so a combined total of 8. This state is applied to all by using a grain 'webserver'.

As the webservers are different (same underlying back end, different directory/file names) I have implemented an if statement which makes use of the inbuilt grains functionality 'host'.

```
{% if grains['host'] == 'dev-server1', 'test-server1' %}

dart-index.html:
  file.exists:
    - name: /opt/tomcat/webapps/example1/index.html

{% endif %}

```

When applying my state to all 'webserver' minions, this worked as expected. It checked for the above file on the target minions (dev-server1/test-server1) and ignored the others. However I have since added another if statement below this for dev-server2/test-server2:

```
{% if grains['host'] == 'dev-server2', 'test-server2' %}

tees-index.html:
  file.exists:
    - name: /opt/tomcat/webapps/example2/index.html

{% endif %}

```

Now whenever I run the state against the minions it fails as it attempts to check whether both of the above files exist across all minions (which of course won't exist).

So, is it possible to use multiple if statements in a state? Alternatively is there another method in which I could achieve the same goal? I would like to identify multiple files with a specific directory/file name in each type of webserver.

# Answer
You can have more than if statement per state. The issue is that your conditional doesn't seem actually be checking the hostname, it's just passing everything. Try this:

```
{% if grains['host'] in ['dev-server2', 'test-server2'] %}

```
> 3 votes

---
Tags: salt, jinja, yaml
---


# Question
Title: Kubernetes image concept compared to Docker?
Reading the K8S images article, I see there support for images from Azure, Google Cloud Registry and Amazon EC.

* But, why is there no support for Docker Hub?
* Is there any sane way to process/deploy Dockerfiles/Docker images in K8S ecosystem?

Given you can run K8S containers from your Docker images - are there any backdrawals? and if yes, what is the "Dockerfile" in the K8S world?

Side note. My overall impression is that Docker is very nice to adapt in development teams because of its easiness, but in terms of high-end scalability K8S is more powerful. So is this again a new gap between Dev and Ops?..

# Answer
* k8s is a container orchestration tool
* a dockerfile is used to build an image
* k8s is able to run docker containers

One could use yaml files to deploy containers:

> ```
> apiVersion: apps/v1beta1 # for versions before 1.6.0 use extensions/v1beta1
> kind: Deployment
> metadata:
>   name: nginx-deployment
> spec:
>   replicas: 3
>   template:
>     metadata:
>       labels:
>         app: nginx
>     spec:
>       containers:
>       - name: nginx
>         image: nginx:1.7.9
>         ports:
>         - containerPort: 80
>
> ```
> 3 votes

---
Tags: docker, kubernetes
---


# Question
Title: What is a minimal viable number of hosts for an experimental Kubernetes cluster?
For example, to play with Docker Swarm in a way which makes sense you need 2 or better 3 hosts, right?

What is that number for K8S?

# Answer
I'd say the minimal viable configuration to test and get a good overview of pod placement would be 5 machines, 2 masters and 3 nodes.
This allows you to play with a master failure, placement of multiple pods "stretched" across the cluster, etc.
> 10 votes

---
Tags: kubernetes, infrastructure
---


# Question
Title: Exemplify CI/CD to a colleague from Microsoft world using product-neutral terms
After a chat with a colleague who collaborates with quite high-end projects on Microsoft stack (for example, ASP.Net + MS SQL Server).

An interesting outcome was that although I believe that DevOps must be possible in the Microsoft world as well, my knowledge is too little so that I cannot give good examples how he could organize a CI/CD approach. Indeed in our community there are some hints about Team Foundation Server and PSBuildStack (?), but an official source does not reveal all that at a glance. The described hosted services make it even less transparent for a newcomer - is that a cloud build environment like CloudBees/BlueMix?

So, my question is now: how could I describe an example of mature toolchain using established DevOps terms which are valid more or less independent from specific environment/tooling - or do we have to admit that this engineering terminology is not there yet?

Something like this:

1. Source code version control system
2. Build agents
3. Automation and packaging scheduling system to encapsulate artefact composition logic
4. A binary repository (different classes of binaries: libraries, distros, container images)
5. A configuration repository for different environments
6. Package and configuration distribution system to encapsulate deployment logic

# Answer
> 3 votes
I would say that instead of describing a CI/CD system by the products/tools that you use, you should be describing the system by the purpose of each step in the system. Instead of saying "I have a Jenkins server that builds the artifacts, stores them in Artifactory, then Puppet deploys the artifacts to EC2." You could say, "Our build server creates and stores our artifacts in an artifact repository. Then our configuration management software detects the artifacts and deploys them to our cloud infrastructure." Your colleague can then make the connections and find the correct solution that Microsoft provides.

---
Tags: terminology, checklist
---


# Question
Title: What are benefits of using third party app metrics tools versus custom build tools?
With tools like Riemann, New Relic, etc., is there any benefit to building custom app metrics tools for monitoring application health?

# Answer
> 3 votes
A third party app metrics tools is great if it already covers all (or most of) the parameters you want to monitor and it presents the information in a convenient enough manner for your use case - you just need to (find it first and then) learn how to install/use/maintain it and then do that.

Advantages:

* avoids the software development and maintenance costs, which can be quite high depending on the capabilities
* can be immediately used, a custom tool needs to be developed first
* technical support (not always great, tho)
* standardisation: if the tool is a popular one - expertise on it can be requested on a job description instead of having to train every user on it

However there are many cases in which custom development is needed, some already mentioned by Michael: reliability, security/confidentiality, lack of integrations for your custom applications.

Depending on each particular case, custom development can go from tiny scripts extending the coverage of a third party tool to complex wrappers assembling multiple such tools to achieve the necessary monitoring capabilities

Such development is not always lightweight:

* it may require in-depth tool knowledge and/or language expertise outside of your team
* it may need management for heavy/complex or even conflicting dependencies
* the adaptation layers for the tool's actual usage environment may be very heavy

Sometimes the cost of such development is higher overall than that of developing a custom system.

Othertimes the functionality actually needed from the third party tools is a relatively small percent of its capability set and the overhead for learning/installing/operating the tool is simply not worthy.

You guessed it - I wrote my own monitoring tool :)

---
Tags: monitoring, metrics
---


# Question
Title: What are best practices for implementing SSL on Jenkins?
One of the options is to configure SSL in NGinx and to put this in front of Jenkins. Are there any other options for securing Jenkins? I am looking for an answer that is focused on best practices that harden the layer 6.

# Answer
> 2 votes
The same rules apply to hardening Jenkins as for all other web applications:

* Put sensitive content behind an authentication/authorization barrier
* Use SSL for transport-layer encryption and verifying the identity of the server
* Restrict network-level access to only networks where clients should be connection from

Regarding client authn/authz - that's pretty straightforward; there are lots of options with Jenkins for that. I use a plugin that hooks into my org's Active Directory instance, for example.

Regarding SSL - while it is possible to configure Jenkins to serve content over HTTPS rather than HTTP (documentation here), I don't find that to be the easiest way to configure SSL. I think using a reverse proxy like you mentioned is a better choice. Nginx and Apache are the two popular options here. Some reasons for choosing one of these reverse proxies: can take advantage of existing tooling/resources for configuration and certificate management; easier and more flexible to configure to allow for common functionality like forwarding all HTTP traffic to HTTPS, rate limiting, or other QoS features; etc.

Regarding network-level access control - this highly depends on the infrastructure you're running Jenkins on, but it may consist of host-level firewall rules, network-level ACLs, NATs, AWS security groups, etc. etc. Generally speaking, standard stuff here - nothing special about Jenkins in this regard.

# Answer
> 2 votes
Besides the transport layer you should first of all enable Access Control

to protect jenkins from public access.

---
Tags: jenkins, ssl
---


# Question
Title: How should I separate micro service modules for the message part(posts, reply, private) of a social network?
I have designed different part of social networking system using micro service and I have reached to the message part of the system. Messages can be private messages (maybe a separate module), the posts on the wall for each user which can be images, text, video (again maybe another module) and reply messages which is again can be a different module.

Now my problem here is that should I design separate modules (service) for each kind of message. Should I consider load of messages as part of this concept of separating modules?

From one view they are all messages that users use to interact with other users in the system.

How should I design micro service for the message part of this social network?

---

**EDIT-1:** Micro-service modules of my project for now:

* user
* authenticator
* notifications (email, push, etc)

Now my social network project has 3 parts for messages:

* users' posts
* users' private messages
* users' replies to other messages

These 3 categories can be 1 module that can manage all these logics. Or 3 different modules `user posts`, `user private messages`, `user replies`.

The question is how should I separate them? **One of** my concerns is that each category would get more logic and it will lead to a bloated module.

**My Final Question:** Is it just the role of micro service that separate a module from another or should I take complexity of module into account for the separation?

**NB:** By messaging I don't mean micro-service module's inter communication, but user's messages in the ecosystem.

# Answer
> 1 votes
From my point of view, you can design microservices just like classes in an application. That is, you are looking for maximum *cohesion* within a service. In other words, all stuff inside one microservice should work together to achieve exactly one well-defined *responsibility*, and no other microservice should have any content for the same responsibility.

Furthermore, you want to minimize coupling. That is, mainly, that each microservice should have a very well defined interface, and should be easily replacable in the grand scheme of things. This also means making use of concepts like dependency injection and such.

So as long as you have these two aspects under control - cohesion and coupling - you are fine. The exact separation of concerns (more coarse or more fine-grained) is not *that* important and will likely differ between different applications.

There are obvious concerns here - for example, it would not make sense to put "Logging" and "GUI" in the same component (ask yourself questions like this: "Would anything else than a GUI use the Logging?" or "Does GUI need to know how Logging is implemented?" or "Does the GUI need to be implemented by the same team that does Logging?" and so on). Another frequent example would be handling of data imports - does that belong into the GUI service just because users usually upload their Excel files through a GUI? Obviously not - you may wish to upload Excel via a REST API or by pulling them from another server in a batch job, etc.

So, in your example, you best sit down and define the responsibilities for every component. Make sure there are as little overlaps as possible. This will usually lead to merging of components you thought to be different; or splitting up of components which turn out to have a well-defined border.

---
Tags: microservices, architecture
---


# Question
Title: Why is my build script only failing when it is run in a vNext build?
I am running a build process on an on-prem server with TFS 2017 and vNext. The build *works*, but it takes 20+ minutes. This is an application that does msbuild, ngc, yarn, and karma. I would be totally fine with leaving it the way it is except for the 20 minutes it takes. I found a way to reduce the build by at least 8 minutes by running two of the yarn scripts in parallel. I have been able to successfully run these two scripts in parallel on my laptop, on a VM, and directly on the build server (I remote into the server to do it). However, every attempt to make this work through vNext fails with the exact same error.

Here is the error:

```
2017-09-13T20:06:17.9961416Z 13 09 2017 15:06:13.808:ERROR [karma]: TypeError: Cannot read property 'profile' of null
2017-09-13T20:06:17.9961416Z     at factoryCallback (D:\Agents\Agent1\_work\126\s\Src\BFY.Admin.Web\node_modules\webpack\lib\Compilation.js:269:13)
2017-09-13T20:06:17.9961416Z     at factory (D:\Agents\Agent1\_work\126\s\Src\BFY.Admin.Web\node_modules\webpack\lib\NormalModuleFactory.js:253:5)
2017-09-13T20:06:17.9961416Z     at applyPluginsAsyncWaterfall (D:\Agents\Agent1\_work\126\s\Src\BFY.Admin.Web\node_modules\webpack\lib\NormalModuleFactory.js:99:14)
2017-09-13T20:06:17.9961416Z     at D:\Agents\Agent1\_work\126\s\Src\BFY.Admin.Web\node_modules\tapable\lib\Tapable.js:204:11
2017-09-13T20:06:17.9961416Z     at NormalModuleFactory.params.normalModuleFactory.plugin (D:\Agents\Agent1\_work\126\s\Src\BFY.Admin.Web\node_modules\webpack\lib\CompatibilityPlugin.js:52:5)
2017-09-13T20:06:17.9961416Z     at NormalModuleFactory.applyPluginsAsyncWaterfall (D:\Agents\Agent1\_work\126\s\Src\BFY.Admin.Web\node_modules\tapable\lib\Tapable.js:208:13)
2017-09-13T20:06:17.9961416Z     at resolver (D:\Agents\Agent1\_work\126\s\Src\BFY.Admin.Web\node_modules\webpack\lib\NormalModuleFactory.js:74:11)
2017-09-13T20:06:17.9961416Z     at process.nextTick (D:\Agents\Agent1\_work\126\s\Src\BFY.Admin.Web\node_modules\webpack\lib\NormalModuleFactory.js:205:8)
2017-09-13T20:06:17.9961416Z     at _combinedTickCallback (internal/process/next_tick.js:67:7)
2017-09-13T20:06:17.9961416Z     at process._tickCallback (internal/process/next_tick.js:98:9)
2017-09-13T20:06:17.9961416Z 13 09 2017 15:06:13.823:ERROR [karma]: TypeError: Cannot read property 'profile' of null
2017-09-13T20:06:17.9961416Z     at factoryCallback (D:\Agents\Agent1\_work\126\s\Src\BFY.Admin.Web\node_modules\webpack\lib\Compilation.js:269:13)
2017-09-13T20:06:17.9961416Z     at factory (D:\Agents\Agent1\_work\126\s\Src\BFY.Admin.Web\node_modules\webpack\lib\NormalModuleFactory.js:253:5)
2017-09-13T20:06:17.9961416Z     at applyPluginsAsyncWaterfall (D:\Agents\Agent1\_work\126\s\Src\BFY.Admin.Web\node_modules\webpack\lib\NormalModuleFactory.js:99:14)
2017-09-13T20:06:17.9961416Z     at D:\Agents\Agent1\_work\126\s\Src\BFY.Admin.Web\node_modules\tapable\lib\Tapable.js:204:11
2017-09-13T20:06:17.9961416Z     at NormalModuleFactory.params.normalModuleFactory.plugin (D:\Agents\Agent1\_work\126\s\Src\BFY.Admin.Web\node_modules\webpack\lib\CompatibilityPlugin.js:52:5)
2017-09-13T20:06:17.9961416Z     at NormalModuleFactory.applyPluginsAsyncWaterfall (D:\Agents\Agent1\_work\126\s\Src\BFY.Admin.Web\node_modules\tapable\lib\Tapable.js:208:13)
2017-09-13T20:06:17.9961416Z     at resolver (D:\Agents\Agent1\_work\126\s\Src\BFY.Admin.Web\node_modules\webpack\lib\NormalModuleFactory.js:74:11)
2017-09-13T20:06:17.9961416Z     at process.nextTick (D:\Agents\Agent1\_work\126\s\Src\BFY.Admin.Web\node_modules\webpack\lib\NormalModuleFactory.js:205:8)
2017-09-13T20:06:17.9961416Z     at _combinedTickCallback (internal/process/next_tick.js:67:7)
2017-09-13T20:06:17.9961416Z     at process._tickCallback (internal/process/next_tick.js:98:9)

```

The two scrips that are running are:

```
"test": "SET NODE_ENV=test&& karma start karma.conf.js"

```

and

```
"build:prod:aot": "rimraf aot && ngc -p ./tsconfig-aot.json && SET NODE_ENV=production&& webpack"

```

The error is coming from karma. I get the same error from vNext no matter how I try to run these two scripts. I have tried

* Adding a script to run them with the `concurrently` npm package. Executed using a Windows batch command.
* Adding a script to run them with the `npm-run-all` npm package. Executed using a Windows batch command.
* Running a powershell .ps1 script to run each command with Start-Job.
* Running a .bat file that in turn runs two other .bat files, one for each script.

In every case I get the exact same error from vNext. In every case I get the expected result when running from my laptop, a VM, and when running the script while logged into the build server.

What am I missing here? Why is vNext behaving differently?

# Answer
The conflict is occurring between `ngc` and `webpack`. When `rimraf aot && ngc -p ./tsconfig-aot.json` is moved to its own script and executed prior to `test` and `build:prod:aot` then running `test` and `build:prod:aot` with `npm-run-all` will work as expected.

Also, running with PowerShell is not necessarily a good idea since it interprets some of the output differently than the Windows shell. For example, some packages will write out "warnings" to the error stream.

```
"build:prod:aot": "rimraf aot && ngc -p ./tsconfig-aot.json",
"webpack:prod": "SET NODE_ENV=production&& webpack",
"test": "SET NODE_ENV=test&& karma start karma.conf.js",
"buildAndTest": "npm-run-all build:prod:aot --parallel test webpack:prod"

```
> 1 votes

---
Tags: powershell, tfs, node.js
---


# Question
Title: Docker Security Scanning: How to integrate Clair with a docker private registry?
According to the following documentation it should be possible to integrate clair with a docker private registry. How to do that?

> **Registry Integration**
>
> Clair can be integrated directly into a container registry such that the registry is responsible for interacting with Clair on behalf of the user. This type of setup avoids the manual scanning of images and creates a sensible location to which Clair's vulnerability notifications can be propagated. The registry can also be used for authorization to avoid sharing vulnerability information about images to which one might not have access.

# Answer
Both Docker Registry and Docker Engine have API interfaces. I consider it should be possible to implement a Docker plugin for this integration to have images scanned for example as somebody pushes them. (side note - Docker registry is an image registry, not container registry).

This scanner functionality is also possible through the commercial Docker Enterprise edition. https://docs.docker.com/docker-cloud/builds/image-scan/

UPD here an example with clair/docker, looks quite simple http://blog.xebia.com/docker-containers-vulnerability-scan-clair/
> 0 votes

---
Tags: security, clair, registry
---


# Question
Title: Cassandra DB: store binary file record via shell
While the dev team works on a REST service to manage binary data in Cassandra, ops would like to store local blobs directly from shell.

Is it actually possible via CQL/bash without a Cassandra driver? I could find so far only an example for Ruby.

# Answer
> 5 votes
The only way to do it with cqlsh is wrap binary file into CSV file and run `cqlsh --some_args -e 'COPY table_name FROM binary_wrapped.csv'`, but in order to do so you have to generate a new primary key to use in CSV-file.

IMO: Since cqlsh is a wrapper around python client, your ops should just write simple python script for this operation, it would be much easier in many ways.

---
Tags: shell-script, cassandra
---