# Question
Title: Is there a premium version of Google translate that is available for purchase to integrate into business applications?
It looks like there may have been at one time a version of Google translate called "Business" that did not employ the free API used by the version that comes up in my browser. If there is, how can I get more information about it (I've already googled it and cannot find anything)? If not, is there some kind of replacement that can be used in its stead?

# Answer
Yes !

There's an API available on Google Cloud & you can find more details about it on https://cloud.google.com/translate/
> 1 votes

---
Tags: google-apps
---


title: Flask, Gunicorn and Docker Swarm

text:
# Question
Title: Flask, Gunicorn and Docker Swarm
so I have this nice rest API implemented in Flask running in docker and I am thinking about scaling. This is how I progressed:

```
CMD ["flask", "run", "--host=0.0.0.0"]

```

and

```
docker run -d -p 5000:5000 pyrest-alpine

```

works fine

```
CMD ["gunicorn", "-w", "4", "-b", "0.0.0.0:5000", "pyrest:app"]

```

and

```
docker run -d -p 5000:5000 pyrest-alpine

```

This also works fine, Gunicorn provided some scaling via prefork worker model in a single container now I want to scale via replication of Docker swarm with healthchecks where both

```
CMD ["gunicorn", "-w", "4", "-b", "0.0.0.0:5000", "pyrest:app"]

```

and

```
CMD ["flask", "run", "--host=0.0.0.0"]

```

work just fine with

```
docker service create --name pyrest-swarm --replicas 2 -p 5000:5000 --health-interval=2s --health-timeout=10s --health-retries=3--health-cmd "curl 0.0.0.0:5000/status || exit 1" pyrest-alpine

```

My question is Do I still use gunicorn when using docker swarm replication?

# Answer
> 2 votes
Yes, when using Docker Swarm you'll still want to use Gunicorn in your Docker containers.

Gunicorn is necessary to facilitate the communication between the server and your web application. In the case of a Dockerized Flask application this is still the case as Gunicorn handles the communication between the Flask application and the Docker container. Adding Docker Swarm to this stack basically just means that you have more instances of your container, and that Swarm will handle balancing the incoming requests to each instance.

---
Tags: docker, docker-swarm
---


# Question
Title: IT staff sizing in the DevOps era - how and when would a DevOps team scale up?
Given that DevOps allows IT staff actually to downscale and do more through all the automation, what are limits to that?

That is, what are rules of thumb - if there can be any - where you would know you need more people despite rapidly progressing rates of automation and introduction of RPA (Robotic Process Automation)?

Note: this is about the staff size of the whole IT departments altogether put in relationship to your business size, structure and function (size of a single team is well defined by the two-pizza team concept).

# Answer
> 8 votes
IMHO it's not a good idea to directly and immediately apply a team's gains from its DevOps transformation into scaling the team **down**. At best you'll lose just the team's motivation to do any such improvement in the future. But most likely you'll lose most/all the talent in that team.

Instead:

* allow the team to appreciate the benefits of the difficult transformation they went through, so that they spread the word to your other teams that will have to go through (or are already struggling with) the transformation
* focus on delivering better quality products/services, faster - with the same team - this is now possible due to the transformation.
* in many cases your business will pick up speed because of the transformation - soon you'll more likely need to hire people - IMHO it's better to use the experience of the ones you already have rather than educating new hires
* allow voluntary attrition to work for you and get the effective downscaling down the road.

But generally the DevOps transformations tend to not happen fast enough to allow for the above choices :)

Now onto scaling **up/out**.

Ideally the DevOps transformation should have greatly increased the predictability of your sw development and delivery process. You should be able to tell with a great degree of precision you existing team's capacity: how much sw/services of a certain quality can your current team get done in a certain amount of time.

**When** you need more stuff done, or done better/faster than the current team's capacity you need to scale the team up/out. The exact details come from whatever process/methodology you use for your day-to-day activities, probably some agile variant (see also Does my organization need adopt Agile Soft. Dev. before adopting DevOps?)

**How** \- *apparently* easy: hire more people to handle the overflowing activities. If you're worried about how fast you can get them onboard and actually handling that overflow work at the expected rates: start hiring earlier, based on projections.

The reality can, however, a bit more difficult than that. Scaling a software product/service or the development process and tools supporting it is not necessarily straight-forward, even for a DevOps-empowered organisation.

One of my favourite examples is the weakest DevOps link **when it comes to scalability**: Continuous Integration. If the CI build fails there can be no Delivery/Deployment (kinda undermining the "continuous" term in the CD context).

Just because a certain team is happily using CI with decent results it doesn't mean the same will happen if the team size doubles. Nope, the results could be devastating - the delivery process can grind down to a halt (in extreme cases). I'm trying to explain this in Congestion in Traditional CI Systems (**disclaimer:** I'm the founder of the company owning the referenced page and offering a solution to this problem).

Another example could be, for example, the version control system (VCS) being used: as the software being stored into a repository evolves so does its change history, pushing the limits of the VCS system, sometimes beyond acceptable performance levels. Switching to a more scalable VCS system or splitting the codebase into multiple repositories could be an approach to address the problem.

But evolving the development processes and tooling supporting them alongside the evolution of the software products/services being delivered are fundamentally just what a mature DevOps team does. They just need to also be taken into consideration when in the team's scaling up/out strategy.

With that in mind, the **how** can be, indeed, reduced to just increasing the team size to cover the additional amount of work necessary for execution of the above-mentioned strategy.

---
Tags: culture, automation, human-resources
---


# Question
Title: Are CI and CD DevOps team tasks or are these separate professions?
DevOps is about preventing silos and let teams work more together then let them throwing things over the wall. In an ideal situation every individual in a DevOps team is able to create programs, test them and make them available to the customer. According to me an important competence of a DevOps engineer is communication as this is required to get acceptance in a team and among teams for new tools and processes.

The more I think about it the more I get doubts regarding the structure and responsibilities in a team. From a DevOps perspective, every individual should be able to do dev, qa and ops activities, but if you compare it with for example football, all players need to be able to kick a ball, but some of them are specialized in goalkeeping and others in scoring goals. So a DevOps team could exist of QA, Ops and Dev engineers, but all of them are able to test, create and deploy software and are a basically able to kick a ball like in football.

As DevOps does not mean specialzed in dev, qa and ops, like goalkeepers will normally not score as much as a striker, is CD a profession or is it one of the DevOps tasks, or should it be everybody's responsibility, but why are Dev, QA and Ops engineer part of a DevOps team as well? From that perspective there should be a CD engineer as well, but CI is not part of CD. Should two professions, i.e. CI engineer and CD engineer be part of DevOps team as well?

# Answer
> 1 votes
DevOps doesn't mean your admins should be doing QA and your testers should be developing. They rather should be aware of what other team members are doing and work as a team instead of just throwing responsibility over the wall and not caring about what other team members will do with it.

You will probably benefit from having someone on the team that knows how to build code integration pipelines. They may be specialized in it just like others specialize in databases, frontend or system fine-tuning. But you can't just assign a person and tell them to do all things CI, developers will write unit tests, testers will write some more high-level tests, administrators will have to make sure the tests actually run on environments that match the production. You can assign a person to oversee it but on small teams it doesn't make much sense. Especially since CI is more than just pipelines it's about changing how developers work.

Same goes for CD, you may have someone who have more experience with how to design software that can be continuously deployed but you have to change the way developers work to actually achieve it. You can't just tell someone to continuously deploy software that wasn't designed for it.

So summing up, if you want someone on that team that's there just for CI/CD that person's tasks will be to teach the team how to work.

# Answer
> 1 votes
DevOps is a combination of two roles Development and Operations, CI & CD are the main role a DevOps must be doing. Continuous Integration (CI) is a part of developer's role and Continuous Deployment/Delivery is a part of operation/infra/release-management role. In the agile methodology of development we've combined these two different role to a new position called DevOps who'll be responsible for release management or deployment. Thus it is a mandatory for a DevOps to know about CI & CD.

---
Tags: terminology, organization
---


# Question
Title: How to check return status of parallel branches in jenkins pipeline
I am running a Jenkins job on multiple slaves. Following is the code structure:

```
def branches = [:]
def allNodes = Jenkins.getInstance().getNodes()
for (int i =0; i < allNodes.size(); i++) {
branches[allNodes[i].name.toString()] = {
    node(allNodes[i].name.toString()) {
    .
    .
    stuff
    }
}
parallel branches

```

Now, I want to get the status of the job parts running on nodes so that in case some job part fails on some node I can mark that node as offline. Is there any way to do this?

# Answer
> 4 votes
I tried storing the slave job part status in a file and stashing it on node and then unstashing it back on master. It works but I am looking for a cleaner way. Following is the current approach i am using:

```
def branches = [:]
def allNodes = Jenkins.getInstance().getNodes()
for (int i =0; i < allNodes.size(); i++) {
String nodeName = allNodes[i].name.toString()
branches[nodeName] = {
    node(nodeName) {
    .
    .
    String outputFile = nodeName + "-output"
    writeFile file: outputFile, text: result.toString()
    stash includes: "*output", name: outputFile
    }
}
parallel branches

for (int i = 0; i < allNodes.size(); i++)
{
    String filename = allNodes[i].name.toString() + "-output"
    unstash filename
    def value = readFile filename

    // Mark node offline based on the variable value
}

```

# Answer
> 3 votes
You may want to try the Join plugin. Some more details about this plugin (from the linked page):

> This plugin allows a job to be run after all the immediate downstream jobs have completed. In this way, the execution can branch out and perform many steps in parallel, and then run a final aggregation step just once after all the parallel work is finished.
>
> The plugin is useful for creating a 'diamond' shape project dependency. This means there is a single parent job that starts several downstream jobs. Once those jobs are finished, a single aggregation job runs.

For more details on that, refer to my self answered question about "Why trigger something remotely (using Jenkins) and then just forget about it?".

---
Tags: jenkins, jenkins-pipeline, jenkinsfile, groovy
---


# Question
Title: I need a good tool for running a series of Spring Boot based tools on Docker, where each needs a web page and a database connection
These are pretty much just tools our support and ops departments need to solve small tasks which crop up as a routine sideeffect of our main business operation. They might need to find all instances of Entity with both Field A and Field B, or change some property in some database without having to go through Ops.

Anyway, I write them in Kotlin on Spring Boot just for speed and I use a different Spring Boot instance on each to avoid dealing with conflicting datasources in a single Spring Boot app. I use Docker just because we all use Docker, so why change now. And Docker-compose for easy deployment.

The problem I have is that I want a single command to git update, rebuild image and restart containers. Or, even better. When one of the tools gets updated, just recreate it and replace the container live.

Obviously I could script this, but it seems like something that would exist already. Is there something out there that's "like" CD, but which doesn't have all the overhead?

I tried Dokku. Didn't like it. Way too complicated for what I needed.

# Answer
According to me a CI like Jenkins or gitlab should be used to build the docker images when the git code changes. Once this works well, you could start to run some scripts that deploy to an environment (CD). First of all, I would like to emphasize that implementing CI is the highest priority as the docker images seems to be built manually at the moment.
> 2 votes

---
Tags: docker, git, deployment
---


# Question
Title: GitLab - trigger pipeline to build on specific runner
**Scenario**

1. Create a new virtual machine with runner installed
2. Trigger a pipeline to run (from another project), which runs on this new virtual machine
3. Do something else, or fail if pipeline above fails on the new virtual machine

**Question**

I'm using GitLabs Triggers API to run a pipeline from a project, however would like it to run on this newly created virtual machine.

What would be the best way of parsing this information to the GitLab API, or would something like tags be the best way of doing this?

---

**.gitlab-ci.yml**

```
stages:
  - deploy to test
  - test
  - deploy to prod

Terraform Deploy To Test:
  stage: deploy to test
  script:
    - cd test_deploy
    - python deploy.py

Testing:
  stage: test
  script:
    - curl --request POST --form token=TOKEN --form ref=master http://gitlab.example.com/api/v4/projects/12/trigger/pipeline

Terraform Deploy to Prod:
  stage: deploy to prod
  script:
    - do something if the above testing succeeds, or fail

```

The above `gitlab-ci` file would generate a new build machine and deploy it to the "test group" within vsphere with a python script. (Deploying it would activate a `gitlab-runner` and tie it to a specific project and have some tags associated with it).

To test, it then triggers a pipeline call using the GitLab API, and I would like that pipeline to run on the new generated build machine on vsphere.

---

Would a work around maybe be something like using gitlab runner exec command or using environments?

# Answer
There currently isn't a solution for building on a specific runner in GitLab, but there is an issue open for Sticky Runners, which hopefully will be out in the next 3-6 months according to the Milestones!

---

The work around I've done so far to build a project on a specific runner is to use the GitLab Runner API, in a rather hacky way, along the lines of:

* Get all project runners
* As I know I've deployed the latest runner, that would have the highest runner "number"
* Pause all the other runners associated with the project in question
* Trigger the pipeline to build on the latest runner
* Poll the GitLab API to get the status of the pipeline
* Once that succeeds, resume all other runners!
* If the pipeline fails, remember to resume the paused runners...
> 2 votes

---
Tags: gitlab, gitlab-ci-runner
---


title: AWS S3 ACLs across accounts

text:
# Question
Title: AWS S3 ACLs across accounts
I have two AWS accounts: A and B (viewable from https://console.aws.amazon.com/organizations). I have given an S3 bucket in account A a policy that allows account B to do everything in that bucket. When I have an EC2 instance on account B upload a file to that bucket, however, account A cannot access it because the owner is account B.

Is there a way of resolving this without having account B specify an ACL that includes account A for every upload?

# Answer
No. The account that uploads an object into a bucket owns the object. All the bucket owner account can do, unless granted permission by the object owner account, is delete the object.
> 3 votes

---
Tags: amazon-web-services, amazon-s3
---


# Question
Title: Build / Deployment process for .Net/Mulesoft/Oracle stack
I have the following challenge to streamline and setup a development process.

The program runs simultaneously, and each one has its own timeline for release, as follows:

* Program 1
* Program 2
* Program 3
* Program 4
* Program 5

Under each of the programmes, there are multiple projects running in parallel. Each project has a different technology, like Web layer `(.NET)`, middle-ware `(Mulesoft)` or database `(Oracle)`.

The challenge is that I have to find a configuration management tool, which considers branching, automated build process, deployment and integration tracking with JIRA.

Can some one suggest the best tool to seamlessly integrate and orchestrate automated build and deployments?

Technologies involved are:

* Oracle (Basically SQL scripts)
* Middleware (Mulesoft - maven script)
* .NET web layer (UI part)

Key points to be considered is the build and deployment process should be technology agnostic, as I have an environment with `java`, `.NET` and `oracle world`.

I am thinking of ( Github - Bitbucket/Bitbucket pipeline) actually but any suggestion would be highly appreciated.

# Answer
> 3 votes
For something that requires being agnostic to the tooling, and having the flexibility you need, your best bet is Jenkins. https://jenkins.io/. They have a ton of plugins that would meet any use case. Be aware that Jenkins can be heavy for small teams, but your other options lack the flexibility and support for different communities.

You can check out other tools:

* Team city
* CircleCI/Travis CI
* Semaphore
* ETC

---
Tags: continuous-integration, deployment, continuous-deployment, java, dotnet
---


# Question
Title: Docker vs CMS - related or a different thing?
I am new to DevOps and I am trying to catch the techniques and their names and meanings.

I looked wikipedia explanations on Puppet (a Configuration Management System example) and that of Docker.

Docker page says it is categorized as a virtualization platform. Docker manages many virtual containers on same Linux machine.

Is there some similar way as with Puppet and other CMS to tell what configuration is set up on container ad-hoc creation as there is in a CMS or is there anything similar on these two (a CMS and Docker and like)? Could they be used only separately or also together?

At least there exists a dockerfile to do some kind of similar sounding thing than system description language makes in CMS. That makes me think they perform similar tasks.

So in short I would like to how to categorize CMS vs Docker.

# Answer
Something like puppet would help automate and configure the Docker Container. CMS is configuration/provisioning and Docker is like you said a virtualization platform. You're trying to compare two completely different things. Here is an article on how to configure puppet with docker, sometimes an example helps. https://puppet.com/blog/building-docker-images-puppet

For another way to classify, Docker is the VM you host your software. CMS is automation on how to provision it.

As a side note, most people are moving to Kubernetes to orchestrate and provision containers on the cloud.
> 4 votes

# Answer
The main difference is that docker images are immutable and using is CMS is mutable. Once a docker image has been created it will work on all servers, while running a CMS could fail due to different environments, e.g. internet down, firewall, package repository unavailable.

Could a CMS be combined with docker? Yes, to configure docker images using packer for example. Packer is able to run existing CMS code inside docker. Another use case is deploying and configuring docker environment variables using a CMS.
> 2 votes

---
Tags: docker, terminology, configuration-management, dockerfile
---


# Question
Title: Internet connection inside Docker container in Kubernetes
I have a Kubernetes pod based on a Jenkins/slave container to which I mount a Docker socket and a Docker binary file with necessary kernel modules in privileged mode.

Inside that pod I build a Docker image, in which I run Docker container. Inside that container I don't have Internet connection at all because the pod container uses a flannel network `(198.x.x.x)` and that container uses bridged docker network `(172.x.x.x)`, which is not available inside the pod container.

How can I make the Internet available inside the second container which is being created inside the Kubernetes pod container?

Using Docker API in Jenkins pipeline is not a solution for me as long as it limits output of error logs and I can not commit changes made in the second container because of immediate removing of that container after build.

# Answer
> 1 votes
In my opinion k8s should not be used to build docker images as it is an orchestration platform, but there is an open issue in the GitHub k8s repository. In conclusion, building docker images on a k8s cluster seems not to be possible at the moment.

---
Tags: docker, kubernetes
---


# Question
Title: How to build all projects in a monorepo by starting a build in Jenkins?
We have a monorepo that consists of multiple projects that contain a build file. How to ensure that all these projects will be built when the job is triggered?

One solution I am thinking about is creating a shell script that detects whether the folders contains a build script, add the name to a set, iterate on this set and trigger a build for all of the projects. I wonder whether this is a good approach, but it feels cumbersome.

# Answer
Build tools like or could benefit you greatly here. It would do exactly what you have in mind as long as you have child project definition (with a `build.gradle` in gradle or `pom.xml` in maven).

I would not recommend the script as it would mean you have to maintain and tweak it in the future. Instead, investing in build tools can give you great ROI as they do much more than just build.

At the end if you cannot integrate a build tool, go for the script. It is a good idea also.
> 1 votes

---
Tags: jenkins, monorepo, gradle
---


# Question
Title: Multiple git profiles especialy with Atom
I have configured Git to use a specific CA certificate, but how can opt-out for it other than using the switch `-c http.sslVerify=false`? (or enforce other certificate which seems better in terms of security).

Okay, a workaround was to create a shell script for a shortcut and use it like

```
gitf clone

```

That is okay for a quick test in a secured and trusted environment but it is a potential security breach. Do not try this at home.

The real gotcha comes then in if I would like to use the Atom editor. Somehow I cannot find an option for the git executable.

# Answer
> 2 votes
You'd better add this CA to a bundle of valid CA and keep using it for all cases with

```
git config --add --global http.sslcainfo=/path/to/cabundle
git config --add --global http.sslbackend=openssl

```

Using https://curl.haxx.se/docs/caextract.html as starting CA bundle sounds a good idea in my opinion.

---
Tags: git, windows
---


# Question
Title: Should performance tests be run using a CI and how often?
There are tools that test the performance of applications, like gatling. It is possible to run such a test once in a while, but in practice is is not run periodically. We were discussing how to run it regularly and add it to every build, but the CI was getting slow. On the other hand it is important to know if a code change decreases the application performance.

# Answer
> 6 votes
I will assume by CI you mean *one* integration server. The best way to get the maximum out of your tests and environment stability is to decouple environment per test type. The best example is production which is lively "tested" by your users.

Be aware that by environment, I mean the server or servers used to deploy a specific version of your application or applications (I will refer this as the build). In such emvironment, you would tipically have similar setup as your live/production environment. Based on your team restrictions, here is what I proposed:

**Restricted from having a new environment**

Simply schedule a job to start load testing when the traffic is minimal. This is usually at night. Upon test completion, a report should be available and granular enough to indicate where improvements are required based on your internal NFR threshold.

**No restriction on having another environment**

* Create a new environment for load tests
* For each new build, if it was successfully deployed and/or tested in the CI server, deploy to the new environment
* Run the load tests and save the results. Depending on your CI tool, you might be able to configure reports and alerts according to your results quality

In both case, you will need to specify your load test suite in a way that allows you to figure out where latency is too high. I would personally suggest you the second solution. Though it requires more setup and maintenance, it is the most benificial for the following reasons:

* Tells you exactly when the build starts to be unstable. For the first solution, you might need to investigate because a lot of change is happening during the day.
* High visibility on *what/who* made the system slower. At this time, the teams are aware of the issue and can remedy right away. The nighly job might be overlooked on the long run.
* Can serve as a performance gate where all builds with a certain response time will stop from going further (i.e. production) until the response time is improved. This is unpractical to put in place with a nightly job. Also, you would need to wait a day to see the result of an improvement or run the test manually.

If you are in the journey to CI/CD, fast failures is prefered as delayed failures. Losing a day to fix a problem can have considerable impact on your time to market.

# Answer
> 3 votes
There is no right answer, in a sense you're facing a requirement conflict. You can't eat the cake and have it, too ;)

If a particular QA verification (any, really, not just performance) is important enough and you want to catch any regression right away - you have to insert it in the CI execution. But, unless you manage to somehow execute it in parallel with other existing jobs in the CI pipeline, it will extend the duration of the CI execution.

You could, if you want, perform it in a "slower" CI pipeline, executed less often than your regular CI pipeline. For example just nightly instead of after every commit. The drawback is that when a regression is detected you'll have to sift through a bigger pile of commits to identify the culprit. Which can be done either manually, though human analysis, or using automated bisections executing the affected QA verifications (or even a mix of both). But that takes time and resources. If it's a rare event it could be an acceptable compromise, if not then you have to decrease the time between the "slower" CI pipeline executions.

You have to decide what's more important: the CI execution speed or catching regressions faster. This decision can also depend on the development stage context. For example:

* if you're in the master/dev development branch where you have a lot of changes coming in the CI execution speed would likely be more important
* if you're on a release branch cut for production, where you have a significantly lower rate of commits than in the master branch, catching regressions faster would have priority (the longer CI execution would have less impact)

Finally, such decision would be easier when using a CI system capable of combining changesets because the impact of a longer CI execution over the *effective* changeset verification time is reduced, see From 6 months to 16 seconds? grokking DevOps speedup data from DevOps report by DORA and Puppet Labs.

# Answer
> 2 votes
One additional way to think about this is to consider alternate approaches, and whether they are suitable for your needs.

One of the axis I break down performance metrics on is whether they are real-user metrics or synthetic metrics. Your performance test is presumably synthetic: it runs the same set of actions every time in the same environment, so as to make it easy to compare runs against each other.

However, synthetic tests don't show the actual effect a change will have on users, which is what RUMs are all about. You can do canary rollouts, and if performance decreases in the canary past an acceptable threshold, roll it back and investigate (I saw a talk from Facebook about how they gather full debug traces on their canary servers and so can find which code was responsible for the slowness and automatically backtrace that to a set of commits and authors). This is more realistic in the user impact, but can be difficult to do without large amounts of traffic, as there is normal variation in what real users do.

Ideally, you'll be using **both** of these methods of tracking performance. But since we don't live in an ideal world, you may be able to retain a focus on performance at your company by shifting more emphasis on one of these versus the other, to free up resources for other needs. In your particular example, is it an ok trade-off to have slower deploys if it means reducing the CI feedback cycle? Maybe - but that depends on your company.

---
Tags: continuous-integration, performance
---


# Question
Title: How to ensure that git subtrees are kept up to date?
**Problem:**

We use Data Transfer Objects (DTO) to ensure consistent interfaces across multiple services using TypeScript.

We're currently using a git subtree in order to make sure that each repository has its DTO dependencies up to date. These DTOs are changed semi-frequently.

The challenging part is that subtree pull / subtree push are outside the traditional git process for most developers.

**Question:**

What process patterns have people found work well to ensure each related repositories subtree is up to date on each push / pull? Post commit hooks? CI scripts? Etc. etc?

# Answer
> 2 votes
GIT hooks *could* work, but the trouble is that they're not automatically installed when the developers pulls a workspace. Which means they *might* end up being installed/used correctly or not. Unreliable.

CI scripts *could* work as well, but the're reactive, simply indicating that something went wrong, in most cases humans still need to identify the culprit (which *might* be caused by an improper DTO subtree or by something else) and fix the problem.

If you want your developers to properly pull their subtrees - offer them a scripted wrapper to do so - put the effort into that script so that it is simple to use and robust/working well such that the developer's workflow is effectively simplified. If you manage to do this (most of) the developers will see the benefit and they will use it.

For the push side - you need to take into account that mistakes in push will happen (even with the above-mentioned script, if it's not used by *all* developers, for example). Everyone makes mistakes.

Personally I favour *preventing* developers from directly pushing their changes into the integration branch and thus blocking possible mistakes from affecting the integration branch.

Instead I'd have all *candidate* changes (i.e. either pushed in separate branches or just as diffs) funnelled through a mandatory *centralized* pre-commit verification system. Automated, of course, with the proper git subtrees management made reliable through that automation. The changes would only be committed/merged into the integration branch automatically by this system, if they meet the quality checks.

This approach is applicable to *any* potential regression-causing mistakes, not just improper DTO subtrees. An example of such approach would be the gerrit-based development on OpenStack.

---
Tags: git
---


# Question
Title: How to replicate a mongodb pod with its persistent storage
## I made these settings :

```
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      labels:
        name: mongo-claim0
      name: mongo-claim0
      namespace: my-app
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 100Mi
    status: {}

```

---

```
apiVersion: v1
kind: ReplicationController
metadata:
  labels:
    name: mongo
  name: mongo-controller
  namespace: my-app
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: mongo
    spec:
      containers:
      - image: mongo
        name: mongo
        ports:
        - name: mongo
          containerPort: 27017
        volumeMounts:
        - mountPath: /data/db
          name: mongo-claim0
      restartPolicy: Always
      volumes:
      - name: mongo-claim0
        persistentVolumeClaim:
          claimName: mongo-claim0

```

---

```
apiVersion: v1
kind: Service
metadata:
  name: mongo
  namespace: my-app
  labels:
    name: mongo
spec:
  ports:
  - port: 27017
    targetPort: 27017
  selector:
    name: mongo

```

When I try to scale this pod, Minikube UI show :

> mongo-controller-xr21r -\> Waiting: CrashLoopBackOff Back-off restarting failed container Error syncing pod

And I got an error on the new pod :

> exception in initAndListen: 98 Unable to lock file: /data/db/mongod.lock Resource temporarily unavailable

Could you help me to be able to scale a pod with a persistent storage ?

# Answer
> 2 votes
https://www.mongodb.com/blog/post/running-mongodb-as-a-microservice-with-docker-and-kubernetes

> MongoDB database nodes are stateful. In the event that a container fails, and is rescheduled, it's undesirable for the data to be lost (it could be recovered from other nodes in the replica set, but that takes time). To solve this, features such as the Volume abstraction in Kubernetes can be used to map what would otherwise be an ephemeral MongoDB data directory in the container to a persistent location where the data survives container failure and rescheduling.

Like @Tensibai indicated the issue is related to the replication. According to this blog it could be solved as follows.

> ```
>  apiVersion: apps/v1beta1
> kind: StatefulSet
> metadata:
>   name: mongo
> spec:
>   serviceName: "mongo"
>   replicas: 3
>   template:
>     metadata:
>       labels:
>         role: mongo
>         environment: test
>     spec:
>       terminationGracePeriodSeconds: 10
>       containers:
>         - name: mongo
>           image: mongo
>           command:
>             - mongod
>             - "--replSet"
>             - rs0
>             - "--smallfiles"
>             - "--noprealloc"
>           ports:
>             - containerPort: 27017
>           volumeMounts:
>             - name: mongo-persistent-storage
>               mountPath: /data/db
>         - name: mongo-sidecar
>           image: cvallance/mongo-k8s-sidecar
>           env:
>             - name: MONGO_SIDECAR_POD_LABELS
>               value: "role=mongo,environment=test"
>   volumeClaimTemplates:
>   - metadata:
>       name: mongo-persistent-storage
>       annotations:
>         volume.beta.kubernetes.io/storage-class: "fast"
>     spec:
>       accessModes: [ "ReadWriteOnce" ]
>       resources:
>         requests:
>           storage: 100Gi
>
> ```

---
Tags: kubernetes, minikube
---


# Question
Title: Is is possible to configure settings in Bitbucket globally?
It is possible to configure settings per repository in Bitbucket. Would it be possible to configure settings globally? There are more than 30 repositories at the moment and I would like to avoid that I have to configure the same settings per repository.

# Answer
> 1 votes
At the moment this seems not to be possible in bitbcuket at the moment. Therefore I have created an enhancement in the Bitbucket issue tracker.

---
Tags: bitbucket
---


title: How to serve RESTful APIs?

text:
# Question
Title: How to serve RESTful APIs?
**Background:** I am developing a Web application for Employee Management. I use `Laravel` for building a RESTful API, and (maybe) `ReactJS` for the frontend.

**My problem:** I am confused on how to deploy my application. My current plan is to use the command `Laravel artisan serve` for the RESTful API, and use `Nginx` for serving HTML files. But is `Laravel artisan serve` possible for production? How to serve RESTful API?

**Some other thoughts:** As far as I know about `Node.js` applications. We can use `ExpressJS` for building a RESTful API, then `pm2 start app.js` for production. But `Node.js` is good at serving dynamic contents. If I build the RESTful API with `Laravel` or `Spring Boot`, or `Ruby on Rails` (`Sinatra`), I don't think a command like `pm2 start app.php (app.java or app.rb)` will work (here `pm2` is only for convenience, I know it is only for a Node.js app) because `PHP`, `Java` or `Ruby` is quite different from `Node.js`. I am afraid I am missing something fundamental here.

# Answer
The confusion might come from the concepts behind the tools you are using. So let's start with a little definition.

> Laravel and Artisan

**Laravel**

*Framework* for building a full MVC Web application in PHP. It is the equivalent of Spring-boot with Java or Rails with Ruby.

**Artisan**

Laravel's console driven interface for development. It provides tools such as the web server coming with PHP 5.4+. When using the "serve" command a simple web server will start. This is perfect for development, but I would suggest you use a server that gives you proper configuration flexibility in production such as Apache or nginx.

> Your other thoughts

**Java/Ruby/PHP/Javascript**

Are programming languages. Node.js is like an ecosystem that allows you to run client/server/middleware/tools Javascript based applications. Someone did a great job of comparing it with a Java ecosystem. Right here.

> But is `Laravel artisan serve` possible for production?

In theory, yes. It would require your development setup to be in production as well. Hence from this assertion, the answer becomes self evident for security, operability, configuration and performance to name a few. You will need a different configuration for production.

> How to serve RESTful API?

No matter which language you use, choose a library or a framework that will give you the tools to implement your APIs. Since Laravel is your framework, you can take a look at this tutorial to get started.
> 2 votes

---
Tags: deployment
---


# Question
Title: What are known component intelligence tools?
What are available - free and commercial - solutions to create reports about components used in a software project (physically, set of source code repositories), with intelligence on features like security situation and license model of used open source components?

One prominent example is - or was - Nexus Enterprise, but are there other solutions?

An open source monitor that comes to my mind is the Census project for security monitoring: https://github.com/linuxfoundation/cii-census

**Note**: my impression about Nexus Enterprise is about 2 years old, since then the product name and scope has changed afaik and there are also competitors like Seerene.

# Answer
<sub>(upon request from comments)</sub>

BlackDuck has tools to monitor use of open source components, and "manage open source compliance & security"
> 2 votes

---
Tags: security, nexus, n3dr
---


title: Docker and shared memory optimization?

text:
# Question
Title: Docker and shared memory optimization?
Imagine you would run a bunch of services say based on the same Tomcat image.

Would Docker optimize memory usage for identical memory segments and reuse these patterns?

# Answer
> 2 votes
This question is discussed in some detail in issue 7950 The conclusion seems to be that:

---
Tags: docker, performance
---


# Question
Title: Are there any guidelines for when to use gitlfs and will it break the CI?
Git large file system git-lfs could be used to store large files and at a txt file to the git repository. The question is when to decide whether a file should go to git LFS and if that is done whether the existing CI configuration will break.

Sometimes jars are added to a repository like the gradlew.jar. According to me, binaries like jars should not be stored in git as these are not text files that change, but a binary does not have to be large.

Currently we have multiple git repositories that contain jar files. I wonder whether all these jars should be moved to git-lfs.

# Answer
> 4 votes
**When to decide whether a file should go to git LFS**

Your files can go to git-lfs at any time, but it would be inneficient to implement this solution for few files. Specially if it's basic gradlew jars.

I have used the gradlew strategy previously. The idea is to push this "wrapper" along your code to make the build process portable. Any platform with the proper JRE will be able to build the application in a similar fashion. This removes the headache of enforcing teams to install specific versions of gradle.

The purpose of gi-lfs is to leverage the benefits of git along with the ability to store large files remotely by caching them and indexing them. It reduces the burden of downloading a big repository when binaries are stored.

In my opinion you should start consider this solution when:

* Your git repository contains large files (anything in the MB magnitude)
* They represent the main content of your repository
* These files are updated regularly
* It is taking your CI tool minutes to pull/clone the repository

If you consider this solution to track binary versions, I would suggest you go for binary repositories instead such as:

**Does the existing CI configuration will break**

If you decide to go git-lfs, the normal git commands won't work. You will need to either create symbolic links toward git-lfs after doing the setup or add `lfs` after `git` in your CI tool. i.e. `git clone` becomes `git lfs clone`.

---
Tags: continuous-integration, git-lfs
---


# Question
Title: Jenkins Exception that is caused by stdout redirection using windows cmd
I'm trying to execute an exec file from a jenkins server, using windows cmd. This is what is run from jenkins Windows batch window:

`C:\MyApp\MyApp.exe`

When run regularly from command line, the app prints to the console reporting its progress.

When I run it from jenkins, I get the following exception (MyApp stands for the name of my actual application):

```
08:13:56 Started by user NO
08:13:56 Building remotely on SFM BSP builder (ComputerName) (projectName) in workspace C:\Jenkins\workspace\MyBuildMachine
08:13:56 Running Prebuild steps
08:13:56 [MyBuildMachine] $ cmd /c call C:\Windows\TEMP\jenkins3066529473565168228.bat
08:13:56
08:13:56 C:\Jenkins\workspace\MyBuildMachine> C:\Users\NO\Desktop\MyApp.exe
08:13:56
08:13:56 Unhandled Exception: System.IO.IOException: The handle is invalid.
08:13:56
08:13:56    at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
08:13:56    at System.Console.GetBufferInfo(Boolean throwOnNoConsole, Boolean& succeeded)
08:13:56    at MyApp.App.AppInit(String[] args)
08:13:56    at MyApp.Program.Main(String[] args)
08:13:56
08:13:56 C:\Jenkins\workspace\MyBuildMachine>exit 255
08:13:56 Build step 'Execute Windows batch command' marked build as failure
08:13:56 Email was triggered for: Failure - Any
08:13:56 Sending email for trigger: Failure - Any
08:13:56 Sending email to: myEmail@email.com
08:13:58 Finished: FAILURE

```

Needless to say, the app didn't do its job. Trying to understand the problem, I found here an explanation about stdout redirection, and Jenkins actually does that automatically, which causes this exception. And indeed also when I run it in cmd with stdout redirection, i.e. `C:\> MyApp.exe >> appOut.txt`, it shows the same error.

I have no access to the source code of the app. I know that it could be solved from inside. My question is: Is there any way to prevent Jenkins from redirecting the stdout to a file, or wrapping the exec somehow such that will allow me to run it? (of course, running it from another exec and just catching the exception won't work. There won't be an exception but also the app won't run).

# Answer
> 2 votes
I found out how to do it without pipeline. It only has to do with command line script:

```
start "" C:\MyApp\MyApp.exe >> text.txt

```

This call is asynchronous (i.e., the command line won't wait for MyApp.exe to return). If you want it to wait, you can use the /w flag, like this:

```
start /w "" C:\MyApp\MyApp.exe >> text.txt

```

---
Tags: jenkins, windows, shell-script
---


# Question
Title: How to test VM provisioning script without provisioning
Currently I'm on a state that testing costs me money and a lot of time...

Background: I'm deploying VMs at softlayer and using a post-deployment script(bash) that will install every software that I need after the VM is ready. The problem is, I can only test this script by deploying one VM, and it's currently taking around 4h for the script to finish... So every change that I make I need to create a new VM(costs money) and wait for around 4 hours to see if the script is broken or not... This is becoming chaotic and I won't be able to move forward if I stay this way.

I need a new way to approach this kind of situation and be able to test the provisioning script more quickly and without needing to deploy a new VM every time.

Do you guys know some tool to help me on this scenario ?

# Answer
> 10 votes
I can see some options:

* Use Vagrant to create your VMs; it separates the process of creating the VM (including the base OS) and the actual provisioning. It also has some options to run certain provisioning steps at certain circumstances only.
* Use Ansible, Puppet or something like that to switch to a provisioning mode where you do not do the same stuff everytime, but only what is needed. This means that you can start the job, and at the first failed part, stop. Fix that part, then continue.
* Use Docker. This is slightly different than the Vagrant/Ansible approach in that it creates containers (which you don't actually need, as far as I can tell). It has the benefit, beyond the Ansible approach, that it gives you a very fine grained step-by-step development process. I.e., if one step fails, you still have all the images leading up to that, so during development, with a bit of discipline, you become very, very fast indeed.

All of these tools do much more than what you need, but all of them give you a way to do your job incrementally. Vagrant, Ansible and Docker are pretty easy to learn, as far as I'm concerned (as long as you are in Dev/Test mode, the "interesting" parts start when you go to production). Ansible is very minimalistic and needs nothing except a ssh connection. Vagrant and Docker might not be feasible in your infrastructure, you will quickly see.

# Answer
> 6 votes
http://www.vagrantup.com

You could use vagrant to deploy VMs on the local laptop.

You could also check whether it is possible to split the script in smaller parts so it will not take four hours to test it.

# Answer
> 5 votes
If testing locally isn't an option, then the most straight forward approach would be to use disk volume snapshots/backups to your advantage. These will still cost $$$, but will save you time in the long run. You should then separate your bash script into different working segments/scripts that can be tested individually. Once your server is provisioned, run a script, then take a snapshot. If it was successful, run the next script, take a snapshot, then rinse and repeat. If your script fails, modify the script, revert to the last successful snapshot, then try again.

NOTE: I'm not sure if you can take snapshots of virtual machine disks in IBM Cloud/Softlayer, but it looks like you can create a VM image pretty easily.

> Backing up virtual machine images
>
> You can back up a virtual machine image in your instance. This function creates a copy of the virtual machine image and cloud configuration that can be restored later. Additionally, you can manage these backup images. Details about the backup image are as follows:
>
> The backup image is an exact copy of the virtual machine image and the cloud configuration. No image cleanup is performed.
>
> * The backup image cannot be deployed as a new instance. It can be used only to restore the associated virtual machine image and cloud configuration.
> * Only the project owner (or an administrator) has access to restore the backup virtual machine images and backup virtual machine.
> * If you are using an OpenStack cloud, only one instance backup operation is allowed at the same time. If another user is running a backup and you start one on the same instance, you receive an error that states there is a conflicting request. To perform a backup, you must wait until the other backup is finished.
> * OpenStack PowerVM® and z/VM® instances do not support this action.
> * If the instance is deleted by using IBM® Cloud Manager with OpenStack, the associated backups are also deleted.

https://www.ibm.com/support/knowledgecenter/en/SST55W\_4.1.0/liacb/liacbsaverestorevsvmw.html

---
Tags: testing, provisioning, virtualization
---


title: DevOps in space related context?

text:
# Question
Title: DevOps in space related context?
If you compare the "mainstream" DevOps if there is such, is a DevOps practice in a spacecraft related context much different, or not? If this is too broad, take avionics software context as example.

> Avionic systems include communications, navigation, the display and management of multiple systems, and the hundreds of systems that are fitted to aircraft to perform individual functions (Wikipedia).

# Answer
The key difference between DevOps in the context of Avionics and, aircraft/spacecraft merely has to do with deployment velocity due to FAA certifications, Aircraft re-grooms and the way in which the Airline industry releases their changes.

But ultimately, DevOps is not exclusive to High Velocity delivery methods (like Agile). And can be done in the context of Waterfall methods or other similar methods.

In some ways, the airline industry is often factory oriented, which aligns well with Kim, Behr, and Spaford's depiction and conceptualization of DevOps.

Development on LRUs and racks can easily be done using an Agile methodology using a DevOps mentality. These many small changes can be stored up and released periodically to undergo final rack testing and certifications (such as AS9100). Airlines could be doing much more to leverage automation for software deployment to fleets and have yet to capitalize on these developments.

At the Avionics company I worked for for several years, we were seeing several successes and challenges with implementing DevOps. I simply can't emphasize enough however that challenges were internal and political, not technical or methodological challenges. These are also *not* systematic issues resulting from industry regulations, policies and standards. Avionics companies who are able to innovate and provide better value, higher reliability and cheaper costs will easily be able to out-innovate their competitors and gain traction and market share. The Avionics industry is ripe for this kind of shakeup and DevOps is well suited to be a cause of changes.
> 8 votes

---
Tags: culture, methodology
---


title: SSH tunneling on steroids

text:
# Question
Title: SSH tunneling on steroids
I have a remote redis machine with no ssh and no open ports except to the same lan + a remote machine (on the same lan) with ssh + my pc outside the lan. What I'm trying to do (mostly to see if it's possible) is `redis-cli on my pc -> ssh tunnel to the ssh machine -> something -> redis` so it would work. Is netcat + mkfifo the right way? Thanks :)

# Answer
Assuming the following:

* YOUR\_PC, has Redis client, SSH client, SSH access to SSH\_SERVER
* SSH\_SERVER, has SSH server, redis access to REDIS\_SERVER
* REDIS\_SERVER, has Redis server

Set up the tunnel from YOUR\_PC, port 1234, to REDIS\_SERVER:REDIS\_PORT, via SSH\_SERVER

```
ssh SSH_SERVER -L 1234:REDIS_SERVER:REDIS_PORT

```

On another terminal, on YOUR\_PC, run the redis client (based on https://stackoverflow.com/a/40678950):

```
redis-cli -h localhost -p 1234

```
> 6 votes

# Answer
We have redis connected through stunnel. That way you don't have to always establish the ssh connection, which can be a problem. Here is an article on how to setup redis through stunnel, which I won't copy into this answer. Feel free to edit.

http://bencane.com/2014/02/18/sending-redis-traffic-through-an-ssl-tunnel-with-stunnel/
> 2 votes

# Answer
This works, I wonder if there is a better solution. SSH tunnel + nc proxy

`ssh $ssh_server -L $port:localhost:$port "mkfifo /tmp/backpipe; nc -k -l localhost $port 0</tmp/backpipe | nc $redis_server $port | 1>/tmp/backpipe"`
> 1 votes

---
Tags: networking
---


# Question
Title: How to perform continuous Integration using Jenkins?
I've the basic idea about the Continuous Integration using Jenkins like- basic project setup, setting up source code management bit-bucket and build the project. I have done this so far and following the same approach, whenever any commit done in bit-bucket, Jenkins trigger the build and i get the build status But in my organization there is no expertise who clearly tell us about how CI tool actually used in real time. So we are not aware of the scope and expansions of the Automation project in terms of CI tool.

Can some one guide us about this and guide us to manage the release using Jenkins ? also somewhere the terms Jenkins pipeline confused me what it is and why it used for?

I'm working on Agile, So after completion on each sprint we have to commit out code in master branch only. We are planning to create new branch for each sprint. The vague in my is, we can create the new branch, sprint wise and also create the Jenkins job as per branch, So after committing the new branch Jenkins will trigger the build for that branch. It is the right approach ?

# Answer
Continuous *integration* means roughly what you said. Whenever something is committed, it is automatically built and tested, with a red/green light showing up at the end.

Continuous *deployment* takes this a step further and whenever a green light turns up, automatically deploys that.

Note that "tested" above can mean several steps: smoke tests (does it crash and burn right away), automated unit/feature/integration tests, and then, if you are not superbly confident, also maybe some final manual tests. Also, obviously, for CD, your test setup needs to be sufficiently conclusive so you can deploy with good confidence. You can also do the fully automatic deploy to internal testing/staging areas first, and then only do a production deploy at a later date.

To your questions:

> But in my organization there is no expertise who clearly tell us about how CI tool actually used in real time.

You need to figure that out for yourself as it depends very much on the persons, processes and software you are building/deploying. At first, just build the tools, work with them, and see what happens. If everything works splendidly, continue along the path you laid out yourself; if you hit problems, solve them as they come up.

> So we are not aware of the scope and expansions of the Automation project in terms of CI tool.

The more automation, the merrier. Automation not only means shaving off a few minutes here and there (but this factor also pays of *hugely* surprisingly fast), but also that everything that is automated, is codified in some script or tool; which means you are much less dependent on individual persons (who may become ill, overloaded, or leave the company) and scale much better if your team grows.

> manage the release using Jenkins ?

I view Jenkins one step "lower" than managing releases. In "pure" CI/CD, the final stage, there would be no releases anymore as every commit would directly and in real-time end up on production anyways. At the other end of the spectrum, you have something like 3 releases per year, each one of them late by 2 months ;). Every project will be somewhere along that spectrum, as dictated by circumstances (not the least of which are your stakeholders and the available money - running a very fast CI/CD is not free).

> the terms Jenkins pipeline confused me what it is and why it used for?

"Pipeline" is just a fancy word for saying that you commit - build - test - deploy. It's just figuratively speaking. The imagery is that you drop your commit into the pipe at the one end, and on the other end you get a running production server.

> So after committing the new branch Jenkins will trigger the build for that branch. It is the right approach ?

Sure, it's worthwhile to have every commit on every branch tested, by the default, if you have the resources for that. It lets you know exactly which build broke everything. Coupled with very small commits, this is a quite powerful tool.

Another useful trigger is to not build every commit, but have Jenkins look at the repository every 15 minutes or so, and build if there is one more more new commits. This may be more realistic when your test suites take longer to run, and if you do not have unlimited amounts of test environments that you can spin up and down regularly.

(By the way, if you are completely new to all of this, including "agile" and branching, then maybe it would be good for you to pick up an existing branching strategy, for example Gitflow, so you don't complicate things too much by (re-)inventing too much stuff at the same time.)
> 6 votes

# Answer
> We are planning to create new branch for each sprint.

It should be noted that this doesn't mean Continuous Integration, but rather just waterfall masquerading as CI, also known as CI Theatre.

Every branch you mention is nothing but a silo and merging it into the master branch can invalidate both the results obtained in the silo as well as those obtained so far in the master branch.

From the agile prospective you'd be constantly creating technical debt - the need to merge the silo branches into master, re-validating the merged result, identifying and fixing any problems that would appear after such merges. Never a known, quantifiable effort since you can't know right away which of the branch commit(s) is/are responsible.

The right approach would be to not pull such branches, instead every commit should be done on the master branch. You'd only have one Jenkins setup - on this master branch, executing verifications after every commit. No branch merges necessary, every commit is potentially ready for delivery/deployment once the CI execution is successfully completed.
> 2 votes

---
Tags: jenkins, continuous-integration, jenkins-pipeline
---


# Question
Title: Jenkins & Bitbucket; cancel previous build if new commit made to same branch?
We have Jenkins running unit tests when commits are made to our repo in Bitbucket. This is controlled by the Bitbucket Plugin, ie via a Bitbucket webhook.

Currently, if a commit is made to Branch A a unit test kicks off. If while that job is running a second commit is made to Branch A a second unit test will kick off, so there are now two unit tests on the same branch but with slightly different code.

**Our preferred behavior is that the first commit's test would be aborted when the second test kicked off, so that only the most recent unit test is being run. Can this be achieved?**

To clarify; We have many branches so we cannot just prevent concurrent builds, cancel the last as soon as the next starts, etc. Whatever method is used must specifically check if the *branch* already has a job running for it, not if the job in general is already running.

I've seen some trigger controls for Git, but not Bitbucket. I also found a script to check if the job's already running and cancel it if so, but as mentioned before that doesn't suit our use case. Am I missing something?

# Answer
**Note:** this answer comes from my background of building custom solutions, it's not a config-only one, which, if available, would obviously be preferable. But it's maybe something to consider otherwise.

You could teach the script executing the unit test job to persist a per-branch execution state.

When the script is launched it would need to obtain the branch name from its invocation parameters. It would then obtain the persisted state for the branch.

If the state is either missing or `not running` it would set it to `running + job ID`, execute the unit test and then set the state to `not running` (or delete it).

If the state is `running + job ID` it means another unit test is already running and you have the corresponding job ID. The script can then (gracefully?) terminate the already running job by its ID and take it's place, updating the state to `running + new job ID`.

But IMHO terminating a job in progress (which may be very close to completion) is not ideal - it leaves room for constantly killing jobs and launching news ones only to be, at their own turn, killed by subsequent jobs - all without actually completing any of them.

Instead I'd modify the logic to:

* leave the job in progress alone and instead just record in the persisted state a `pending job ID` (either a unique value, overwritten to reflect the most recent pending job submitted or a list of all pending jobs, sorted by submission time)
* when the job in progress completes it checks the `pending job ID` values and, if any, launches the most recent one (or the oldest one, or even follow some other selection logic, if using a list)

This approach avoids wasting unit test resources and can easily be adjusted even for more than one but still limited resources.
> 3 votes

---
Tags: jenkins, jenkins-plugins, bitbucket
---


# Question
Title: How to apply using Terraform to launch multiple EC2 Resource with different configs (VPC not maintained by TF)
We are planning to use Terraform for provisioning EC2 instance which will host Oracle DB. While we are able to achieve this, there are some questions for which I am searching for suggestions / feedback's.

**Below is the directory structure we are planning:**

```
|aws
  |Environments
    |STAGING
        |Modules
           |Compute
           |Networking

    |PRODUCTION
        |Modules
           |Compute
           |Networking

```

**Questions:**

* We are able to create EC2 instances using existing VPC. As VPC is not something that will be owned by my team. Is it enough to keep VPC ID, SUBNET ID, etc as variables in EC2 resource or is there a best practice to keep them in modules for existing VPC ? Overall, our tf codes are not going to manage the VPC's, just use what is being giving to us, but want to make it dynamic so that changes are simple, yet DRY
* We are planning for Continuous Development, meaning, developers / requester will have the ability to request for a new ec2 instance with certain version of database. Right now, we are installing db with bootstrap scripts. Though this works as expected. Increasing count for more than 1 instance works, but what if the configuration slightly changes.

> **For example:**
>
> 1) If I need more capacity on root volume for the new instance, changing the existing resource shows a plan that will destroy existing resource and recreate. **How can we achieve this ?** Is duplicating the **aws\_instance** resource, the only way when the configuration changes for new EC2 instance?
>
> 2) If new instances launch as t2.large(Instance type passed as variable) while old ones run on t2.micro, will it recreate old instances too if I just increase the count

*Also, is there a way to automate this process i.e. append something like a template for new EC2 instance to existing .tf file.*

**What do we have now**

***Main.tf***

```
provider "aws" {
  access_key = "${var.access_key}"
  secret_key = "${var.secret_key}"
  region     = "${var.region}"
}

resource "aws_instance" "oracle_ec2" {
  ami           = "${lookup(var.ami, var.region)}"
  instance_type = "${var.instance_type}"
  key_name      = "${var.key_name}"
  subnet_id     = "${var.subnet_id}"

  vpc_security_group_ids = ["${aws_security_group.oracle_sg_new.id}"]

  root_block_device {
    volume_type           = "gp2"
    volume_size           = 10
    delete_on_termination = true
  }

  ebs_block_device {
    device_name           = "/dev/xvdb"
    volume_type           = "gp2"
    volume_size           = 200
    delete_on_termination = true
  }

  count = "${var.instance_count}"

  tags {
    Name = "Ora"
  }

}

```

# Answer
> 6 votes
You can use a remote backend as a data source. That is working well for us thus far but this project is not very mature and we will likely refactor a time or ten.

```
data "terraform_remote_state" "network" {
  backend = "s3"
  config {
    // some variables related to config would likely go here
  }
}

```

then for the teams not managing the network to consume that data you could create an instance like (special attention to the `subnet_id` here):

```
resource "aws_instance" "oracle_ec2" {
  ami           = "${lookup(var.ami, var.region)}"
  instance_type = "${var.instance_type}"
  key_name      = "${var.key_name}"
  subnet_id     = "${data.terraform_remote_state.network.subnet_id}"
}

```

The network team should be able to manage the networking infrastructure and you should be able to consume those values based on current state.

Terraform Remote State Data Source

> 2) If new instances launch as t2.large(Instance type passed as variable) while old ones run on t2.micro

If you change the instance type from `t2.micro` to `t2.large` then yes the `t2.micro` instances will be destroyed and replaced with the `t2.large`. The name of the game is immutability outside of meta data changes like altering tags. Look into lifecycle hooks to alleviate some of this pain

```
lifecycle {
  create_before_destroy = true
}

```

> will it recreate old instances too if I just increase the count

No, current instances would not be destroyed. Terraform will check the state to see how many instances you currently have. If more instances are necessary, terraform will know how many instances to create and create the instances necessary

> Also, is there a way to automate this process i.e. append something like a template for new EC2 instance to existing .tf file.

Not sure what exactly you mean by this one. What I will say is that having the data coupled to the instance itself makes instance immutability and using terraform more difficult. A (untested and unproven) thought and potential solution in my head is to:

```
//create an ebs volume
resource "aws_ebs_volume" "oracle-data" {
  ...
}

//attach that volume to the instance
resource "aws_volume_attachment" "oracle-data-attachment" {
  ...
}

```

with some bash involved for an init script that creates and mounts the new volume to be used in conjunction with the `aws_volume_attachment` resource. That could potentially create a new instance preserving the EBS volume then reattaching it to the new instance.

---
Tags: amazon-web-services, terraform, amazon-s3, infrastructure-as-code
---


# Question
Title: Does Ansible have component akin to PuppetDB?
I know Puppet from (limited) experience and have noticed that in configuration management there is now a strong trend towards Ansible.

On the one hand, I understand that Ansible does not require an agent, because it makes intelligent use of `ssh`.

On the other hand, these are features have come to like about Puppet:

* access to system-wide configuration state and history (PuppetDB) via REST API
* ability to keep backups of overwritten files (filebuckets)
* ability to encrypt part of the Hiera configurations (`.eyaml`)

Among these, PuppetDB seems to me the most important and useful (e.g. for integration with other tools). Therefore my question is this: Does Ansible have something akin to PuppetDB, i.e. a component that offers an API where it could e.g. be asked "What packages are installed on host *x*?" or "On which hosts is package *y* installed?"

(This question was migrated over from StackOverflow).

**UPDATE** Significant downside of Puppet it my experience so far: Not so much the fact that it requires an agent (from what I have seen, Ansible's use of Python also introduces an agent of sorts in the form of a Python interpreter ;-), but that it wants its agent to act as `root` only and always.

# Answer
> 5 votes
The direct answer is "no".

Ansible does not have any database, because it is supposed to be run by different operators from different computers and it's purpose is to be sure that target systems are in the exact state, as specified in playbooks.

There's an option to attach external facts cache backends to Ansible. So you can setup redis as your cache, Ansible will populate it with gathered facts during playbook run and you can later query redis for them. But it is very limited solution, because there are not too much facts gathered by Ansible, for example there are no information about installed packages – you will have to gather them yourself an use `set_fact` with `cacheable` option to put them into external cache.

---
Tags: ansible, configuration-management, puppet
---


# Question
Title: How to update all Bitbucket Repository Issues at once?
**Background:** There are multiple Bitbucket Repository Issues (BRI). If an issue needs to be edited then an edit button could be clicked and several items, e.g. title and priority could be changed. This works well when some issues need to be changed, but when more than 250 issues need to be changed this approach is tedious.

**Aim:** to update all BRI at once.

**Method:** The current method is to update the issues manually, but when multiple issues need to be updated then it is time consuming. Another approach could be using the API.

*Consulting the API:* `curl -X POST -u "<TOKEN>" https://api.bitbucket.org/2.0/repositories/<username>/<repository-name>/issues?` did not return any issues.

In order to update all the issue the following was tried:

```
curl -H "Authorization: Bearer <access-token>" https://api.bitbucket.org/2.0/repositories/<username>/<repository>/issues -d "priority=trivial" -X POST

```

Instead of updating all BRI, the following was returned:

```
{"type": "error", "error": {"fields": {"title": "required key not provided"}, "message": "Bad request"}}

```

**Discussion:** As mentioned in the method section, perhaps the API could be used to close all BRI at once, but according to the API documentation there seems to be no update option.

*Why create this Q&A on DevOps?*: The reason is that the ultimate goal is to close an issue in bitbucket by Jenkins when a pull request has been merged. If updating issues using the API works then that will hopefully work as well.

**Conclusion/question:** It is possible to update all BRI at once?

# Answer
> 1 votes
At the moment it seems not be possible to update all issues at once using the API and therefore this issue has been created.

---
Tags: bitbucket
---


title: What are signs to hire a CIO?

text:
# Question
Title: What are signs to hire a CIO?
If for example the number of complex Java issues is over growing exponentially on the backlog this could be a sign to hire a Senior Java Engineer, but what are signs for hiring a CIO?

# Answer
The sign to hire a CIO is that your CEO is not able to handle the relevant tasks on his own, plain and simple. The reasons for that may be manifold - maybe he has too much on his plate, as the company grows, or he does not have a sufficiently technical background, etc.
> 1 votes

---
Tags: culture
---


title: What are Pros and Cons of being a CIO?

text:
# Question
Title: What are Pros and Cons of being a CIO?
What I have heard of Chiefs is the following:

**Pros**:

1. Bigger influence of decisions that will be made

**Cons**:

1. Less time to be technical. Most of the chiefs told me that they only can spend a maximum of 10% of their time to be technical.

What are other pros and cons of being one of the Chiefs or are some chiefs, e.g. CTO, CIO, COO spending more time to fix technical problems by themselves?

# Answer
A CIO is part of the company leadership. His primary duties is to make policies, create organization (i.e., structure the company, at least the part that reports to him), make strategic decisions etc. In a large company, many technical people report directly or indirectly to the CIO; if there are technical problems to solve, he is supposed to hire people to solve those problems, or - more likely - have his direct reports hire those people.

I would be *very* wary if a CIO starts fixing technical problems himself. This probably would mean that he has trouble delegating, or trusting his employees.

There may be a phase in a very small company (like 5-10) where you might have some kind of senior technical person who starts to grow into the CIO role. But that should certainly be a path that develops itself (often it is quite obvious who is a good fit for that role and who isn't), and nothing where you start pondering whether you want to become CIO or not out of the blue.
> 2 votes

---
Tags: culture
---


title: Debug Docker DNS

text:
# Question
Title: Debug Docker DNS
In a stack with higher number of services it seems that Docker DNS does not work anymore which leads to effects that container's names are not known even to themselves.

While I have filed this bug, I wonder whether there is a way to debug this more extensively to "prove" the DNS error, or for example manually "fix" the DNS name.

Will be there some deeper error message?

**Note**: The ping from outside does not work; I just find cumbersome that is not possible to ping your own DNS name.

# Answer
There is a comprehensive document about k8s' DNS. According to this document one could validate whether the DNS is working by running:

busybox.yaml

> ```
> apiVersion: v1
> kind: Pod
> metadata:
>   name: busybox
>   namespace: default
> spec:
>   containers:
>   - image: busybox
>     command:
>       - sleep
>       - "3600"
>     imagePullPolicy: IfNotPresent
>     name: busybox
>   restartPolicy: Always
>
> ```

and deploy it by issuing:

> ```
> kubectl create -f busybox.yaml
>
> ```

Once deployed, one could run:

> ```
> kubectl get pods busybox
>
> ```

and validate whether the DNS is working:

> ```
> kubectl exec -ti busybox -- nslookup kubernetes.default
>
> ```

There are additional validation steps that could be executed, including:

> ```
> kubectl exec busybox cat /etc/resolv.conf
>
> ```

verify the DNS policy, check whether the DNS pod runs, checking erros in the DNS pod:

> ```
> kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c kubedns
> kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c dnsmasq
> kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c sidecar
>
> ```

does the DNS service run?

> ```
> kubectl get svc --namespace=kube-system
>
> ```

exposed DNS endpoints?

> ```
> kubectl get ep kube-dns --namespace=kube-system
>
> ```

There are also multiple known issues regarding the k8s' DNS:

> Linux’s libc is impossibly stuck (see this bug from 2005) with limits of just 3 DNS nameserver records and 6 DNS search records. Kubernetes needs to consume 1 nameserver record and 3 search records. This means that if a local installation already uses 3 nameservers or uses more than 3 searches, some of those settings will be lost. As a partial workaround, the node can run dnsmasq which will provide more nameserver entries, but not more search entries. You can also use kubelet’s --resolv-conf flag.
> 2 votes

---
Tags: docker, dns, networking, virtualization
---


# Question
Title: How to setup testkitchen with ansible-provisioning from localhost for different vagrant-vms?
As pointed out here, testkitchen is in principable usable with ansible if the right plugins are installed.

Now I've done all the installing and am at the point, where I wonder, what I missunderstood:

As a driver I use vagrant and I thought it might be a good idea to use the vagrant-host as the machine which does the ansible-provisoning since the ssh-access is obviously cared for by vagrant and python is installed on my boxes per default. Now, when I look at my logs, I see:

> Setting up ansible (2.4.1.0-1ppa~xenial) ...

How can I prevent the installing of ansible? How can I tell testkitchen to use localhost as the machine which provisions the vagrant vms?

To be more clear, I am trying to find a configuration for testkitchen (as seen e.g. here under "Example .kitchen.yml file") that can accomplish this.

So far I got this configuration in my .kitchen.yml:

```
[...]
provisioner:
    name: ansible_playbook
    hosts: localhost
    require_chef_for_busser: false
    require_ruby_for_busser: false
    playbook: server.yml
[...]

```

This still installs ansible on the vagrant boxes, which I don't want.

# Answer
> 1 votes
According to this documentation it is possible to install ansible and run a ansible playbook by using the following snippet:

> ```
> Vagrant.configure("2") do |config|
>
>   #
>   # Run Ansible from the Vagrant Host
>   #
>   config.vm.provision "ansible" do |ansible|
>     ansible.playbook = "playbook.yml"
>   end
>
> end
>
> ```

---
Tags: ansible, testing, vagrant
---


# Question
Title: Docker's value and scenarios in native Windows environment
AFAIK, typically Windows enterprise stack components are quite monolitic services, like Windows IIS and MS SQL Server. Still, there are "Windows containers".

* Are they indeed runnable in containers and this makes sense?
* Are there other Windows native containerization scenarios?

In this example Microsoft refers to a ".NET-nanoserver" image. The base image is there a ".NET runtime" - I consider this is something else than an OS base image.

# Answer
> 2 votes
> Are they indeed runnable in containers and this makes sense?

According to this documentation it is possible to run windows in docker. Yes it could make sense as this would be comparable to the positive effects of running linux in docker. Note that:

> Disclaimer: This lab is still in work, and is based off of the blog, but you can test and leverage the example walkthroughs now, if you want to start experimenting. Please check back as the lab evolves.

Whether it makes sense: From a Linux perspective it does not as a Windows container will be significantly larger than a Linux one at the moment. From a Windows perspective it does not make sense as well as Windows itself is not as lightweight at the moment as Linux. Imagine that one runs 50 war files on a dedicated Windows server at the moment. If one would use Windows Docker containers for that purpose then a dedicated server should be used as a base image, each war file should be run in a separate container. This means that the number of required resource would grow exponentially. I assume that the creators of the native Windows Docker container take this into account and that this issue will be solved in the future.

---

> Are there other Windows native containerization scenarios?

What is the definition of a microservice? One could also argue that running SQLServer and IIS are microservices if one compares it with running IIS and SQLServer and other Windows services on a Windows server instead of separately. If one compares it with running Linux in docker then one could use a windows server as a base image by defining `FROM windows` in a dockerfile and run tomcat in a Windows container instead of running it on a dedicated server.

---
Tags: docker, windows, dotnet
---


# Question
Title: What are formal criteria for establishing declarative pipelines, or pipelines as a code?
Stepping from "crafted" DevOps toolchains to declarative pipelines is imo next transformation step towards even more productivity which requires additional effort.

Any experiences what can be the "critical mass" where this investment is a clear pay-off and not an overkill?

# Answer
> 2 votes
**Pipeline as Code (PaC)** means that the CI/CD configuration will be stored in the VCS instead of in the CI/CD tool itself, e.g. configuring it manually by using a UI.

> Any experiences what can be the "critical mass" where this investment is a clear pay-off and not an overkill?

Imagine that more than 100 jobs have been configured manually the last couple of years and one would like to transform all these jobs to PaC then it would mean that less resources will be available to create new features and therefor not advisable, but if one would create tasks to address this and transform a manual job to a PaC one every sprint then it could be beneficial in the long run.

**Advantages of PaC**

*No endless searching for configuration anymore*

One advantage is that one does not have to look in various places, e.g. CI/CD tool, documentation how a CI/CD job has been configured as the relevant stuff resides in the same place as the code, i.e. VCS.

*No endless and unnecessary debugging anymore*

Another advantage is that someone cannot break the CI/CD job easily like it is possible when it is configured using a UI. If for example someone has to change the CI/CD could then one has to change the code in the VCS. As every code change could be tracked, it will be immediately clear who changed the pipeline and what was changed when a pipeline is broken.

---
Tags: continuous-integration, build-pipeline, methodology
---


# Question
Title: Application monitoring in multiple EC2 instances
I am running 10 AWS EC2 instances. Each of them running different applications. I am looking for application level monitoring solution. Suppose, if Apache fails in any instance, it should notify me with logs. I know about "Nagios" but it is mainly for infrastructure monitoring. There are few tools for application monitoring (like, "Monit") but I have to install them manually in each server, which is not possible when you are having multiple number of EC2 instances (A global set up would be better, because number of instances will keep increasing).

# Answer
> 2 votes
It looks like your problem is two fold:

1. You want to monitor the status of critical applications provisioned on virtual machines
2. You want to be able to install these checks rather intuitively.

I'll answer these in the reverse order.

The easiest answer for #2 is that you can use configuration management like Chef, Ansible, or Puppet to provision machines. That way you could write a cookbook that as a software package is detected on the system a simple agent based check is dropped in as well. Codifying these things makes them much more manageable in the long term.

In terms of #1 you're probably looking for an agent based monitoring tool. Some examples of this would be Prometheus or Sensu but there's an endless list of them out there. These allow you to describe checks on the server and the master just collects whatever information is given to it.

---
Tags: amazon-ec2, monitoring
---


title: How to become a CIO?

text:
# Question
Title: How to become a CIO?
According to this blog, one could become a CIO if one meets the following criteria:

1. BSc in Computer Science
2. Proven experience in Project Management
3. Specialized in an certain area by becoming certified
4. IT Governance and Risk Management
5. MBA

**Discussion**

I wonder whether the above mentioned criteria would be sufficient to become a CIO. I think that soft skills are important as well to be promoted to such a function right? I do not think whether only mastering hard skills would be sufficient to become a CIO.

# Answer
> 1 votes
I really hate this kind of discussion, as if there's a mold that fits a good CTO or CIO.

Let's examine the qualities of a good executive rather:

1. Practices enabling and empowering engineers and local team leadership
2. Deflects bureaucracy from external sources
3. Helps remove organizational blockers to change and improvement
4. Does not mandate architecture from the top down
5. Acts as an evangelist for software first solutions
6. Promotes automation and solid foundations for solving problems rather than implementing patches
7. Knows how to avoid unrealistic deadlines and demands quality, but not perfection, over quantity of features
8. Uses data to solve problems and constantly evaluates the integrity of the data gathering
9. Values iterative development rather than long, cumbersome planning processes
10. Values education and internal training & advancement over outsourcing

I'll add some more if I can think of them. We need to stop assuming a CTO or CIO is a list of resume qualifiers and start hiring people who really understand the process of building, maintaining, and delivering software from sun rise to sun set.

---
Tags: culture
---


# Question
Title: Where does TFS 2017 actually store artifacts?
We had been using a TFS build process that copies build output to some fileshare for later use. But with TFS 2017 we want to use that fancy "Publish Artifacts" build stage that has the TFS server provide a .zip file of the artifacts via the RESTful API.

But now I want to be careful about where all these build artifacts are going to be stored and what disk space to keep an eye on. Do these build artifacts just become massive binary blobs in the SQL database that TFS uses? Can we configure what location TFS uses to hang onto these artifacts? Most documentation I've found just describes the artifacts from an external perspective with the implementation abstracted away.

# Answer
There are two options when publishing artifacts: Server and File Share.

If you publish to the server, they are stored in the TFS database. You can't configure the location, however you **can** configure retention policies to ensure that old builds aren't kept around forever.

If you publish to a file share, they're stored on, well, a file share.
> 5 votes

---
Tags: artifacts, tfs
---


# Question
Title: Does TFS Build have something like a Jenkinsfile?
I've seen some custom settings files used in combination with TFS build, but nothing native. Does Team Foundation Server or Visual Studio Team Services have a Jenkinsfile-like, declarative method for defining a build process?

# Answer
> 4 votes
The strict answer is no, in that there is no direct equivalent to a `Jenkinsfile`. However, there are Build Process Templates and you can use the Psake Build Automation tool in combination with templates to reduce the barriers to entry for creating a new build, test and deployment build definitions.

It is nowhere near as fluid as Jenkins' Pipelines as Code, AppVeyor's `appveyor.yml`, CircleCI's `circleci.yml` and Travis' `.travis.yml`. There is a wealth of information on the ALM Rangers Library of tooling and guidance solutions.

# Answer
> 4 votes
Visual Studio Team Services has a YAML Build feature in public preview that allows you to store your build definitions as source controlled files.

---
Tags: jenkins, tfs, jenkinsfile
---


# Question
Title: Jenkins - sending email after build with the build version as a subject
I would like to send an email at the end of my build to inform success or failure. I'm using the email ext-plugin for Jenkins. I want the version number of the build to be the subject of the email.

The problem is that the version is something that changes in each build, so I need to somehow edit an environmental variable to hold the version number each time (I have a script that saves it into a file). This is the default configuration:

I would like to have, for example, set the version number into a variable using windows batch script, like this:

```
rem # Loading version File number
set MY_BUILD_VERSION=<myVersionFile.txt
rem # Loading content
set MY_BUILD_CONTENT=<myContentFile.txt

```

And using it in the email plugin, like this:

The problem is that the environmental variables aren't recognized outside the .bat file they are being run from, although I'm using the envInject plugin, the documentation says that what's run from a script won't be recognized outside of it.

Is there any way to load data into the email plugin from a file, or any other dynamic way to load my version number dynamically?

# Answer
I found a way to set environmental variables globally from a file! I'm using envInject for loading variables from a file.

For example, lets say I have a file that is called "versionNumber.txt" and that it's created during the build process. Maybe I update it in every build, or any other way. then I create a prosFile that can be identified by the envInject, by running a batch script:

```
set /p VERSION_NUM=<versionNumber.txt

```

Let's assume my version number was 1.5.5.10. so now my VERSION\_NUM equals that string. now VERSION\_NUM has the value that I want. But it's local, it won't help me. so I rewrite it to a file in the following format (also, using a batch script):

```
echo VERSION_NUM=%VERSION_NUM% > C:\Temp\prosFile

```

now I have a file that contains the following content:

```
VERSION_NUM=1.5.5.10

```

And now I just need to load the file using the envInject plugin:

And now I have an environmental variable named VERSION\_NUM that can be used anywhere in Jenkins. But I wanted to make it the subject of my email, so I'll simply do (in windows, in linux it might be different):

That's it!
> 1 votes

# Answer
You should be able to pass the current params as an argument to the batch file and then assign them to a variable used in the batch file. You can have this step as the post-build step before you call the email plugin.
> 0 votes

# Answer
Sorry.. the previous post was incomplete as I hit enter. Though you have solved the issue, I'm pointing out how you can capture env variables into the batch file.

Use the same method to read the contents of the file into a variable:

`set /p VERSION_NUM=<versionNumber.txt`

and when calling the batch file:

`call catchmyvar.bat %VERSION_NUM%`

and in the batch file 'catchmyvar.bat\`, set the parameter to another variable that's local to your batch file:

`SET VER_NUM=%1`

This should ensure you can capture the env variables set outside the batch file into the current context
> 0 votes

---
Tags: jenkins, windows, shell-script
---


title: What Chief is accountable for DevOps?

text:
# Question
Title: What Chief is accountable for DevOps?
CIO

> Chief information officer (CIO), chief digital information officer (CDIO) or information technology (IT) director, is a job title commonly given to the most senior executive in an enterprise responsible for the information technology and computer systems that support enterprise goals.

and

> CIOs manage a business’s IT systems and functions, creates and delivers strategies and policies, and places great emphasis on internal customers.

CTO

> A chief technology officer (CTO), sometimes known as a chief technical officer, is an executive-level position in a company or other entity whose occupation is focused on scientific and technological issues within an organization

and

> CTOs place emphasis on the external customers to the organization and focus on how different technology can make the company more profitable.

COO

> The COO is responsible for the daily operation of the company, and routinely reports to the highest ranking executive, usually the chief executive officer (CEO). The COO is usually the second in command at the firm, especially if the highest ranking executive is the Chairman and CEO.

**Discussion**

As DevOps is closing the gap between several disciplines, e.g. Development, Operations, Quality Assurance one could argue that the CIO is accountable for DevOps as all these parties are internal customers. However, as a CTO is also focused on the outside world, one could also say that the CTO is accountable as he or she could implement DevOps because he or she heard in the outside world that this discipline was beneficial for some companies.

# Answer
> 2 votes
From my experience (where I currently work and another company where a friend works at) the DevOps team's report to the CIO.

Also to note that our DevOps team works a lot with, mostly **alongside** the CTO on a "day to day" basis compared to the CIO.

# Answer
> 0 votes
It all very much depends on the type of organisation. We don't have a CDO or CIO, so I report to the COO who manages IT as well.

The important thing is to bring digital, IT and DevOps together.

---
Tags: culture
---


# Question
Title: How to hot deploy to Tomcat with Ansible
Currently I build my app with maven and use the tomcat manager to hot deploy the war file (both on windows servers).

Now I want to use ansible to move away from the manual process.

1. Would it make sense to build with ansible? My assumption is, that it does not.
2. How could I use the tomcat manager to hot deploy, triggered by ansible. I have searched the web but I did not find a solution.
3. Is there an other option to hot deploy the war file, which I might have overseen?

# Answer
> 2 votes
At the bottom line I found this method as stable and reliable, in case you want to deploy via ansible.

```
### This play deploys a webapp war to an tomcat server

- name: Copy the war file {{ webapp_name }}.war as {{ webapp_name }}.war.tmp to {{ webapp_destination_folder }}
  win_copy:
    src: files/{{ webapp_name }}.war
    dest: "{{ webapp_destination_folder }}\\{{ webapp_name }}.war.tmp"
    force: yes

- name: Remove current war file {{ webapp_destination_folder }}\{{ webapp_name }}.war
  win_file:
    path: "{{ webapp_destination_folder }}\\{{ webapp_name }}.war"
    state: absent

- name: Wait until the webapp folder {{ webapp_destination_folder }}\{{ webapp_name }} is deleted
  win_wait_for:
    path: "{{ webapp_destination_folder }}\\{{ webapp_name }}xxx"
    state: absent
  register: folder_info

- name: rename the {{ webapp_name }}.war.tmp to {{ webapp_name }}.war
  win_command: "cmd.exe /c rename {{ webapp_destination_folder }}\\{{ webapp_name }}.war.tmp {{ webapp_name }}.war"
  register: cmd_result

```

# Answer
> 1 votes
I've got your point. Ansible is not the right choice for hot deployment to get rid of manual work and deploy the war files continuously to a server or where ever you want. Ansible is a configuration management tool. For this, you need an Automation deployment tool which is Jenkins. You need Jenkins in order to automate this. Create a Job, install a maven plugin within Jenkins and select the maven file before deploying the Code.

For Tomcat, You need to define about the tomcat instance within Jenkins and write the script in such a way it will automatically deploy the war file everytime you commit the code.

Best Practice, You better get a GIT repository and push your code there. Connect GitHub with Jenkins. The moment you will commit the code to GitHub it will automatically deploy the changes within tomcat through Jenkins and will Run the code.

---
Tags: ansible, deployment
---


title: Why to overrule role vars?

text:
# Question
Title: Why to overrule role vars?
There is quite a simple setup I am sure many of us have come across:

1. You have lean, slim and dedicate roles
2. Every role has its own role vars
3. Playbooks include the roles

Let's assume you have a base path for an application, most likely you will need this path in many roles. To keep the roles independent you would define this base path in every role vars where needed.

Now you include some roles in a Playbook and in case of a change on e.g. the base path you need to change that in every role.

One help would be to use extra vars to overrule the role vars, but what if there are many potential candidates for a change?

So I would like to be able to `include_vars` in a playbook to overrule the variables in the roles. But this is not possible as:

1. `include_vars` is not allowed in Playbooks
2. `var_files` (what would work in Playbooks) will be over written by role vars

---

**Question:** how do you do solve this situation? Maybe I want something odd and there is a much simpler way?

# Answer
Variables have different priorities, see vars precedence.

As of Ansible 2.x:

> role defaults
> inventory file or script group vars
> inventory group\_vars/all
> playbook group\_vars/all
> inventory group\_vars/\*
> playbook group\_vars/\*
> inventory file or script host vars
> inventory host\_vars/\*
> playbook host\_vars/\*
> host facts
> play vars
> play vars\_prompt
> play vars\_files
> role vars (defined in role/vars/main.yml)
> block vars (only for tasks in block)
> task vars (only for the task)
> role (and include\_role) params
> include params
> include\_vars
> set\_facts / registered vars
> extra vars (always win precedence)

For roles you would generally use `myrole/defaults/main.yml` for easily overridable vars (e.g. installation path or smth) and use `myrole/vars/main.yml` for vars that you don't want to be easily overridden (smth like required packages).
> 2 votes

# Answer
I missed one point which provides the answer: role defaults are the lowest in precedence. So just these vars and file\_vars can overwrite in the playbook.
> 0 votes

---
Tags: ansible
---