# Question
Title: Best practices on how to erase everything in destination deployment folder and restart the app after deployment using Appveyor
So far I have managed to get a simple deployment procedure running via SFTP. I really need to make two more steps:

1. Erase everything inside the folder that I am deploying to before I deploy a fresh build into it or (even better not sure this is possible) see what has changed and upload only the required and erase the not required.
2. Restart the app after deployment is done.

What could be the best solution for this with Appveyor? So far I see the easiest way is to possibly include a power shell script that will connect via ssh and clear out the folder, not sure how that would work with restarting the app though. Would be really great to hear some more experienced users about this solution.

# Answer
So it actually only took one line of code:

1) First take the SSH public key from the settings page inside the appveyor project and make sure you put it into the authorized\_keys inside user\_that\_you\_are\_going\_to\_login\_with\_from\_appveyor/.ssh/authorized\_keys

2) To clear the folder before deploying to it on a Ubuntu server

```
before_deploy:
  ssh user@host -t "rm -r -v /pathtofolder/*;exit; bash --login"

```

3) Now about the restart of the app I wanted to use after\_deploy however there is a problem with that. After deploy fires straight after deploy and doesn't wait untill upload is finished. I wanted to use it to run

```
systemctl restart myapp

```

However the problem seemed to solved itself because systemctl restarted my app at the moment when it was substituted with a new one or maybe at the moment when I call it the first time after the files have been reloaded. So far it seems I have a new app running after I run the script.
> 1 votes

---
Tags: deployment, appveyor
---


title: What is DevOps?

text:
# Question
Title: What is DevOps?
Please bear with me as this might get a bit deep. I’ve been reading though some documentation of a error logging SaaS I’m using in an application I’m building.

Quite a few times they gave me the impression that DevOps is sort of the responsibility of all Developers. E.g it’s something someone implements as part of a project, vs DevOps being a title/role in a business... I’d akin it to... Imagine, me as a developer having to write a micro-service because it’s required to be effective, vs another guy walking around calling himself a “Micro service” doing only micro services.. sounds a bit strange right?

I can see merits to both and if anything I know the lines are pretty blurry at the best of times in all dev fields anyway... my best guess is maybe a DevOp would be more specialized?

**Question** Based on the above...

What is the expected role of a DevOps person then, vs say a Developer, if going by the school of thought that DevOps is the responsibility of all devs?

... like what’s there left to do then for a DevOp then?

# Answer
> 7 votes
There are two strongly connected but still a bit controversial things in the industry that makes this kind of question being asked all the time since 2010.

First one is a nature of DevOps methodology itself. It was born all around idea of better collaboration between different parties of software delivery process. This involves a lot of non-technical aspects, like shared responsibility, better communication, knowledge sharing. Another improvement that was aimed - was product delivery itself, starting from designing software and up to deployment and operating live systems. So it brought us technical improvements as well. There was nothing new, actually, as all those things you see in majority of “DevOps job openings” like CI/CD, configuration management, log management, monitoring, whatever – were existing for decades, before even term DevOps was used for the first time. So DevOps as methodology just put all this bits to one plate.

And here comes the second thing: hype. Popularity of the methodology arisen - as it was intended to solve problems that majority of industry players had more or less. So some people started to see non-existent “silver bullet” here. Some magical potion, which simply could solve issues they have. So we got high demand here. And as a result – supply started to grow as well. And obviously businesses were looking for someone who could implement “DevOps” for them. How do you usually initially identify someone who could work with some tool/methodology/whatever? I bet - the answer is - by looking for appropriate badge in their CV:)

People realized, that being badged as DevOps engineer, not simply being a sysadmin or developer – makes them more unique and valuable on the market by that time. That is how market works overall - you make offering and advertise what demanding person is looking for, so he prefers you as a supplier, not the other guy.

And somewhere here we got that focus shifted from whole methodology to technical side of it. I have no exact answer, why this happened, probably because majority of market players missed the key point of DevOps ideas, and understood only those which could be measured easily.

Do we have Jenkins running now? Tick! Do we use Chef/Puppet? Tick!

It is way harder to make same kind of measurement for collaboration between team members, communication improvements, whatever. And higher management usually looking for kind of measurable results, rather then ephemeral ones.

So you are absolutely right – DevOps should be done by everyone in a team, just because it is about team itself, not some person who makes changes, and everyone and everything else remain old-fashioned.

# Answer
> 6 votes
**DevOps these days...**

> DevOps is known/meant to be more of a **culture** than a job title, but we also cannot ignore the fact that a vast amount companies all across the world have "DevOps engineers" employed.

**What Wikipedia says**:

> "The main characteristic of the DevOps movement is to strongly advocate automation and monitoring at all steps of software construction, from integration, testing, releasing to deployment and infrastructure management. DevOps aims at shorter development cycles, increased deployment frequency, more dependable releases, in close alignment with business objectives."

**Answer** *(my perspective )*:

As a DevOps engineer myself(*opens a can of worms*), I realize that I am more of a **support role** to development teams.

* Aiming to facilitate and create useful systems/tools that I as well as the developers can easily manage and use.
* Automate any manual work being done and creating streamlined CICD pipelines ensuring the developer's code gets tested and then deployed to production with ease.
* Ops and Monitoring to notify us and the developers of any issues so they can be resolved ASAP.

Making all of the "DevOps" responsibilities easy to "share" and deal with from all the technical and sometimes even non-technical teams or individuals of a company.

# Answer
> 3 votes
The role of DevOps is still being formed so the responsibilities can be different between organizations. Also, this was the developers responsibility not too long ago.

**What is DevOps**

I would say DevOps is encompasses managing source control, managing the build pipeline such as CI/CD and managing tests(unit tests, integration test...), managing the infrastructure including managing the cloud service being used(if any). With the tools that are being offered to teams, there is added complexity. The increase complexity required more time for a team to setup correctly. So what would happen was that one or two team members would gain experience in setting up all the necessary tools. These team members would need to spend more time with the operations side of development and these team members became the DevOps.

All developers should have some knowledge in the operations side of development, but it has grown to the point that it its too much to ask all developers to have sufficient knowledge current operations and the development work they need to know (how all components are tied together, any frameworks that are needed...)

---
Tags: terminology, culture
---


title: docker as slave for jenkins

text:
# Question
Title: docker as slave for jenkins
I have installed Jenkins and docker on the same machine Ubuntu.

* Ubuntu 17.10
* Jenkins ver. 2.73.3
* Docker version 17.09.0-ce.

I am trying to set up jenkins docker containers as slaves for running my automation test suite.

I was able to correctly setup the docker plugin in Jenkins for spinning up docker containers and added a docker template with a image that I created for setting up docker environment.

The image has been built on docker hosted in Ubuntu.

Problem is now when I run a job from Jenkins. It gives an error message:

> (pending—Jenkins doesn’t have label docker-slave)

When I check the Jenkins logs in Ubuntu machine I see the following error message:

```
> com.github.dockerjava.api.exception.NotFoundException: {"message":"pull access
denied, "message":may require 'docker login'"}

```

In the ubuntu machine, I have already given done docker login.

The image that I am trying to build containers from is locally on ubuntu, not pushed to any repository, so why is trying to pull the image?

Also, what is the permission issue that I need to resolve? When build a job from jenkins, it's the jenkins user building the container. Do I need to add something else there?

# Answer
> 1 votes
Sounds like the docker image you are using doesn't have the correct name, or is stored in a private repository which requires login.

Please post more information

---
Tags: docker, jenkins, jenkins2
---


# Question
Title: API gateway and service discovery on Amazon AWS
I'm trying to build an application infrastructure on Amazon AWS following a microservice pattern.

However the configuration API gateway and service discovery tool together completely puzzles me.

According to a pattern I need both (and I understand that). I decided to go with Amazon API gateway as API gateway and Hashicorp Consul as service discovery.

From what I see in the configuration guide on Amazon, API gateway configuration is pretty much straightforward: I specify path and URL to follow and that's it.

How in the world do I explain API gateway to get information from Consul?

PS. There is a Consul quickstart guide, but it is just a deployment instruction. There is nothing about connecting it to the API gateway.

# Answer
> 0 votes
It turns out there is no existing solution for such a configuration.

I have to develop something on my own.

A nice source for help might be a series of articles on the Amazon developer blog and REST API documentation for the Amazon API gateway service.

---
Tags: amazon-web-services, microservices
---


# Question
Title: Should I use filebeat or an app-logging module with lumberjack capabilities to collect my logs
I have a pretty basic elk setup which aggregates logs from around 15 VMs that are running various services.

All of them have configured filebeat syslog forwarder with a single input in logstash.

I have app servers where I run NodeJS web apps, which output JSON logs. In order to make my logs structured I have two options :

### Option A

Tell the NodeJS apps to output the logs in a `/var/app/app.log` rotatable file. Add the `app.log` to my log propspect in filebeat and push to logstash, where I setup a filter on `[source] =~ app.log` to parse JSON.

### Option B

Tell the NodeJS app to use a module ( e.g. `node-bunyan-lumberjack` ) which connects independently to logstash and pushes the logs there, without using filebeat.

My question is :

**Which one is a more common setup and why?**

# Answer
> 5 votes
Both are common and comes with their own pro and cons.

It depends if you can accept loosing logs if logstash is dead and your app restart and on the other hand if you can write logs to disk and consume place there.

Those are the basic points to choose for one or the other, you may end up with more caveats in term of IO/caching/etc.

There's no 'best' option, there's the best option for your organization/teams which we can't answer for you as we don't know your environment.

---
Tags: logging, logstash
---


# Question
Title: Suspend autoscaling processes through Cloudformation template
I have the following autoscaling as a Cloudformation template :

```
  MyAutoScalingGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      DesiredCapacity: 1
      MinSize: 1
      MaxSize: 1
      LaunchConfigurationName: !Ref MyLaunchConfiguration
      AvailabilityZones: !GetAZs

```

I am struggling with defining suspended processes inside Cloudformation template.

Trying an intuitive approach ( adding it as property) and it does not work at all :

```
  MyAutoScalingGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      DesiredCapacity: 1
      MinSize: 1
      MaxSize: 1
      LaunchConfigurationName: !Ref MyLaunchConfiguration
      AvailabilityZones: !GetAZs
      SuspendedProcesses:
        - AZRebalance
        - Terminate

```

# Answer
> 0 votes
Got it from official documentation of UpdatePolicy . Indeed, we can suspend processes by using the `UpdatePolicy` attribute for the `AWS::AutoScaling::AutoScalingGroup` resource :

* So instead of `SuspendedProcesses`, the right term is `SuspendProcesses`.
* Also instead of putting `SuspendProcesses` under `Properties`, it should be under `UpdatePolicy > AutoScalingRollingUpdate` .

_

```
  MyAutoScalingGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      DesiredCapacity: 1
      MinSize: 1
      MaxSize: 1
      LaunchConfigurationName: !Ref MyLaunchConfiguration
      AvailabilityZones: !GetAZs
    UpdatePolicy:
      AutoScalingRollingUpdate:
        SuspendProcesses:
        - AZRebalance
        - Terminate

```

---
Tags: amazon-web-services, autoscaling, infrastructure-as-code, yaml
---


# Question
Title: How to rebuild multiple jobs at the same time in Jenkins?
I'm using Jenkins Rebuild Plugin to execute a parametrized builds with a single *Rebuild* button. However when I need to rerun dozens of parametrized jobs (either selected, or from the same folder), it's still not efficient.

Is there any simpler way (eventually using different plugin) to re-run multiple jobs at the same time using the last used parameters?

# Answer
You can use a pipeline to tie the jobs together, executing them either in parallel or sequentially through steps.
You can pass the relevant parameters to each job as you call it from the pipeline script.
Then you just need to rebuild a single pipeline job. You can tie the jobs together with a common build name through a variable as well.

```
build job: 'buildjob1', parameters: [
  [$class: 'StringParameterValue', name: 'BUILD_NAME', value: ${env.JOB_BASE_NAME}-${env.BUILD_NUMBER}"],
  [$class: 'StringParameterValue', name: 'FIXEDPARAM', value: 'some-string'],
  [$class: 'StringParameterValue', name: 'PARAM1', value: "${PARAM1}"]
]

```
> 8 votes

# Answer
You can also build a list of parameter objects, then pass that into the build step:

```
def paramsMap = [:]
paramsMap['param1'] = 'value1'
paramsMap['param2'] = 'value2'
def paramsObjects = []
paramsMap.each {
  key, value ->
    paramsObjects.push([$class: 'StringParameterValue', name: key, value: value])
}
build job: jobName, parameters: paramsObjects

```

paramsObjects would look like this:

```
[
    [$class:StringParameterValue, name:param1, value:value1],
    [$class:StringParameterValue, name:param2, value:value2]
]

```
> 1 votes

---
Tags: jenkins, jenkins-plugins
---


# Question
Title: Compare Strings to get Unique values in Jinja2
I have a variable with list of node IP's and its respective Availability Zone.

```
seed = {
  10.18.13.12 = us-east-1a
  10.18.37.93 = us-east-1b
  10.18.68.147 = us-east-1a
  10.18.21.55 = us-east-1b
}

```

I will be using this variable in Jinja template but want to filter out the variable based on unique Avalability Zones and get the IP. I want to compare the Avalability Zones to get unique values and have IP from each unique Avalability Zone. For instance I want the output in form of

```
"10.18.13.12, 10.18.37.93"

```

or

```
"10.18.68.147, 10.18.21.55"

```

Is this possible to perform such operations of comparing strings and get its respective value in Jinja template? Anything would help. Thanks in advance.

# Answer
Yes, this can be done. The following should do the trick:

```
"{%- for ip, az in seed.iteritems() %}
  {%- if 'us-east-1a' in az %}
    {%- if firstloop is not defined %}
    {%- set firstloop = 1 %}
{{- ip }}
    {%- else %}
{{- " " + ip}}
    {%- endif %}
  {%- endif %}
{%- endfor%}"

```

To test it with https://cryptic-cliffs-32040.herokuapp.com/, you can use the following JSON:

```
{
    "seed": {
      "10.18.13.12": "us-east-1a",
      "10.18.37.93": "us-east-1b",
      "10.18.68.147": "us-east-1a",
      "10.18.21.55": "us-east-1b"
    }
}

```

You might give the documentation a read if you are going to be doing much jinja templating. I have found it indispensible.
> 2 votes

---
Tags: ansible, jinja
---


# Question
Title: Real-World Validating the Integrity of a Docker Container
I'm interested in information and experiences surrounding Docker security. I'd like to understand ways that a user/client could validate the contents of an image to ensure that the packaged applications/files are in compliance with their organization's/industry standard practices.

I've been reading online about security as far as establishing trust to validate the integrity and the publisher of data received from a registry(https://docs.docker.com/engine/security/trust/content\_trust/), as well as Security Assurance of Docker Containers(https://www.sans.org/reading-room/whitepapers/cloud/security-assurance-docker-containers-37432)\>, and I see some useful information around tools and practices to validate the security of containers and images.

Some tools I'm interested in exploring more are OpenSCAP docker tool(https://www.open-scap.org/resources/documentation/security-compliance-of-rhel7-docker-containers/) as well as Docker Bench for Security(https://github.com/docker/docker-bench-security) and Dockscan(https://github.com/kost/dockscan).

My question is, does anybody have any experience with dealing with security/compliance when deploying containers and sharing images or considering Docker as a solution for deploying applications internally/to external clients? If so, how would you recommend an organization best approach this topic?

# Answer
> 1 votes
# 2017 expectations

Docker is about getting some freedom from host management at the expense of reinventing all the things package management does. Docker images are just another package format. This is all very new, so I think this question is part of the growing pains of the Docker ecosystem.

## basic requirements/criteria

For secure trusted containers, only use them if they have been 1. tested by current standards 2. cryptographically signed 3. checked against a vulnerability database for insecure dependencies

Maybe there is a way, but I don't see a way to un-trust a signed, trusted Docker image after a researcher finds a new vulnerability. This is just one gap in the toolset cited in the question.

Are you planning to use a cloud Docker service? Are the containers' memory encrypted or at least obfuscated at the kernel (containers are just processes)?

## Are your Docker hosts secure?

http://ieeexplore.ieee.org/document/7562068/ http://ieeexplore.ieee.org/document/7397244/

# economy of security

In the end, you can never escape the economics of checking everything, so there needs to be some respect paid to the speed Docker provides by not worrying about all of this stuff. When you know you are handling sensitive information and you know you need to lock down the execution, it will require measures that are not available yet as commodity solutions (2017). Anything is better than nothing, so get some tools deployed and get your foot in the door.

## Hubblestack?

Keep your eye on HubbleStack. It provides a Docker image which can be dropped in to run audits (and remediations) against a live database of vulnerabilities. I'd be surprised if they left Docker trust out of scope forever. Also, if you own the Docker hosts, you can use root to scan all the containers. This kind of thing belongs in the DevOps automated test and validation cycle.

1. get some kind of checkpoint in place to get security in the pipeline
2. keep a backlog of unmanaged security concerns (gaps in the tools? CVEs?)
3. keep the backlog prioritized by what you can get done now, at current funding
4. get involved in devops development and release planning
5. deliver software confident that you did the best with the resources you had

---
Tags: docker, security
---


# Question
Title: How to evangelize DevOps and tools in a low acceptance environment?
An evocative learning about innovative disruptive technology is that you might find out that **not** everybody is going to be excited about tools which can give you more productivity.

To make this question less generic, what are do's and dont's - so as they also say, *lessons learned* \- of evangelizing Docker in your organization based on real experiences with different roles?

# Answer
> 9 votes
This is not docker-specific, but this general rule for evangelism applies: different audiences require different evidence. In general, software developers (and managers from a development background) want to see it in action, so POC's with measurable outcomes are preferred. Other disciplines and executive stakeholders may be OK with case studies and presentations before giving the greenlight; your goal is to identify which audience you're talking to, and begin to build a comprehensive argument.

Start small; find one key stakeholder, and present your case using the evidence that best suits them. Be patient; change in mature organizations is usually slow, so getting a small project up and running is a major accomplishment.

# Answer
> 4 votes
A couple of months ago I gave a presentation about the pros and cons about docker. My expectation was that the developers would start to use tool immediately after the presentation, but it took a couple of months. It turned out that a (huge) problem was required in order to get it accepted by the team members:

> There a six services at the moment that all run in vagrant. My system is unable to handle it anymore. Shall we use docker? You addressed in your presentation that it omits an OS, that it is more lightweight compared to a VM so the machine will consume less resources and I will be able to run the services locally right?

When it turned out that the services were now able to run, more and more developers started to use docker.

# Answer
> 0 votes
You could put the names of colleagues in a sheet and decide per individual if one is easy or hard to change, has a lot of influence and is willing to share knowledge. Start sharing the information with the one that has the highest score and try to convince him or her to convince the next one that is most likely to change.

**References**

https://hbr.org/2015/03/convincing-skeptical-employees-to-adopt-new-technology

> You want people who are able to work horizontally across the organization and who have good communication and networking skills.

https://hbr.org/2014/09/convincing-employees-to-use-new-technology

> Identify your committed digital champions early – individuals who network well and can create horizontal influence to help implement behavior change across silos.

https://gethppy.com/talent-management/convince-employees-adopt-new-technology

> Ensure that the supervisors or other employees whom the rest of the staff looks to for advice and leadership will show enthusiasm.

---
Tags: docker, culture, transformation
---


# Question
Title: Is it possible to run every testing.xml file with different jenkins job?
I've around 100 test method in my testing.xml file. and that testing.xml configured in pom.xml file so I'm able to run my all those testcase as maven test and I'm getting the proper reports and all.

I have created one Jenkins job and configured maven project in that and mentioned pom.xml file path of local. I'm able to run successfully.

I'm planning to separate some testcase category wise and manage then in another testing.xml file. lets say 2 xml file having 50 50 test methods.

I'm stuck here. Is it possible to configure every testing.xml file in separate Jenkins job ? if yes then how do i configure ? (Note: I have to run maven test for getting the reports)

# Answer
I got the answer on StackOverflow :

You can pass the xml file name as parameter to the maven test command. First need to change the pom file as follows.

```
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.19.1</version>
        <configuration>
            <suiteXmlFiles>
                <suiteXmlFile>${SuiteXmlFile}</suiteXmlFile>
            </suiteXmlFiles>
        </configuration>
</plugin>

```

Then, you pass the different file name for each execution in goals and options like :

```
test -DSuiteXmlFile=mytestng1.xml

```

For second xml file :

```
test -DSuiteXmlFile=mytestng1.xml

```

You can configure each Jenkins job with different parameter.
> 2 votes

---
Tags: jenkins, testing, maven, java, selenium
---


title: pre-DevOps deployment metrics challenge

text:
# Question
Title: pre-DevOps deployment metrics challenge
TL;DR, how do you *prove* devops, specifically deployment automation, improves change failure rates?

We're all trying to capture metrics on 'deployment failures' using *current* (mostly manual) means. Unfortunately, a 'failure' rarely happens, right? Because when something goes wrong, the team comes together (typically with heroics) to fix the issue (typically permissions, missed configs, you know the drill). So... when we ask how the deployment went, the answer is "it worked."

But, intuitively we all know that's not good. The 2017 state of devops report says there's about a 31-45% "*change failure rate.*" While that intuitively sounds about right, are they tracked as incidents? Nah. Because they get fixed pretty quickly, usually during validation. It's much more rare to actually roll back a deployment.

So, it takes discipline to report failure rates accurately. We're disincentivized to report like that because we want things to work and we do what it takes to make it happen.

**So, how do you prove devops, specifically deployment automation, improves change failure rates?**

(PS tried to tag this with "#devops-capability-model")

# Answer
> 7 votes
A technique we've used in the past in similar situations is to get "management commitment" that imposes these rules to every team member:

1. Access to perform updates to the target deployment areas (i.e. production) is limited to selected automated systems, which have appropriate audit trails (= logging) of any kind of updates to the areas they manage.
2. Manual updates to the target deployment areas, for whatever reason, are no longer allowed by the typical team members (user ids) that used to be able (authorized) to perform these updates. Instead NEW (additional) user IDs will be created which will have all required permissions to (still) perform such manual updates, whenever needed. But to actually be able to use those new user IDs (= perform a logon with them), the team member who wants to perform a logon with such new user ID will have to perform "some" extra step to get access to the password for such new user Id. Ideally this extra step is automated also (use your own imagination how it should look like), but if anything else fails: just contact (= eMail, call, etc) the gate-keeper of the required password, including "which issue they have to get fixed" (similar to your question).
3. Getting such gate-keeper in place is not an easy job. And the most resistance will come from ... the team members (for all sorts of reasons). Therefor a variation of those new user IDs (as in the previous step) is that each team member gets an extra user ID (with the password they decide themselves), but with an extra string attached to it: they are only allowed to perform a logon with that (extra) user ID if they actualy have a good reason to do so. And each time they perform such logon, they are required to file some type of report about "which issue they fixed" (similar to your question).

With these procedures in place, all that's left to do is to periodically review each of those reports / reasons why it was required to use such special user ID, and ask the question "*Is there anything that can be done to further automate this, to further reduce the need for such special login?*".

**Update**:

Quote from your extra comment below this answer:

> I think adding artificial barriers to fixing a deployment issue is counter-productive.

True it adds an extra barrier, but I'm not conviced it is "artificial". Because this is, to my knowledge, the only way to become aware of things those team members otherwise won't ever tell you, for reasons such as:

* job security.
* bad things/practises they prefer to keep hidden.
* power they do not want to loose.

# Answer
> 6 votes
> The 2017 state of devops report says there's about a 31-45% "change failure rate." While that intuitively sounds about right, are they tracked as incidents? Nah. Because they get fixed pretty quickly, usually during validation.

An issue that gets fixed quickly is still an issue. If you're not reporting these as such, that's a problem.

> So, it takes discipline to report failure rates accurately. We're disincentivized to report like that because we want things to work and we do what it takes to make it happen.

If your goal is actually to have things work, then you need to be honest about failures so you can prevent them in the future. It sounds like the team here is lying (perhaps to themselves, certainly to management) about failures because their goal is to have things *appear* to be working.

These are different things. For instance, take the old joke that QA produces bugs - "my code was fine until QA got ahold of it, and then they made all these bugs!". The bugs were there all along, but the developer was ignorant of them. An operations team's goal should be *actual reliability*, and they need to be incentivized as such by their management. That means that if they put more monitoring in place that leads to discovering new issues, they should be rewarded, not penalized for the subsequent drop in reliability metrics.

---

> TL;DR, how do you prove devops, specifically deployment automation, improves change failure rates?

If you're trying to motivate change in your organization, then you shouldn't be trying to prove anything, but provide evidence of what other organizations say about their own transitions. If you're trying to measure the processes that you already have in place and justify their continued existence, then you should be tracking the standard reliability metrics, like mean time to repair (MTTR).

But devops principles are not merely about increasing reliability. Even site reliability engineering is not merely about increasing reliability. Rather, you want to get to an *appropriate* level of reliability - something that benefits the business but doesn't hinder development. And that brings up the real motivator in devops, which is to **empower change**. You want to allow the business to respond quicker to market stimuli, which happens by decreasing developer friction, increasing the rate of deploys, automating manual processes, etc. while staying within an acceptable bound of reliability. This means you need to measure reliability, but you also need to measure velocity, because your goal is to increase the latter while keeping the former relatively static.

---
Tags: metrics
---


# Question
Title: If a piece of work requires manual testing/QA, do we need to wait until opening PR?
First question here (i'm usually on StackOverflow)...not sure if the right place (if not, please vote to move where it belongs).

If we have a dev process using traditional GitFlow approach, what criteria should be met before a Pull Request (PR) is opened? Specifically, if the work requires manual testing/QA signoff before being *merged* into `master`.

There are two sides of the fence here:

1. **PR's should *not* be opened until the work is "ready"**. (ready = ready for production, pending any code reviews, which should just be based on code quality/standards). *Pros*: reviewer doesn't need to check if the work is 'ready to be merged'. It's defined by the process. *Cons*: longer time (e.g waiting for test signoff) before the code is reviewed. PR's are open longer.
2. **Open the PR, even if it's not ready**. Basically, allow the QA testing & code reviews to operate in parallel. *Pros*: code review and testing can operate in parallel. *Cons*: if issues are found in testing, the code needs to change, even though it has already been reviewed. So reviewing could happen multiple times, and communication needs to be fed correctly between parties.

Thoughts on either 2 approaches, or is there an option 3? What standards/advice is there in this process?

# Answer
The guidance that we've always followed in my various scrum teams is developers should get what they've got into testing as soon as possible. It may mean a little rework later on down the line if there are issues found in code reviews or developers haven't completely finished coding, but the earlier developers get feedback on what's been done, the earlier they can fix any issues, and take account of this feedback in any further coding.

You should make testers aware when they aren't getting completely finished output, but if they've been encouraged to think about the benefits of preventing bugs as well as just finding them, they should hopefully be keen to adopt this process.

It isn't always practicable - for example, if you don't have deployment tools that are flexible in which branches they deploy from, but when it is practicable, it's what Scrum and agile are all about - get feedback and get it early.
> 5 votes

# Answer
> Open the PR, even if it's not ready. Basically, allow the QA testing & code reviews to operate in parallel. Pros: code review and testing can operate in parallel. Cons: if issues are found in testing, the code needs to change, even though it has already been reviewed.

When code review prompts changes, testing has to be redone. This encourages the building of automated tests over manual tests, as then this can happen automatically without any human involvement.

When testing prompts changes, code review has to be redone. Code review is ideally primarily about a 10,000-foot view of how to approach the problem, but it certainly does cover implementation specifics as well. But this is a problem mostly solved by review tools - they'll show you the changes *since you last reviewed* so you don't have to review everything all over again. Some tools handle this well with rebasing, but if you're using one that doesn't, stick to adding new commits as you work, and then squash them all together at the end if that's what you want.
> 4 votes

---
Tags: git, process, agile, github
---


# Question
Title: Using GSuite or Github to Manage who can log into Linux or Windows VMs on Azure
I want to manage the members of our team who are able to log into our cloud-hosted VMs, both Linux and Windows. We currently use G Suite and GitHub, to manage access to our code and documentation etc. I would really prefer to not have to also manage a separate Active Directory Domain within our VNets or deal with revoking ssh keys or updating multiple usernames/passwords on every single VM we deploy.

Are there ways to use either GSuite or Github to control login access to our VMs?

If there isn't, what are the best practices to deal with 20+ developers who are all empowered with DevOps responsibilities?

# Answer
> 3 votes
> I would really prefer to not have to also manage a separate Active Directory Domain within our VNets

You've already found the answer: centralized authentication.

If you use a tool like Okta, you can indirectly tie GSuite to AD (or an LDAP server). And then of course you can tie OpenSSH there as well.

> or deal with revoking ssh keys

Centralized authentication solves this.

> or updating multiple usernames/passwords on every single VM we deploy.

And this as well.

You've already got the answer, even if you don't like it.

---
Tags: authentication
---


# Question
Title: Are there any best practices and/or tools for cleaning old Docker images on a regular basis?
Is there an elegant way or a best practice when it comes to deleting old images from the Docker registry?

I see a lot of requests/issues here: https://github.com/docker/docker-registry/labels/delete, but didn't find a good/popular solution for it.

**So, is there a tool or a technique which would help me do that?**

Also, is there any best practices which you follow while doing it?

# Answer
I've had good luck with Spotify/docker-gc. It can be run from cron or as a docker container.

> A simple Docker container and image garbage collection script.
>
> * Containers that exited more than an hour ago are removed.
> * Images that don't belong to any remaining container after that are removed.
> 10 votes

# Answer
Can't call it's the best practice but this is what we use triggered by cron, happy to see better suggestions.

```
echo "safely removing untagged images"
docker rmi $(docker images | awk '/<none>/{print $3}')

echo "safely removing stopped containers"
docker rm $(docker ps -a -q)

echo "safely removing old containers"
docker ps -a | awk '/weeks ago|months ago|days ago/{print $1}' | xargs --no-run-if-empty docker rm

echo "safely removing old images"
docker images | awk '/weeks ago|months ago|days ago/{print $3}' | xargs --no-run-if-empty docker rmi

echo "safely removing old volumes, custom rebuild of martin/docker-cleanup-volumes image"
docker run -v /var/run/docker.sock:/var/run/docker.sock -v $(readlink -f /var/lib/docker):/var/lib/docker --rm example/docker-cleanup-volumes

echo "native cleanup Docker => 12"
docker system prune -f

```
> 9 votes

# Answer
On my local machine (mac) I have a little script I found that I run periodically which cleans all the excess images up clean-docker-for-mac.sh

For my servers I run meltwater/docker-cleanup which periodically cleans up exited containers and removes images and volumes that aren't in use.

I use a lot of different Docker images for my work on Codemason that scripts like these are necessity for me. Enjoy!
> 4 votes

# Answer
`docker system prune --volumes`

Is the way to do this since v17. Docs here
> 0 votes

---
Tags: docker, toolchain
---


# Question
Title: Pause/Suspend guest in KVM without virsh and libvirtd
I have installed KVM without libvirt/virsh and virtualmanager.

Packages which installed are kvm qemu-kvm bridge-utils libguestfs-tools

How can I pause/suspend VM in this case?

# Answer
> 1 votes
After long searching and experiments, the best option for me became qemu monitor. See https://nico.schottelius.org/blog/control-and-shutdown-qemu-kvm-vm-via-unix-socket/ for more details regarding flexible management guest by using qemu monitor.

---
Tags: linux, virtualization
---


# Question
Title: How to Shutdown Docker Stack on Container Failure
Say I have several docker services using swarm defined as such:

```
version : '3'
services:
    server:
        image : my_server:latest
        deploy:
            replicas: 5
            restart_policy: any
    database:
        image : my_db:latest
        deploy:
            replicas: 5
            restart_policy: any
    logs:
        image : my_logger:latest
        deploy:
            replicas: 1
            restart_policy: any

```

I then deploy this as a docker-stack as such. The services are **server**, **database** and **logs**. I make great use of the network abstractions provided by docker swarm, but my failure-case is a bit undockerish.

\*In this case when all containers in **database** fail, then I want the *entire* stack to be shutdown or otherwise made unavailable\*. Is there any built-in way to accomplish this task? Does the concept of a stack exist in a way for the individual services to access?

This question can be alternatively answered with a generalization : "How, with built-in docker features, can I be made aware of container failure"? Because I'm assuming there is no way to do exactly what I am asking.

# Answer
If you don't want the containers to restart, you can set the `condition:` to `none`. See https://docs.docker.com/compose/compose-file/#restart\_policy

On the swarm manager, you could use `docker stack ps` to check the status of the containers on a regular basis and shut down the stack if the database containers aren't running (using cron or whatever). For example:

`docker stack ps --filter "desired-state=Running" your-stack |grep database || docker stack rm your-stack`.

If containers are not able to restart, they shouldn't show up in the list once they fail, and once none are found, you can remove the stack.
> 1 votes

# Answer
Using Docker's healthcheck feature (a side process for an additional script) you could register and sync your container health status with an external service e.g. Consul. The API of this service would then allow further services to take some action, for example as you said redeploy the whole stack or raise an issue.
> 0 votes

---
Tags: docker, docker-compose, docker-swarm
---


title: How can I pipe jq output

text:
# Question
Title: How can I pipe jq output
I recently learned about an outstanding little tool, jq that allows you to pipe unformatted JSON output into it and then have that output reformatted and output to the screen in a very nicely formatted color-coded json layout. Eg:

```
echo '{"value": "New", "onclick": "CreateNewDoc()"}' | jq

```

I have found however, that I am unable to pipe the formatted results (from jq) into additional tools in my standard toolset, such as grep, sed, awk, etc. How can I make make this output pipe-able? Eg, this doesn't work:

```
'{"value": "New", "onclick": "CreateNewDoc()"}' | jq | grep value

```

# Answer
Ok tried to understand your problem exactly... This is what is happening:

```
$ echo '{"value": "New", "onclick": "CreateNewDoc()"}' | jq | grep value
jq - commandline JSON processor [version 1.5-1-a5b5cbe]
Usage: jq [options] <jq filter> [file...]
[rest of output is omitted for brevity]

```

The important thing to notice here is the usage line (emphasis is mine):

> Usage: jq \[options\] **\<jq filter\>** \[file...\]

The filter is not optional, if you don't give one, jq tries to parse the rest of the command line as its filter and throws an error. A workaround for your case here is:

```
$ echo '{"value": "New", "onclick": "CreateNewDoc()"}' | jq '.' | grep value
  "value": "New",

```

which indeed greps the "value" line as the filter passed is '.' - this just pretty prints the json content, but that's far from the best use of jq. If you wish to get only this line it would be better do:

```
$ echo '{"value": "New", "onclick": "CreateNewDoc()"}' | jq '.value'
"New"

```

If you wish to get the output without quotes then you can add add `-r` option to jq like so:

```
$ echo '{"value": "New", "onclick": "CreateNewDoc()"}' | jq -r '.value'
New

```

from `jq --help`:

> -r output raw strings, not JSON texts;

that's a little introduction to jq, that's probably not enough to solve your problem at all but as you didn't specify your problem I can't help more than that.
> 16 votes

---
Tags: json
---


# Question
Title: How to only allow API Gateway requests to reach our EC2 instances
We have multiple EC2 instances hosting our micro-services. The autoscaling group of servers has an ELB. All traffic is routed via the AWS API Gateway. The problem is that the ELB has its HTTPS port open to the world.

How to protect our servers, so that traffic is only allowed to pass through API Gateway?

# Answer
Since November 2017, it is now possible to directly interact with servers in a VPC \o/

See:
> 6 votes

# Answer
The best protection for your servers if you don't want to expose their HTTPS to the world would be to isolate them in a VPC.

However, API Gateway can't be configured to directly interact with servers in a VPC/subnet (yet). In order to get around that limitation, you can proxy your traffic from the API Gateway through AWS Lambda to reach the VPC. AWS blog has an excellent blog post that explains exactly how to do so.

Isolating your servers in a VPC will be more secure than keeping them on the public Internet and trying to build something to detect if the traffic is legitimate (coming from API gateway).
> 4 votes

---
Tags: amazon-web-services, security
---


# Question
Title: How to automate the magento2 installation process too via docker
This is what I have accomplished up to now. Written a docker file where I have done the following and for now I have not used the docker compose.

1. Used alpine as base Os
2. setup nginx
3. setup php-fpm
4. Installed magento2 file

So my docker build is ready and the container is ready and can see the setup page when opening it on the browser.

**My problem** How do I automate this Magento2 installation process from **dockerfile**. Like setting up database and the other installation process.

# Answer
Instead of creating a dockerfile yourself you could also pull a docker image, e.g. alexcheng/magento2.
> 1 votes

# Answer
I'm going to talk about general process here, rather than addressing Magneto specifically, so that it's useful to more people and situations.

> How do I automate this Magento2 installation process from dockerfile. Like setting up database and the other installation process.

Run through the wizard process once. Track down the files it generates, which isn't straightforward and may require a few tries. Pull those configuration files into your app configuration, and try a fresh container that just uses those.

That is, reproduce what the installation wizard is doing in your own configuration tool. This sounds difficult, but often the wizard is complex because it needs to address many situations, and you are fine hard-coding many things for your specific situation.

As a final note, since we're talking about Docker, presumably the database resides on another machine. Therefore, you only need to set it up once, and that process will be part of *its* provisioning, not this container's.
> 1 votes

---
Tags: docker, dockerfile
---


# Question
Title: Why a chef is the boss of the kitchen but a fullstack dev is not often the leader of a dev team?
In one another comment here on DevOps SE we read:

> "Why a chef is the boss of the kitchen but a fullstack dev is not often the leader of a dev team, that's a question for another day"

So given the assumptions contained here are correct, why is so? Are we yet not there in terms of overall improvement and progress or is this economically nonsense (or, assumptions not correct)?

# Answer
IMHO the comment is a bit misleading, in the sense that `a chef is the boss of the kitchen` mixes up and/or attempts to compare the unique leader/chef **role** (in French, `chef` literally means chief/boss/leader) with the **expertise** required to fill that role. By contrast, the `full-stack developer` is just an expertise, it is not a unique role.

If I'm not mistaken to become a chef one has to have experience in every other position in the kitchen. Similarly, a full-stack developer's expertise covers frontend, backend, etc - every other more specialized expertise required for the team's product. Which is probably the starting point for the comparison in the comment.

As I mentioned in my answer to that post, scale matters.

In a large team, the requirement for the leadership position isn't necessarily the technical expertise. Which is why often in software development the team leadership's role is a management position, not a technical one. Clear labour division example, if you want.

It's not impossible to even have multiple persons qualified to fill the chef role in the same kitchen, but only one can actually fill that position at any time, `a chef` in this context really refers to the role, not to the expertise. Also - there may be many types of chefs in a kitchen, see, for example Chef Jobs on Cruise Liners :)

But in a small resource-starved team where the team leader has to be selected from between the only full-stack developer and one or more less-experienced ones - probably the full-stack developer will be the leader, just like the only person qualified to be a chef in a small kitchen.
> 9 votes

# Answer
The quote is misleading since there is not "a" chef in the kitchen. There are many different chefs, i.e. pantry chef, soup chef, fish chef etc. Wikipedia lists many others.

What the quote is thinking of is the "chef de cuisine", which is the overall manager of the kitchen, the CEO if you will. There is also the whole executive hierarchy we know from our IT companies, e.g. sous-chef (second in command), commis chefs (department manager) etc.

So, your fullstack dev is first and foremost a someone who can cook well. But even if you cook the greatest delicacies, that does not make you a good manager. On the contrary, there are people who are actually *trained* to be managers/leaders. The skillset, and daily work, of managing the logistics of a large kitchen is vastly different from the cooking itself.

Instead, a fullstack dev specializing in AWS, Azure, CI/CD, "xxx as code" or any of the other tools that we associate with DevOps, would be comparable to a soup chef specializing in soup or a fish chef specializing in fish (with no managerial tasks attached to the job description).

And finally, remember that in IT, "Agile" is all about *removing* leadership from the immediate development process and placing leadership where it can do good: enabling the devs to do their work. For that, they do not need to know that much about software development, or be a great dev themselves.
> 4 votes

# Answer
Firstly, because leadership is not very much to do with any particular technology, but about having a deep understanding of the problem that the team is writing software to solve, and about having great interpersonal skills: mentoring, conflict resolution, self-awareness... A person who thinks it is about technical ability alone lacks at least one of those.

Secondly, let's be honest - the vast majority of "full stack" developers really only know JavaScript, and even then, only in the context of applying a pre-written framework to a well known problem, like an intranet or a shopping cart. A person who is seasoned in C++, or the major RDBMS's, or similar traditional skills will not find this impressive and will not take leadership from someone who believes that they are a master of all skills in the stack - unless that person swallows their ego and adopts a more realistic title.
> 2 votes

---
Tags: culture, project-management
---


# Question
Title: How do I trigger scripts on a merge or when a branch is deleted?
I am using jenkins multibranch pipelines. If a branch is merged and deleted jenkins removes the branch from the pipeline as well.

This is great, but I want to run a command/script when a branch is removed, how can I have jenkins run a command when a branch is removed? Is there a way to have jenkins trigger a job before it removes the branch?

Edit: I should add I'm using bitbucket. It would be neat if Jenkins could trigger an event when purging a branch of a multibranch pipeline. Ideally it wouldn't matter what my git provider is.

# Answer
> 4 votes
*"This is great, but I want to run a command/script when a branch is removed, how can I have jenkins run a command when a branch is removed?"*

> You will need to use a **webhook** for this, I use the following settings in Github:

> **Bitbucket** only supports the following webhooks:

*"Is there a way to have Jenkins trigger a job before it removes the branch?"*

> Not that I can see, but in Github you can create a custom label and trigger a job via a webhook when the label is added to a branch.

---
Tags: jenkins, git, jenkins-plugins, jenkins-pipeline, jenkinsfile
---


# Question
Title: ELK - micro service and network latency monitoring
Does ELK stack provide micro service and network latency monitoring in kibana?

Zipkin provides details about service request and service response duration.

In behind ELS stack should trace span events:

cs - Client Sent

sr - Server Received

ss - Server Sent

cr - Client Received

# Answer
ELK stack does not provide monitoring. It only provides reporting unless you configure additional add-ons such as ElastAlert.

ELK also only reports (or alerts if configured) on the data that you feed into it. So to turn the question around:

> Do you have your network devices configured to send micro service and network latency logging information to your ELK stack?

If yes, then the answer will be yes. If no, then the answer will be no.

ELK is nothing more than a system for processing and reporting metadata and generating structured data from plaintext logs, so it will do whatever you configure it to do.
> 5 votes

---
Tags: network-monitoring
---


# Question
Title: docker container for running automation scripts using jenkins
For my test environment, I am trying to setup the following setup.

Create scripts in Frisby

Have a jenkins job, which creates a slave which is a docker container. The docker container will checkout my latest scripts from my git repository, and run the tests in the container. Since its api tests, I do not need any product code, my scripts is all that is required which will be connecting to API to test.

My question about the dockerfile. When I create the image for spinning the container. What is the best way to create the image file.

My assumption is as following

* use evagra/jenkis slave image and add on it,
* installation for Frisby
* do a checkout of my scripts
* run command for my scripts.

Is this the right approach.

# Answer
Yes, you are doing the correct approach. Or rather, "a" correct approach, in your specific circumstance.

You are proposing to build the image beforehand, not as part of each individual Jenkins job. Then your Jenkins job will consist of running the image; and in that ephemeral state (i.e., inside the *container*) you are checking out the test script, and running it.

The more usual alternative would be to leave the actual `git checkout` to Jenkins, and pull the script into your container with a bind volume. Jenkins has to be able to access your git repository anyways, so you get this checkout for free. There are two sides to the coin here: if you do the `git checkout` from inside the container, then you can run your test outside of Jenkins with no change whatsoever - say, during development, on your own machine. In any case, if you do the `git checkout` from inside, and if you need credentials to access the git repository, be sure to handle those securely. Don't put cleartext passwords in the image, or ssh keys with an empty passphrases.

The benefit you are getting from this is that your test environment (libraries, 3rd party test drivers and so forth) is well defined, so you can run it on any Jenkins node you have available. By not creating new images whenever your code changes, you avoid creating and distributing new images for every job.

> I do not need any product code, my scripts is all that is required

So you would never actually *deploy* those images anyways - hence it is perfectly fine to not build them each time. You still should be *automatically* building your images - no matter if you do it as part of the actual job, or as a separate job which only does it when the Dockerfile (and whichever files it pulls in with `ADD` / `COPY`) changes. Not for technical reasons, but to avoid a manual step.
> 4 votes

# Answer
I do not understand why a docker container should perform the checkout as this could be done using Jenkins.

One of the advantages of docker is that it is immutable, but the way you use it right now is mutable.
> 1 votes

---
Tags: docker, jenkins, dockerfile
---


# Question
Title: What Key Performance Indicators (KPIs) are used to measure DevOps?
I am trying to drive good behaviors within a DevOps transformation programme, to support this I am looking at identifying actionable metrics around the operations disciplines:

* Problem and Incident Management
* Capacity Management
* Change and Release Management

To be absolutely clear, these are functions that used to belong to the operations organization and are now owned by the Agile/DevOps organization. There are existing KPIs that drive bad behaviors are:

* Time to Root Cause Analysis completed:
    + *Drives incomplete RCAs just to get them into the system on time.*
* Test execution duration:
    + *Disables long running tests, regardless of their business value.*
* Average utilization of cloud services:
    + *Encourages over-commitment of compute resources, resulting in slow response times*

**What Key Performance Indicators can be used to encourage good behaviour in a DevOps Programme?**

# Answer
I don't think there are any "universal" DevOps KPIs. For example, velocity is great, unless it's not a key driver for your business. Amazon cares a lot about velocity because they have a massive retail operation. That's less important for a small app with 100 users.

This begs the question: how do *you* select the best KPIs relevant to *your* business? That's a research and discovery process that involves your entire Enterprise.

What do you care about?

* Quality
* Reliability
* Maintainability
* Velocity
* Process Improvement
* Service Levels

What keeps your business stakeholders up at night? What determines whether you make money this quarter or not? The list above might include some of those things, or it might not. Make your list, then figure out how to align incentives *across every department* to achieve them.

Incentives drive behavior, so decide collaboratively on SMART goals. Pick two or three items off your brainstormed list, and start a measure/fix feedback cycle for each. Don't pick too many at once- you're more likely to succeed by focusing hard on two or three things.
> 12 votes

# Answer
**DevOps** does not have any **KPI**. It would be like asking what are the KPI of Love. But some of the things you mentioned (*Problem and Incident Management*, *Capacity Management*, *Change and Release Management*) do have good KPI, some of which are based on the theory behind DevOps.

In general, for any business process, you can create a **Value Chain Map** describing how value flows from **Customer**, through the enterprise back to **Customer**. The entire loop always has to start and end with Customer, but sometimes, for a service organization, the Customer can be internal. The **throughput of value** through such chain can be a good way to design your KPI in a tamper-proof way. Measuring any KPI in any individual link of the value chain *only makes sense as long as that particular link is the bottleneck* of the process and you try to exploit or elevate the bottleneck.

A common problem with KPI is when it starts halfway through the chain. For example, a Change and Release process often starts with developers and ends with deployment. This process excluded:

* Customer having a problem
* Support team identifying the problems
* Product team defining the problem for backlog
* Solutions team customizing the deployment for the customer
* Customer realizing value from the solution

The problem is that measuring a cycle time alone could lead to two major problems:

1. The bottleneck is in any of the excluded parts mentioned above and your customers are not realizing the value and you are not realizing the revenue proportional to the speed of your cycle time. So while your engineering is excellent, your business suffers.
2. The disconnect from Customers will make your release cycle spin on empty - not producing any value, despite change being done - or even counter the needs of your Customers and the work being done could have negative business value.

Another problem with KPI is that while starting and ending with a customer it might not actually measure the value to the customer. A good example would be a Problem and Incident Response process and MTTR (Mean Time To Repair) as KPI. Does the problem even affect anyone? What is the value to customers? You could have excellent MTTR of 3 hours over 100 incidents. But if most of them were internal, with no or minimal impact to customers and resolved in minutes, while the one large incident with huge customer impact took 3 days to handle, the business value is lower than if you had 1 day MTTR, because you ignore most of the internal incidents, but you responded to the huge customer impact incident within 1 hour.

**NOTE:** For an internal customer, in case of the support team business process, the derived value is not the value of the work to the internal customer, but the value gained by the business in unblocking the internal customer in their own business process. Unblocking a team that is a bottleneck in their own process derives higher value than unblocking a non-bottleneck team or individual. All KPI of such support team need to include the business value in their calculation.
> 3 votes

# Answer
1. Deployment frequency
2. Speed of deployment
3. Deployment success rate
4. How quickly service can be restored after a failed deployment
And finally,
5. DevOps Culture, which actually can’t be measured
> 1 votes

---
Tags: culture, metrics, kpi
---


# Question
Title: Recipe for hiring a winning DevOps empowered team?
The comparison of a software delivery team to a football team (all can kick the ball but only some will become an excellent goalkeeper or a forward) can be a nice input for further thought regarding division of labour and specialists vs. generalists.

So, it seems more to me than just "you should never try to hire a DevOps team" - because if you hire "a football team" you do not just want "somebody kick the ball in the FIFA World Cup", but you possibly want to have an idea what makes the team having chances to win.

So, winning teams can be a product themselves (can't find the reference for that but there are enough startup acquisions, I'm sure it is not always just about the product) - what is the recipe for success? For sure there is a lot of soft skills/cultural related context, but this would be true for virtually every team.

What are *mandatory* DevOps specific requirements to skill sets for a winning team setup? Will be there a relationship between numbers of CI/CD experts and those who focus on development, or operations?

Note: this question is not a duplicate of "why you shouldn't try to hire a DevOps Engineer" as a specific question is after required skills setup. E.g. would you have a CI-focused profile or expect developers to include pipelines as code to their projects? Clearly, just hiring somebody with the DevOps claim does not answer this question.

# Answer
I believe DevOps is orthogonal to your question, i.e., it changes nothing compared to a "classical" approach (or to hiring any team at all, not only for software development). You identify what your key needs are (for example, an "architect" who is able to structure large software systems; some "hacker" who is able to fix a Kernel driver if needed; a "tester" who likes to test and such; and finally maybe a "DevOps engineer" who excels in creating good CI/CD tooling).

But all of them need, to stay with your image, to be able to kick the ball. I.e., they all need to work together in the context that the team is working in. They all need at least some basic understanding of what the other team members are doing; if you have a strict CI/CD pipeline, then they all need to be able to develop in that frame; and so on.
> 8 votes

# Answer
Since DevOps is a hot trend in IT, companies are seeking top-notch DevOps talent. However, before you begin assembling your team, there are a few things you should consider:

1. What kind of DevOps?

Imagine a rock band is searching for a new guitarist. There are lots of people who know how to play the guitar, but there are so many music styles that can be played on the guitar, plus there is also another important factor, which is how well this guitarist gets along with the rest of the band.

When you are searching for candidates, imagine asking an IT guy if he plays DevOps. There are so many successful manifestations of DevOps teams that you need to understand it in the context of the business that they support, because, ultimately, this is the value of DevOps. The customers expect a certain pace and DevOps is the only way to deliver at that pace.

2. Emphasize culture over tools and roles

When you get down to it, DevOps is all about culture. If you take a group of IT engineers and traditional developers who understand and embrace the DevOps culture you can build a successful team. However, if you hire so-called DevOps experts, who know Puppet, Chef and Docker inside and out, but do not accept and adapt to the collaborative and cooperative nature of DevOps, this team is doomed to fail.

3. Be careful of how you word the job description

What happens a lot of times is that companies throw in a bunch of buzzwords and tool names and expect to find the right person for the job. Do not dismiss perfectly good candidates just because they do not call themselves “DevOps experts.” More importantly, pay particularly close attention to the personnel you already have. With your leadership skills, the talent you already have can be quite successful at DevOps.
> 3 votes

# Answer
DevOps is a "team" of people that includes the development and operations teams. (security/compliance combined).

You shouldn't think about the devops "team" as a separate entity but part of the larger organization. Checkout http://web.devopstopologies.com/ for organizational structure. You can see it's about moving together.
> 1 votes

# Answer
I'm currently extending my DevOps team, without forgetting skills, for me, the most important thing is the "mindset". Amazingly, not a lot of people around with the right mindset of continuous improvement.
> 1 votes

# Answer
I tend to agree with AnoE's answer. Getting the people and giving them the vision are primordial before DevOps should be put in the picture. I would also like to add two more things on top of that. The additions come from the experience with a team that got excited about "DevOps".

You already have the definition in mind, but I'd like to bring Wiki's:

> DevOps (a clipped compound of "development" and "operations") is a software engineering practice that aims at **unifying** software development (Dev) and software operation (Ops). The main characteristic of the DevOps movement is to strongly advocate **automation and monitoring** at **all steps** of software construction, from integration, testing, releasing to deployment and infrastructure management. DevOps aims at **shorter development** cycles, **increased deployment frequency**, more dependable releases, in close alignment with business objectives.

In short:

* You want to build a team of developers, testers and operators
* Enforce short development cycles
* Enforce shorter deployment cycles
* Automate the above processes
* Due to the agility of the process, enforce monitoring and alerts at all stage of development (e.g. test, qa, production)
* You need a set of tools that will increase efficiency accross every stage (i.e. Jenkins, Jira, Gerrit, etc.)

With these in mind, I can now bring the points previously mentioned at the beginning.

**Failing is good**

A baby doesn't learn to walk by standing up and walking. He learns through constant tries and failures. Even if, for an unknown reason, he gets up and starts walking, we shouldn't expect our babies to achieve the same. I usually point that out when people compare themselves to Google, Amazon or any other company with great achievements. Their failures thought them how to get stronger more than ingenious ideas all the time. Right now for example I am improving my english through constant writing and failure to make my points clearer and clearer.

Failures will indicate your football team where improvement is required. These weaknesses will vary from a team to another. It might be in tooling, leadership, lack of vision, roadmap pressure, lack of experience, etc. No matter the reason, it is **primordial** that you take time to review each development cycle with everyone. Otherwise, you might enter the stage below, which is hell to come back from.

**Always failing is not good**

Something was wrong. Then something kept being wrong and that something got culturally accepted even though it is slowing down the team. Skipping failing tests, non automated provisionning management, ***deploying with crossed fingers*** are among few I had the chance to experience. These are triggers that indicates improvements are required. Failing to notice them can bring you to a hellish state where coming back will require a Genesis project.

My answer is basically, go ahead for all the pros. But be aware that this sort of process will require culture and team monitoring in the sense that when things start getting bad, your ***team must*** learn from it or it ***will die*** from it.
> 0 votes

---
Tags: team-role, human-resources
---


# Question
Title: Meaning of metrics in Google Cloud Console
I have a 3-node kubernetes cluster with 3 `n1-standard-1 :1 vCPU, 3,75 Go RAM`, in a gcloud projet. Recently, I ran a test that created quite a few jobs (about 150). Half the jobs were running an hello world script, the other half were failing due to configuration error, and those jobs kept retrying by restarting new pods.

In my console I observed the following graph:

It seems that my cluster is capped at 3% of maximum utilization, if I read the units correctly. What bothers me is that in my test, the amount of pods/utilisation should not be constant, so I was not expecting the utilization to be a flat line like this.

Also, I noticed about about 30 pods were in unschedulable state, so my initial thought was that I reached maximum utilization of the instance.

# EDIT

It is clear now that my test is too poorly designed, so I cannot conclude about maximum utilization so far.

# Answer
> 4 votes
> It seems that my cluster is capped at 3% of maximum utilization, if I read the units correctly. This makes no sense at all, why isn't it capable of reaching 100% ?

Why do you think it does make no sense at all?

> I ran a few tests that created quite a few pods

What is a few tests?

> (each pod running basically an hello world script)

What does this script do? Print helloworld? If that is the case then it could be possible that the cluster will not use 100%.

By the way, how large is the cluster? How many agents? How many memory? So if the cluster possesses 1024GB of memory then it could be the case it is using 3%.

In summary, to indicate whether `something makes no sense at all` a number of facts is required:

* content of the helloworld script
* characteristics of the cluster, e.g. totol cpu, memory, type of disks, storage capacity, number of agents, masters
* definition of `few tests` and `few pods`. How many tests were run, frequency, e.g. 10/s, how many pods

If you would really like to loadtest a k8s cluster, kubernetes has some documentation about this, e.g. this document explains how to load test a k8s cluster and in this case force autoscaling.

---
Tags: kubernetes, google-cloud-platform
---


# Question
Title: How to run Ansible with sudo no password commands with become syntax
When I run ansible against a host machine that has some commands I need to run as root, I can't run these with "become" because it runs the commands as root instead of just running them with sudo.

What I've been doing is just using `command: sudo ...` and then suppressing warnings with

```
args:
  warn: no

```

is there a better, more standardized, way to run a linux commandline with sudo sans password?

# Answer
> When I run ansible against a host machine that has some commands I need to run as root, I can't run these with "become" because it runs the commands as root instead of just running them with sudo.

That's not true; `become` uses a variety of methods for privilege escalation, and defaults to using `sudo`.

Are you perhaps running into one of the known limitations with the become module, like trying to limit sudo access to certain commands?
> 7 votes

# Answer
Did you try -K already? I always use this option and then a sudo password has to be entered
> 2 votes

---
Tags: ansible, passwords
---


# Question
Title: Is it possible to use a JenkinsFile in one repo to build and deploy a related repo?
Our team has two separate repositories for a frontend/backend system. They would like to have these two repositories deployed to the shared team environment together.

My understanding of the JenkinsFile is that it will only work on the repository it committed to and only run when the SCM system sends a request to Jenkins informing it of new changes.

Is it possible to have the JenkinsFile in the two separate repositories communicate with each other so that when one is built the other is built as well and only after both are built, they will then be deployed?

My main goal here is to avoid creating a separate Jenkins job within the Jenkins UI.

# Answer
> 3 votes
Yes, this is pretty easy with Jenkinsfiles with no need for any third-party plugins or anything along those lines: use the built-in Pipeline build step.

I use this to trigger builds of projects in a dependency chain, so that after one project builds successfully, other projects that depend on it will pull in the updated dependency and build against it.

Here is what this looks like in a Jenkinsfile:

```
build repoName

```

where `repoName` is a variable containing the name of the repository you wish to build.

Things can get a little more complicated, such as if you're using folders or Multibranch Pipelines:

```
build "${projectName}/${repoName}/${branchName}"

```

This example builds a job within a Multibranch project where the variables are:

* `projectName` \- the name of my Multibranch project
* `repoName` \- the name of the repository
* `branchName` \- the name of the branch to build, usually `master`

Or this example builds a job with parameters and also triggers the build asynchronously (parent job won't wait for child job to complete before moving on to the next step):

```
build(
  jobName,
  wait: false,
  parameters: [
    [
      $class: 'BooleanParameterValue',
      name: 'someBooleanParameter',
      value: true,
    ],
    [
      $class: 'StringParameterValue',
      name: 'someStringParameter',
      value: 'some string value',
    ],
  ],
)

```

As you can see, the invocation of `build` can get relatively complex. See the documentation linked above for more complete information.

# Answer
> 1 votes
You can use Seed Jobs to accomplish this. Seed Jobs (which are actually freestyle jobs) are included in the Job-DSL Plugin.

---
Tags: jenkins-pipeline
---


# Question
Title: What could be a valid definition of DevOps to introduce it to a novice?
I've done / created lots of SCM related presentations, and now I'm trying to "upgrade" to a DevOps successor of it.

What I always try to do in my presentations, is to come up with an introduction slide which somehow includes the message I want to deliver (and which I then elaborate on in the rest of my presentation). When doing so, I try to answer my own question like "What would me 1 to 3 phrases be that I'd want to use if I got like 10 to 20 secs (only!) to explain it to somebody new to it?\*".

I thought I knew what ***DevOps*** actually means, and what it is about. But I've seen some bizarre usages/contexts of *DevOps* (even on DevOps.SE ...). It makes me wonder if maybe what I think DevOps is, is completely wrong.

So what is generally agreed to be the definition of DevOps?

# Answer
> 13 votes
### DevOps in a nutshell

From Wikipedia:

> DevOps (a clipped compound of "software **DEV**elopment" and "information technology **OP**eration**S**") is a term used to refer to a set of practices that emphasize the collaboration and communication of both software developers and information technology (IT) professionals while automating the process of software delivery and infrastructure changes.
>
> It aims at establishing a culture and environment where building, testing, and releasing software can happen rapidly, frequently, and more reliably.

From Overview:

> Venn diagram showing DevOps as the intersection of development (software engineering), operations and quality assurance (QA)

While there is no single "tool" for DevOps, but rather a set of tools, also known as a DevOps toolchain:

> Illustration showing stages in a DevOps toolchain

### Illustrations of DevOps

Below are a few quotes from some of the questions on DevOps.SE, which all seem to somehow fit / confirm part of the DevOps description above:

### DevOps is NOT a role

Below are a few quotes from some of the questions on DevOps.SE, which all seem to illustrate that DevOps is **NOT** a role:

# Answer
> 13 votes
I have been practising and advising on DevOps as a consultant with different clients for almost five years now, before my current position, I held roles in software development, web operations and systems administration. In my **personal** experience DevOps comes in many flavours.

# Organization Patterns

DevOps Antipatterns:

* **NoOps** and **NoDevs** \- not strictly DevOps in the most strict sense, however, these teams both build and operate software without a dividing line between Development and Operations. The challenges with these teams come down to maturity, Development teams may be expert Software Developers but novice Operators and visa versa.
* **The DevOps Bridge** \- this is where one or more teams are given responsibility for taking work from the development teams and "Productionizing" it to make it operable. The challenge comes down to now there are two hand-offs, i.e. Development → DevOps and DevOps → Operations.
* **The DevOps Team** \- this can, arguably, work if the team has responsibility for building tools that support the DevOps enabled Operating Model, however, it should probably be called a "Tools Team" or "Platform Team".

DevOps Patterns:

* **Embedded DevOps** \- more commonly referred to as Platform Engineering, whereby there is someone within the team who is *accountable but not responsible* for delivering automation, tools and infrastructure for the provisioning and deployment of the solution, sometimes also including operating the software - in my mind, it's the latter that is actually representative of DevOps.
* **Institutionalized DevOps** \- where a project team jointly is responsible for both development and operation of a software package building shared ownership and positive feedback loops.

# Practices

The actual practice of DevOps builds on top of several other practices, namely:

Each of the above practices builds upon the other, it's possible to **not** follow a practice, however, it means an important feedback cycle is missing which may be indicative of a "missed opportunity". The key differentiator between following any of the other practices and DevOps is the *operation of software in production*.

# The Three Ways

In The Phoenix Project Gene Kim and his co-authors describes the three ways of DevOps:

## Systems Thinking

> The First Way emphasizes the performance of the entire system, as opposed to the performance of a specific silo of work or department — this as can be as large a division (e.g., Development or IT Operations) or as small as an individual contributor (e.g., a developer, system administrator).

In my experience starting to get Developers to consider Operational Concerns and Non-functional requirements achieves this goal. This is very much part of the *culture* aspects of DevOps.

## Amplification of Feedback Loops

> The Second Way is about creating the right to left feedback loops. The goal of almost any process improvement initiative is to shorten and amplify feedback loops so necessary corrections can be continually made.

I generally achieve this through Continuous Integration/Delivery/Deployment and shared monitoring and alerting, thus it very much fits in with the **tools** component of DevOps.

## Culture of Continuous Experimentation and Learning

> The Third Way is about creating a culture that fosters two things: continual experimentation, taking risks and learning from failure; and understanding that repetition and practice is the prerequisite to mastery.

This fits very much in the *culture* space, although it depends heavily on tools and process to enable the culture to grow.

# Answer
> 6 votes
I've heard many, many different definitions of DevOps. They include:

* Developers handling operations tasks
* One person doing twice as much work (in the same amount of time)
* Developers and operations teams working with each other
* Operations work for developer tooling (in the vein of "Web Ops")
* A job title for someone who creates and maintains developer tooling
* The use of automation in operations
* The use of public clouds in operations
* A job that combines aspects of operations, development, and quality assurance
* A job that entails helping the development and operations teams work together
* A philosophy of breaking down barriers between teams
* Treating infrastructure as code
* What you get when former software engineers move into operations
* A completely meaningless buzzword

There is no public consensus on what DevOps actually *is*. A few years back we had similar problems with "Agile", and that has a written definition.

When introducing your concepts to a newcomer, I'd focus on introducing the concepts, rather than applying a label, or else they'll end up hearing conflicting definitions and being confused. For instance, if you're trying to talk about infrastructure as code, tell them that you're talking about infrastructure as code. The more specific you can be, the better, as even with agreed-upon definitions most companies focus more on certain parts of a philosophy.

# Answer
> 2 votes
The definition I always use in this situation is the following:

“A software creation culture that emphasizes communication and collaboration between software development and operations teams while automating the software delivery process and infrastructure changes. The goal of DevOps is to make software building, testing and deployment process as frequent, fast and as possible.”

However, along with the definition, it is also important that they understand WHY we need DevOps. Be sure to tell them that DevOps mitigates software defects faster, allows for better resource management, less human error, better version control, stable operating environment etc.

# Answer
> 1 votes
By the following scientific research paper to explore exactly this question,"What is DevOps", the proposed derived definition of DevOps is:

> DevOps is a development methodology aimed at bridging the gap between Development (Dev) and Operations (Ops), emphasizing communication and collaboration, continuous integration, quality assurance and delivery with automated deployment utilizing a set of development practices.

\[Jabbari et al.\] "What is DevOps?: A Systematic Mapping Study on Definitions and Practices" (2016)

# Answer
> -2 votes
Devops is the development practice of writing applications whose **business domain** is operations. Where most application development focuses on building applications which do finance or health care or logistics or cat videos, devops focuses on applications which enable builds, deployments, monitoring, and metrics gathering.

The overriding goal should always be to enable *decision-makers* to become *decision-takers*. Imagine your bank's mobile application. When you request a transfer, it happens when you hit the button. You *made* a decision, then **took** the decision. Same thing with your operations. When the appropriate person decides some work is ready to deploy to production, they should be able to hit a button and the "Right Things Happen". Similarly, they should have all the necessary information they need to make correct business decisions.

It's not about giving business people shell access to servers - that's confusing purpose with implementation. It's about providing the correct knobs and levers with the right information **and the right guard-rails** to the right people so that decision-makers are decision-takers.

---
Tags: terminology, culture
---


# Question
Title: How to move the "All" tab to the right in Jenkins?
Is it possible to move the "All" Jenkins tab from the main view all the way right? It's just my browsing preference to see it at the end to not obstruct my always specific habit of using views. In other words, I would like my first view to be what I see at the beginning on the left.

# Answer
> 6 votes
One option is to add a new view, remove the "All" view, create a new view, select "All" and let it start with "z", e.g. "z-all" as the view is sorted alphabetically.

There is an open issue that indicates that the ordering of tabs in Jenkins is alphabetically sorted and cannot be changed at the moment, e.g. change the order manually.

# Answer
> 3 votes
I don't have currently any Jenkins instance to test it but what worked 4 years ago is that when you rename tab if you place a space (" ") symbol as name prefix and save it does not show up (since HTML hides trailing and leading spaces) but it takes part in alphabet sorting. So you could move part of tabs to the left or right, or even mix them all as you wish.

---
Tags: jenkins
---


# Question
Title: What are the signs of an understaffed DevOps team?
What are the typical signs and signals of a DevOps team being understaffed? How would you justify/explain a request for a new addition to a team?

---

I would love to keep the question generic, but here is some additional information:

We currently have 2 DevOps specialists working together as a team, but the demands and the amount and complexity of products are growing. We are thinking to request a new addition to the team, but having some difficulties explaining and proving why it would be a good idea.

# Answer
> 13 votes
There are four main reasons why you can **feel** your team is understaffed:

1. Poor organization and planning of work
2. Doing work someone else should be doing
3. Doing work that should not be done at all
4. Being actually understaffed

Start with a review of the first three points. Read the **Phoenix Project** on ideas how to do the first. Ask yourself for every task you help anyone with if it should be done at all and if it is you that should be doing the task or if you should simply enable whoever needs it done to do it themselves. This will give you some documentation on why all the work you do is necessary.

Next review the four types of work mentioned in the Phoenix project:

1. **Business Projects** \- what you do for other teams in the organization
2. **Internal Projects** \- what you do to make your work easier in future
3. **Scheduled Maintenance** \- what you do to keep the lights on
4. **Unplanned Interrupts** \- what you do because something went wrong

If the work of your team is sustainable, you will spend roughly the same amount of time on each of the four. If the unplanned work starts to creep up close to 50% of your time, it is a sign you are definitely understaffed.

You should be able to hire to stay about one person ahead of the unplanned work reaching 25% of your time, otherwise, one person leaving will send your entire team into a tailspin you might never recover from. Overprovisioning of people and technology have same reasons and benefits.

# Answer
> 9 votes
**Background:** Besides for providing support to our current infrastructure and to our Developers, we do monthly planning as a DevOps team for what we want to accomplish on top of helping dev teams within sprints and new projects that are launched. However, during the month we often notice extra things that need to be done and improved, which we then add to our backlog. We also are responsible and assist with various other things that fall beyond our scope, but we assist the business were we can :)

**Answer**: As soon as you notice you are not getting round to or postponing lots of tasks especially maintenance, I think that is a good indicator(from what I have experienced). Also, the more new projects and dev teams that come in the thinner the DevOps team gets spread, the more people you will need.

Its super easy just to get caught up in the day to day completing tasks, but I believe its super important (even once a month) to take a step back and assess this.

# Answer
> 6 votes
I actually take a page from the SRE Handbook on this one, which I think is very relevant. DevOps specialties are not meant to grow horizontally with an organization. Rather, if you see that things aren't getting done then it's a signal you're not properly empowering developers to self-service.

Evaluate your processes and see how they align to the commonly accepted DevOps Principles and how well you're following industry best-practices.

# Answer
> 4 votes
I assume this team of two is going from project to project and establishing DevOps stuff there (creating CI/CD pipelines, supporting the other devs creating Dockerfiles, or whatever technology you are using). In other words, type 3, 4, 5 or 6 as per http://web.devopstopologies.com/ .

In this case, a sign of shortage is simply too much workload for those two; too many projects requesting their services; too many tickets; overtime; stress, burnout. These factors should be reasons enough for a responsible leadership to add more capacity. I don't see a DevOps specific sign in this, it's just a function that is understaffed.

Another sign to change something is if you take a good hard look and if you notice that you are creating a "DevOps silo", in which all DevOps know-how is concentrated in those two guys/gals, and everybody else just leans back because those two are "doing DevOps". That is not the point of DevOps. If this is the case, think about the cultural aspect, and modify them to be more evangelists/teachers/coaches for the other teams.

In both cases, the deeper reason of why having DevOps in the first place is a good thing (the general Good Stuff) should be clear to the upper management. If you cannot bring that message across, then scale down the work which your team is doing, by shifting it onto the regular Devs/Ops (as should be the case, anyways).

# Answer
> 1 votes
I was under the impression DevSecOps was a mindset, not a team - if you have a Dev(Sec)Ops "team" you're doing it wrong... I'm trying to wrap my head around putting two "DevOps Engineers" together and calling them a "DevOps Team."

We have development teams, SCM, Application Security and Systems Engineers *all working in tandem for a rapid deployment/release model for pushing code and configuration/system changes through to a given end point - either staging or production*

This has *nothing* to do with any "devOps" engineers, as such.

# Answer
> -1 votes
# Grouping of tasks

An approach we've used in the past in similar situations is to organize the work of a team in 4 major groups of tasks, and allocate the equivalent of 2 FTE (Full Time Equivalents) to (try to) complete those tasks. In our case it was related to running an SCM helpdesk in a mainframe environment, with about 300 developers asking for all sorts of help / interventions from those 2 FTEs. The groups of tasks are organized in 4 possible priorities:

* Tasks of Priority 1 and 2 ***must*** be completed (no excuses / negotiations possible)
* Priority 3 tasks are to be completed "***as soon*** as time permits".
* Priority 4 tasks are to be completed "***if*** time permits".

Read on for more details about the kind of tasks in each of those 4 groups ...

# Task descriptions

### Priority 1 - Operate the helpdesk

* By experts who are easily accessible and always available.
* Via phone, eMail or ticketing system during business hours.
* Compliant to predefined SLAs.
* ITIL-based registration of all helpdesk calls with periodic reporting of all calls.
* Apply emergency customizations (work arounds) for critical issues.

### Priority 2 - Watch duty services

* 24h/day, 7d/week on-call availability.
* Compliant to predefined SLAs.
* Reporting of all watch duty calls.
* Management escalation where needed.

### Priority 3 - Routine maintenance

* Administration.
* Application boarding.
* Housekeeping.
* Performance enhancements.
* Space management.
* Tuning of resource consumption.
* Suggest enhancements for customisations to reduce the number of helpdesk calls and/or watch duty interventions.

### Priority 4 - Fixes and enhancements

* Create and maintain user documentation.
* QA testing of new customisations.
* Develop and implement enhancement requests.
* Participate in DRP-testing.

# Evaluation

If you're using an approach as described above, various things may (will!) start to happen:

* If the team is understaffed, probably most of the time will go to priority 1 and 2 tasks, while it may take a while to get the priority 3 tasks ... and priority 4 tasks might suffer starvation (there appears to never be time for those tasks).
* The more there is (becomes) time available to "*invest*" in priority 4 tasks, the more time needed for priority 1 and 2 tasks will be reduced, so that even more time (of the available budget of 2 FTEs) can be "invested" in priority 4 tasks.
* You'll be amazed to see how, after a while, the number of priority 1 and 2 tasks will go down. And if you do adequate reporting about those tasks, that's something management loves to hear. In our case that number went down from about 300/month, to below 100/month ...
* If however the 2 FTEs appear to never (or hardly) have time for priority 4 tasks, then you have a perfect explanation and proof for your management ... that you're understaffed.

---
Tags: project-management, team-role
---


# Question
Title: Is it possible to show a code coverage badge in bitbucket repositories?
In github it is possible to show a code coverage button. Is this possible in bitbucket as well?

https://github.com/dwyl/repo-badges

**Attempts**

This site was found https://bitbucket.org/site/master/issues/12797/provide-project-status-build-status-badges

Navigating to `https://bitbucket.org/<user>/<project>/addon/pipelines/home#!/badge/branch/<master>?svg=true` returns a dead link

# Answer
> 2 votes
You can certainly show a code coverage badge in a bitbucket repository, but it would have to be from an external source like jenkins, codeclimate, etc.

According to your link, if you are using pipelines they now show a code coverage badge on the project overview page. If you want to add that to your README or display it somewhere external, that functionality has not been implemented.

---
Tags: bitbucket
---