title: Disable creation of labels in Jira?

text:
# Question
Title: Disable creation of labels in Jira?
Following the base instructions I can't create new Jira item through API request due to failing because of trying to create labels. Even if I am using add\_group\_labels: false it tries to create label anyway, isn't this option supposed to disable this behavior? From what I read I need to allow the user to create labels through rest api, but I just don't want the albels on the ticket.

Testing with: curl -H "Content-type: application/json" -X POST -d '{"receiver": "prometheus-customer-non-production", "status": "firing", "alerts": \[{"status": "firing", "labels": {"alertname": "TestAlert", "key": "value"} }\], "groupLabels": {"alertname": "TestAlert"}}' http://xx.x.xxx.xx:9097/alert

Output: {"Error":true,"Status":500,"Message":"JIRA request https://customer.atlassian.net/rest/api/2/issue returned status 400 Bad Request, body \\"{\\\"errorMessages\\\":\[\],\\\"errors\\\":{\\\"labels\\\":\\\"Field 'labels' cannot be set. It is not on the appropriate screen, or unknown.\\\"}}\\""}

Here is the configuration jiralert.yaml:

```
# Global defaults, applied to all receivers where not explicitly overridden. Optional.
defaults:
  # API access fields.
  api_url: https://customer.atlassian.net
  user: 'example@customer.com'
  password: 'customerpassword'

  # The type of JIRA issue to create. Required.
  issue_type: Alert
  # Issue priority. Optional.
  # priority: Low
  # Go template invocation for generating the summary. Required.
  summary: '{{ template "jira.summary" . }}'
  # Go template invocation for generating the description. Optional.
  description: '{{ template "jira.description" . }}'
  # State to transition into when reopening a closed issue. Required.
  reopen_state: "To Do"
  # Do not reopen issues with this resolution. Optional.
  wont_fix_resolution: "Won't Fix"
  # Amount of time after being closed that an issue should be reopened, after which, a new issue is created.
  # Optional (default: always reopen)
  reopen_duration: 0h

# Receiver definitions. At least one must be defined.
receivers:
    # Must match the Alertmanager receiver name. Required.
  - name: 'prometheus-customer-non-production'
    # JIRA project to create the issue in. Required.
    project: customer
    # Copy all Prometheus labels into separate JIRA labels. Optional (default: false).
    add_group_labels: false

# File containing template definitions. Required.
template: jiralert.tmpl

```

Below you can find versions info:

Package release utilized -\> https://github.com/prometheus-community/jiralert/releases/download/1.0/jiralert-1.0.linux-amd64.tar.gz

```
cat /etc/*-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.2 LTS"
NAME="Ubuntu"
VERSION="18.04.2 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.2 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

```

```
kubectl version
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.1", GitCommit:"b7394102d6ef778017f2ca4046abbaa23b88c290", GitTreeState:"clean", BuildDate:"2019-04-08T17:11:31Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.3", GitCommit:"5e53fd6bc17c0dec8434817e69b04a25d8ae0ff0", GitTreeState:"clean", BuildDate:"2019-06-06T01:36:19Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}

```

```
helm list prometheus-operator --tiller-namespace=monitoring
NAME                    REVISION        UPDATED                         STATUS          CHART                           APP VERSION     NAMESPACE
prometheus-operator     3               Thu Apr  9 16:44:37 2020        DEPLOYED        prometheus-operator-8.12.12     0.37.0          monitoring

```

# Answer
Based on the github reply I received, it looks only way is to allow it in Jira as it will spam same alert otherwise:

> Labels are how JIRAlert matches alerts to tickets. Without them, it would simply create a new ticket every time it gets called by Alertmanager for the same alert, i.e. once every few minutes.
>
> If I understand the error correctly, it looks like you'll have to enable labels for your JIRA project. I don't have admin access to a JIRA instance, so I can't tell you exactly how to do that.
> 0 votes

---
Tags: prometheus, jira
---


# Question
Title: Is there a way to tell if an Azure Devops Extension is secure to use?
How do I know that an Azure Devops Extension does not do anything really bad?

A company that I know of uses Microsoft's Azure Devops platform to manage its software development.

Microsoft also offers a "marketplace" where one can get extensions (plugins); third-party software that provides additional features meant to make the platform more useful.

This is helpful, but is there way a to know what such an extension actually does and does not? Can be absolutely 100% certain it does not send any information anywhere? Or changes anything?

**Example**: Azure Devops does a poor job of listing all your git repositories, so there are extensions which does that for you. On installing such an extension, it asks for permission for "Code (read)". From a security point of view, that looks like a yuuuge red flag it's a third-party piece of software that has read access to all the source code in the project.

Now, that's only dangerous if it can also reach the outside world, as long as it cannot send any information at all about my code anywhere, I'm safe.

But can I be certain of that? Am I guaranteed that

* It cannot connect to site or service outside the project
* It cannot connect to or talk to other extensions
* It cannot change its permissions through extension updates.

In short: Can an Azure Devops Extension be determined to be secure?

# Answer
> 1 votes
Can't comment yet, but most extensions host their source code publicly. So you could read the source code to check if something is going on that you don't want.

If the source code is not available, you could check the company for certain certifications like ISO 27001 or the like so you know they get audited.

---
Tags: azure, security, permissions, azure-marketplace
---


# Question
Title: How does Kubernetes determine which node to use?
I am researching using Kubernetes on baremetal for our application. There's an issue I can't find info about. If a worker node dies, how does Kubernetes determine which node(s) to redirect its workload to? What I'm mostly worried about is that if a node dies due to being overloaded with work, Kubernetes will redirect its workload and another node will die due to the workload increase, which leads to a domino effect of nodes dying. How does Kubernetes prevent this?

# Answer
> 2 votes
In order to avoid this kubernetes reserves resources on the nodes for its daemons. See full explanation and configuration guidelines here

---
Tags: kubernetes
---


# Question
Title: How to add job dependency to gerrit trigger in Jenkins pipeline?
We do have this configuration in our GUI based Jenkins project, trying to convert it to Jenkins pipeline, couldn't find how to add `precurse-build` as dependent build as shown in red below.

I added below below pipeline code to trigger the build when`run_job` is added as gerrit comment, and it works good, would anyone please show me how to add `Other jobs on which this job depends` to this?

```
pipeline {
    triggers {
         gerrit dynamicTriggerConfiguration: 'true',
                triggerConfigURL: 'https://some/url/path/trigger.txt',
                serverName: 'gerrit-server',
                triggerOnEvents: [
                    commentAddedContains('^run_job$')
                ]
    }

    stages {
       ...
       ...

```

# Answer
> 1 votes
The "correct" way to determine the names for Pipeline step parameters is to use the Pipeline Snippet generator in the web UI:

https://www.jenkins.io/doc/book/pipeline/getting-started/#snippet-generator

However, based on browsing the source code, I am guessing the name of the parameter you are looking for is `dependencyJobsNames`.

---
Tags: jenkins, jenkins-pipeline, groovy
---


# Question
Title: What is the way to update images in a configuration git repository when doing gitops?
i'm trying to do gitops

I have a Configuration repo containing k8 manifests and different app repos.

1) App repo's CI build docker images and pushes into registry.

2) App repo's CI commits into Configuration repo with newly built image tags.

A gitops CD like ArgoCD is listening to configuration repo and deploys(manual/automatic) it to kubernetes.

Is this the right way to do it ?

# Answer
> 3 votes
> 2. App repo's CI commits into Configuration repo with newly built image tags.

Yes, this makes sense. Instead of using *image tags* you can use *image digest*, since the digest is generated from content, it is immutable, while tags may be updated (intentionally or unintentionally).

This manifest update can be done with multiple tools:

Using yq

```
yq write \
  --inplace deployment.yaml \
  'spec.template.spec.containers(name==myappname).image' \
  gcr.io/my-image-repository/myappname:labelOrDigest

```

Or using a newer tool; kpt with the kpt set command.

# Answer
> 2 votes
Versioning is generally a pain point in microservice management. I just wrote a Medium article how we are dogfooding our tool Reliza Hub to help with this problem - https://medium.com/@taleodor/building-kubernetes-cicd-pipeline-with-github-actions-argocd-and-reliza-hub-e7120b9be870

In short, we have prepared templating engine there which automatically resolves versions based on metadata and approval matrix from Reliza Hub. It then populates k8s definitions for each instance with specific version just for that instance.

If you do it another way, in my experience it usually tends to be manual. Because in most cases it is just to risky to always push latest to your production or even staging.

---
Tags: kubernetes, git, cicd
---


# Question
Title: Default StorageClass set up for Docker EE Kubernetes
We're using Docker EE Kubernetes on-prem set up and noticed that cluster(s) don't have default StorageClass set up. To the best of my knowledge this is supposed to be set up during install/set up, but not done here.

I am aware of steps and have YAML in place to create a PV and StoageClass but trying to understand if there are any best practices and 'right' approach to set that up?

I came across nfs-volume but still not clear if we need a separate NFS drive/server, or can we use one of the nodes for creating the StorageClass(SC)?

Also we don't want the SC to be available for other namespaces or pods as I need to set up SC as a pre-requisite for installing an Ingress Controller.

# Answer
Found out that there is no specific 'default' storageclass(SC) type for Docker EE or others, it depends on the requirement and specific to Kubernetes provider(s). I went ahead and created a sc with 'local' provisioner type.
> 1 votes

---
Tags: docker, kubernetes, ingress
---


# Question
Title: How do you reference a code commit tag in IAM
If I add a tag to a codecommit repo with code commit =\> the repo =\> Settings =\> repository tags =\> add tags how do i reference this tag in IAM

I have in the policy document a statment i am trying to use to reference the tag Im not sure what i use in the resource section to refrence a tag I add.

statement { sid = "AllowCodeCommitAccess" actions = \[ "codecommit:GitPull", \] resources = \["?????????"\] }

Also I haven't actually added the tag yet to codecommit repo as something on aws said it could impact access. I assume that just means If IAM uses it to denny resources but wasn't %100 sure and haven't added it yet.

# Answer
I found what i needed to allow or the eqivalent StringNotEquals to deny useing the tag aws:ResourceTag

```
"Condition": {
                "StringEquals": {
                    "aws:ResourceTag/Key_-1": "Value_-1"
                }
            }

```
> 1 votes

# Answer
Honestly, I'm confused your question. It's meant to prevent git pull IAM user for specific repository or something else?

Here is example of how to deny policy for specific repository that can be attached to your IAM users.

```
{
  "Version": "2012-10-17",
  "Statement" : [
    {
      "Effect" : "Deny",
      "Action" : [
        "codecommit:*"
      ],
      "Resource" : [
        "arn:aws:codecommit:us-east-2:111111111111:myrepo"
      ]
    }
  ]
}

```
> 0 votes

---
Tags: aws-iam
---


# Question
Title: Kubernetes installation, configuration and verification in a local environment
I am a regular user of Docker since a few years and I have the Docker service installed and running locally on my Win10Pro host. It works well. Now I want to get started with Kubernetes so I used the gcloud command-line installation to install Kubernetes and got the following message:

> WARNING: There are older versions of Google Cloud Platform tools on your system PATH. Please remove the following to avoid accidentally invoking these old tools:
>
> C:\Program Files\Docker\Docker\Resources\bin\kubectl.exe

Looks like my Docker installation brought in another version of kubectl which Google advises me to delete. Should I really just delete the file from my Docker installation?

There appears to be two versions of kubectl.exe installed now, one which came with Docker and one which I just installed from Google. Kubernetes in the Docker GUI is not enabled. Should I enable it in the Docker settings GUI?

After a while I tried to install Minikube and start it which results in the following output:

```
C:\WINDOWS\system32>minikube start
* minikube v1.5.2 on Microsoft Windows 10 Pro 10.0.17763 Build 17763
E1114 05:07:00.856628    3836 driver_windows.go:74] Can't find VirtualBox registry entries, is VirtualBox really installed properly? The system cannot find the file specified.
* Automatically selected the 'hyperv' driver
* Creating hyperv VM (CPUs=2, Memory=2000MB, Disk=20000MB) ...
* Preparing Kubernetes v1.16.2 on Docker '18.09.9' ...
* Pulling images ...
* Launching Kubernetes ...
* Waiting for: apiserver
* Done! kubectl is now configured to use "minikube"
! C:\Program Files\Docker\Docker\Resources\bin\kubectl.exe is version 1.14.7, and is incompatible with Kubernetes 1.16.2. You will need to update C:\Program Files\Docker\Docker\Resources\bin\kubectl.exe or use 'minikube kubectl' to connect with this cluster

C:\WINDOWS\system32>

```

But despite the above message, I can run and expose a service as expected:

```
C:\WINDOWS\system32>kubectl create deployment hello-minikube --image=k8s.gcr.io/echoserver:1.10
deployment.apps/hello-minikube created

C:\WINDOWS\system32>kubectl expose deployment hello-minikube --type=NodePort --port=8080
service/hello-minikube exposed

C:\WINDOWS\system32>kubectl get pod
NAME                              READY   STATUS    RESTARTS   AGE
hello-minikube-797f975945-tj4g6   1/1     Running   0          13s

C:\WINDOWS\system32>minikube service hello-minikube --url
http://172.17.100.11:30575

C:\WINDOWS\system32>

```

The service does show its output at the above url. Do you think that my installation and configuration of Kubernetes is alright or is there something that I need to fix? Did I do this correctly?

I did not remove the kubectl.exe from the Docker installation and I installed Minikube without being adviced to do it but it seems correct. What's peculiar is that the client and the server now show different versions:

```
C:\WINDOWS\system32>kubectl version
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.7", GitCommit:"8fca2ec50a6133511b771a11559e24191b1aa2b4", GitTreeState:"clean", BuildDate:"2019-09-18T14:47:22Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"windows/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.2", GitCommit:"c97fe5036ef3df2967d086711e6c0c405941e14b", GitTreeState:"clean", BuildDate:"2019-10-15T19:09:08Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}

C:\WINDOWS\system32>

```

# Answer
You probably don't want to delete the file, just remove it from your path. You will need to look through your System and Account environment variables and look for `PATH` and remove `C:\Program Files\Docker\Docker\Resources\bin\`.

There is a good chance it that that will just work, I have both Docker Desktop and `gcloud` installed on this computer and I don't have the folder above in my `PATH` and both products work just fine without any warnings about old tools hanging about.

If you do continue to run into problems, for example with MiniKube, it can be helpful to use the `Get-Command` PowerShell command to figure out which executable is getting called:

```
Get-Command kubectl | Format-List

```

Will give you the full path to the executable that would be run if you issued the command `kubectl`:

```
Name            : kubectl.exe
CommandType     : Application
Definition      : C:\Program Files\Docker\Docker\Resources\bin\kubectl.exe
Extension       : .exe
Path            : C:\Program Files\Docker\Docker\Resources\bin\kubectl.exe
FileVersionInfo : File:             C:\Program Files\Docker\Docker\Resources\bin\kubectl.exe
                  InternalName:
                  OriginalFilename:
                  FileVersion:
                  FileDescription:
                  Product:
                  ProductVersion:
                  Debug:            False
                  Patched:          False
                  PreRelease:       False
                  PrivateBuild:     False
                  SpecialBuild:     False
                  Language:

```
> 1 votes

# Answer
The following are the instructions for setting up and installing Minikube and its dependencies for use on Windows Pro or Enterprise with Docker Desktop and HyperV.

**Install Kubectl**

1. Create a new directory that you will move your kubectl binaries into. A good place would be `C:\bin`
2. Download the latest `kubectl` executable from the link on the Kubernetes doc page:

https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl-on-windows

3. Move this downloaded `.exe` file into the bin directory you created.
4. Use Windows search to type “env” then select “Edit the system environment variables”
5. In the System Properties dialog box, click “Environment Variables”.
6. In System Variables click on the “Path” Variable and then click “Edit”
7. Click “New” and then type `C:\bin`
8. Drag the newly created path so that it is higher in order than Docker's binaries. This is very important and will ensure that you will not have an out of date `kubectl` client.
9. Click "OK"
10. Restart your terminal and test by typing `kubectl` into it. You should get the basic commands and help menu printed back to your screen. If this doesn't work try restarting your machine.
11. Run `kubectl version` to verify that you are using the newest version and not the out of date v1.10 version.

**Install Minikube**

1. Download the Windows installer here:

https://github.com/kubernetes/minikube/releases/latest/download/minikube-installer.exe

2. Double click the `.exe` file that was downloaded and run the installer. All default selections are appropriate.
3. Open up your terminal and test the installation by typing `minikube`. You should get the basic commands and help menu printed back to your screen. If this doesn't work try restarting your machine.

**Configure HyperV**

1. In Windows Search type "HyperV" and select "HyperV Manager"
2. In the right sidebar click "Virtual Switch Manager"
3. Leave selected "New Virtual network Switch" and "External" and click "Create Virtual Switch"
4. Name the switch "Minikube Switch" (or whatever you would like to name it)
5. Click Apply and acknowledge the "Pending changes" dialog box by clicking "yes"
6. Once the switch has been created, click "Ok"

**Starting Up Minikube**

Since by default Minikube expects VirtualBox to be used, you need to tell it to use the hyperv driver instead, as well as the Virtual Switch created earlier.

Start up a terminal as an Administrator. Then, in your terminal run:

`minikube start --vm-driver hyperv --hyperv-virtual-switch "Minikube Switch"`

**NOTE:** all minikube commands must be run in the context of an elevated Administrator.
> 0 votes

---
Tags: docker, kubernetes, windows, learning
---


title: Regarding Use On Docker

text:
# Question
Title: Regarding Use On Docker
I have some questions regarding docker.

In docker image template, there is template that define for Ubuntu.

1) May I know is it possible to build and run this Ubuntu docker image on Redhat server?

2) Is it possible to use this docker image as environment for development for solution that use debian dependencies packages?

I not sure whether such question can be asked in this forum. if no, please enlighten me to the correct forum to raise such question.

# Answer
> 0 votes
1. You can run that image on Windows/macOS/Linux and so on as long as docker is installed. See: List of all environments.
2. "Development for a solution" is a bit confusing though. It is more like encapsulation on container level (OOP encapsulation). Containers allow a developer to package up an application with all of the parts it needs, such as libraries and other dependencies.

# Answer
> 2 votes
1. Sure. You can run a Linux-based Docker image on any Linux host.
2. Yes, it is possible. You can run & develop in a debian-based docker container.

# Answer
> 0 votes
Yes you can run Docker on any Linux distro. These steps listed below are for Ubuntu Desktop 18 LTS. You can find the full official docs and steps for other Linux distributions here:

Ubuntu: https://docs.docker.com/install/linux/docker-ce/ubuntu/

Fedora: https://docs.docker.com/install/linux/docker-ce/fedora/

CentOS: https://docs.docker.com/install/linux/docker-ce/centos/

Debian: https://docs.docker.com/install/linux/docker-ce/debian/

**Installing Docker**

The Docker docs suggest setting up a Docker repository to install and update from.

This is where you should start:

https://docs.docker.com/install/linux/docker-ce/ubuntu/#set-up-the-repository

After completing the installation steps, test out Docker:

`sudo docker run hello-world`

This should download and run the test container printing "hello world" to your console.

**Installing Docker Compose**

Unlike the Mac and Windows Docker Desktop versions, we must manually install Docker Compose. See the instructions for the installation steps (Click on the tab for Linux)

https://docs.docker.com/compose/install/#install-compose

After completing, test your installation:

`docker-compose -v`

This should print the version and build numbers to your console.

**Run without Sudo**

Follow these instructions to run Docker commands without sudo:

https://docs.docker.com/install/linux/linux-postinstall/#manage-docker-as-a-non-root-user

The docker group will likely already be created, but you still need to add your user to this group.

**Start on Boot**

Follow these instructions so that Docker and its services start automatically on boot:

https://docs.docker.com/install/linux/linux-postinstall/#configure-docker-to-start-on-boot

You may need to restart your system

---
Tags: docker
---


# Question
Title: Unable to use the class defined in src in vars of jenkins shared library
Here is the folder structure in my shared library repo:

```
$ tree src vars
src
├── org
    └── common.groovy

vars
├── testSrc.groovy

```

src has:

```
$ cat src/org/common.groovy
package org.common

class CommonFuncs {
    def sayHi() {
        echo "Hi from CommonFuncs!"
    }

    def sayHello(str) {
        echo "Hello ${str}"
    }
}

```

and vars:

```
$ cat vars/testSrc.groovy
def call() {
    def tst = new org.common.CommonFuncs()
    return tst.sayHi()
}

```

pipeline script in Jenkins has:

```
@Library('jenkins-library') _
pipeline {
    agent { label 'my_servers' }

    stages {
        stage('test') {
            steps {
                testSrc()
            }
        }
    }
}

```

When I run job, I get below error:

```
org.jenkinsci.plugins.workflow.cps.CpsCompilationErrorsException: startup failed:
/var/jenkins_home/jobs/Test/jobs/mave/jobs/test_src/builds/8/libs/jenkins-library/vars/testSrc.groovy: 2: unable to resolve class org.common.CommonFuncs
 @ line 2, column 15.
       def tst = new org.common.CommonFuncs()
                 ^

```

What is wrong in here? I am new to groovy, and this is my first attempt to create a class in shared library src/, any help is appreciated.

# Answer
It looks like Groovy can't find your class at the path specified.

Try renaming the file common.groovy to match the class name and move it to the common folder:

```
src/org/common/CommonFuncs.groovy

```
> 3 votes

---
Tags: jenkins-pipeline, groovy
---


# Question
Title: Reuse Alertmanger Slack Integration into Jiralert template
What kind of changes need to be made to a slack template in order to reuse into Jiralert one? Can I utilize the whole body as it is, can somebody post an example of a working resuse, even for the default example in vendor site.

# Answer
In the end solved it myself, I had to just define jira.summary and description respectively and copy/paste the contents from the slack integration in the given fields.

```
{{ define "jira.summary" }}]
    {{ .Alerts.Firing | len }} {{ .CommonLabels.alertname }} for {{ .CommonLabels.job }}
    {{- if gt (len .CommonLabels) (len .GroupLabels) -}}
    {{" "}}(
    {{- with .CommonLabels.Remove .GroupLabels.Names }}
        {{- range $index, $label := .SortedPairs -}}
        {{ if $index }}, {{ end }}
        {{- $label.Name }}="{{ $label.Value -}}"
        {{- end }}
    {{- end -}}
    )
    {{- end }}
{{- end }}
{{ define "jira.description" }}
    {{ with index .Alerts 0 -}}
    :chart_with_upwards_trend: *<{{ .GeneratorURL }}|Graph>*
    {{- if .Annotations.runbook }}   :notebook: *<{{ .Annotations.runbook }}|Runbook>*{{ end }}
    {{ end }}
    *Kubernetes Cluster:* `default`
    *Prometheus Alert Details*:
        {{ range .Alerts -}}
        *Prometheus Alert:* {{ .Annotations.title }}{{ if .Labels.severity }} - `{{ .Labels.severity }}`{{ end }}
        *Prometheus Alert Description:* {{ .Annotations.message }}
        *Prometheus Alert Details:*
            {{ range .Labels.SortedPairs }}  *{{ .Name }}:* `{{ .Value }}`
            {{ end }}
    {{ end }}
{{- end }}

```
> 1 votes

---
Tags: prometheus, jira, go
---


# Question
Title: Add permissions to readers (DevOps Web Portal)
I have a public Azure DevOps project, and I want to allow everyone to add Work Items.

However, adding the **Readers group as a member of Contributors** for the permissions doesn't change anything, and I can't find *"Add Work Items"* but only the *"Delete Work Items"* toggle for changing its own Readers perms so I am really stuck here.

The contributors perms:

Having to add every mail account to allow people to add work items isn't possible in my situation (Game Engine project) so I really have to find a solution.

# Answer
Usually each projects would have teams defined with the team members. So if you have a team defined then add the team to the contributors list.
> 1 votes

---
Tags: azure-devops, permissions
---


# Question
Title: How to customize a Jenkins job dsl method
I want to customize the powerShell() method in jobDSL

a .groovy file with some job dsl defined jobs

```
def betterPowerShell(def cmd) {
    powerShell("$cmd; exit \$LastExitCode")
}

```

Since I am out of the

```
job {
  step {
  }
}

```

context this is not working...

Maybe I need something like...

```
def betterPowerShell(def cmd) {
    javaposse.jobdsl.dsl.helpers.step.StepContext.powerShell("$cmd; exit \$LastExitCode")
}

```

# Answer
What you want is a Jenkins shared library which will allow you to (among other things) define custom Pipeline DSL steps in a repository you own. See the "Defining custom steps" section in that document.

Here's the example custom step from that document:

```
// vars/sayHello.groovy
def call(String name = 'human') {
    // Any valid steps can be called from this code, just like in other
    // Scripted Pipeline
    echo "Hello, ${name}."
}

```
> 1 votes

---
Tags: jenkins
---


# Question
Title: Ansible: Use inventory_hostname variable in lineinfile module
I'm using Ansible 2.7.9 and I'm having trouble with the regex definition in the lineinfile module. My file has the following line:

```
host.example.com = /path/to/a/directory

```

I want to remove that line. As {{ inventory\_hostname }} contains dots, I'm trying to escape those with `replace`. The task in my playbook looks like this:

```
- name: Remove LE webroot definition
    lineinfile:
      path: "/etc/path/to/config/{{ inventory_hostname }}.conf"
      regexp: "^{{ inventory_hostname | replace('.', '\.') }} = /path/to/a/directory"
      state: absent

```

When I'm executing the playbook, the following error occurs:

```
[...]
The offending line appears to be:
      path: "etc/path/to/config/{{ inventory_hostname }}.conf"
      regexp: "^{{ inventory_hostname | replace('.', '\.') }} = /path/to/a/directory"
                                                      ^ here
We could be wrong, but this one looks like it might be an issue with
missing quotes.  Always quote template expression brackets when they
start a value. For instance:
[...]

```

As you can see, I'm using the double quotes, and all examples that I've found are using this syntax. What I'm doing wrong here? Thanks for any help.

# Answer
The issue seems resolved, If I escape the backslash in the regex too:

```
- name: Remove LE webroot definition
    lineinfile:
      path: "/etc/path/to/config/{{ inventory_hostname }}.conf"
      regexp: "^{{ inventory_hostname | replace('.', '\\.') }} = /path/to/a/directory"
      state: absent

```
> 4 votes

---
Tags: ansible, yaml
---


# Question
Title: Bitbucket Pull Request Builder Configuration Jenkins
I need to integrate BitBucket with Jenkins, so that whenever a PR is opened in BitBucket, it will trigger a build in Jenkins. Can someone help describe the exact configuration step that I should follow in BitBucket and Jenkins to achieve this?

This is my configuration so far:

# Answer
You can install 'Bitbucket Server Webhook to Jenkins' in Bitbucket. https://marketplace.atlassian.com/plugins/com.nerdwin15.stash-stash-webhook-jenkins/server/overview

Once install you will need to enable the 'Post Receive' Hook on the required repository.

Remember to enable '**poll SCM**' for the Jenkins Job.

**Update** You can test the configuration of the plugin here:
> 9 votes

# Answer
This is how I've setup my BitBucket Pull Requests Builder.

**CRON**

Use `* * * * *` and it will poll BitBucket every minute to check for PRs.

**Credentials**

Setup a shared credential that you can use amongst all projects.

**Repo Owner & Name**

You should take the data from your repo URL/SSH like:

`git@bitbucket.org:REPO_OWNER/REPO_NAME.git`

**Branches Filter**

You can choose to only build specific branches or leave it blank to build all.

**CI Identifier & Name**

I set this as the name of our CI account we setup on BitBucket. If you create a new BitBucket account just for your pull requests, grant permission to the repositories that it needs. Then whatever username you set that new account to have enter it as the CI Name and CI Identifier.

Finally, leave the bottom two options ticked.

That's the settings that work for us.
> 4 votes

# Answer
We were able to achieve this using Jenkins multibranch pipeline, upon pull request creation and update pipeline would create a new job. It'll check out the pull request so we can build / run unit tests / sonarqube analysis on it.

Refer following for step by step guide, https://www.whatsuptec.com/trigger-jenkins-pipeline-on-bitbucket-pull-request/
> 0 votes

---
Tags: jenkins
---


# Question
Title: Troubleshooting VPC flow logs with an S3 bucket using SSE-KMS encryption with CMK
I'm using Terraform and trying to set up automatic export of VPC flow logs into an S3 bucket in the same AWS account and region (ca-central-1) that has default encryption turned on with AWS-KMS (using a CMK). Even after trying many permutations of policies for KMS and the S3 bucket, the flow logger still always ends up in `Access error` status.

**Update**: When the S3 bucket is reconfigured to use AES-256 as the default encryption (instead of KMS) the VPC flow logs get written normally. So it's definitely a KMS problem.

The `aws_flow_log` Terraform resource is configured exactly according to the documentation.

```
resource "aws_flow_log" "vpc_flow_log" {
  log_destination_type = "s3"
  log_destination      = "${aws_s3_bucket.awslogs.arn}"
  traffic_type         = "ALL"
  vpc_id               = "${aws_vpc.vpc.id}"
  depends_on           = ["aws_s3_bucket_policy.awslogs"]
}

```

And the result of `aws ec2 describe-flow-logs`:

```
{
  "CreationTime": "2020-05-15T16:22:40.142Z",
  "DeliverLogsErrorMessage": "Access error",
  "DeliverLogsStatus": "FAILED",
  "FlowLogId": "fl-REDACTED",
  "FlowLogStatus": "ACTIVE",
  "ResourceId": "vpc-REDACTED
  "TrafficType": "ALL",
  "LogDestinationType": "s3",
  "LogDestination": "arn:aws:s3:::REDACTED-awslogs",
  "LogFormat": "${version} ${account-id} ${interface-id} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${protocol} ${packets} ${bytes} ${start} ${end} ${action} ${log-status}"
}

```

Curiously, it works fine in my second "sandbox" AWS account where I exclusively use the AWS web console, never Terraform. This account is configured the same way with AWS-KMS on the S3 bucket.

Both accounts seem to have the same configuration, so I can't figure out why it works in the sandbox, but fails in my terraformed account.

---

### S3 bucket

Default encryption is enabled and and Custom KMS arn is selected.

S3 bucket policy includes statements to allow VPC flow logs delivery from `delivery.logs.amazonaws.com` as written in Publishing flow logs to Amazon S3.

```
    {
        "Sid": "AWSLogDeliveryAclCheck",
        "Effect": "Allow",
        "Principal": {
            "Service": "delivery.logs.amazonaws.com"
        },
        "Action": "s3:GetBucketAcl",
        "Resource": "arn:aws:s3:::REDACTED-awslogs"
    },
    {
        "Sid": "AWSLogDeliveryWrite",
        "Effect": "Allow",
        "Principal": {
            "Service": "delivery.logs.amazonaws.com"
        },
        "Action": "s3:PutObject",
        "Resource": [
            "arn:aws:s3:::REDACTED-awslogs/*",
            "arn:aws:s3:::REDACTED-awslogs/AWSLogs/REDACTED/*"
        ],
        "Condition": {
            "StringEquals": {
                "s3:x-amz-acl": "bucket-owner-full-control"
            }
        }
    }

```

### KMS Key Policy

KMS key policy includes a statement that allows usage by VPC Flow logs as instructed by Required CMK key policy for use with SSE-KMS buckets.

```
    {
        "Sid": "Allow VPC Flow Logs to use the key",
        "Effect": "Allow",
        "Principal": {
            "Service": "delivery.logs.amazonaws.com"
        },
        "Action": [
            "kms:ReEncrypt",
            "kms:GenerateDataKey",
            "kms:Encrypt",
            "kms:DescribeKey",
            "kms:Decrypt"
        ]
    }

```

---

I'm at a loss here. What else can I do to troubleshoot this?

# Answer
Turns out I was missing one very important line in my KMS key policy:

```
    resources = ["*"]

```

Now it works fine, and my full policy looks like this:

```
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Allow VPC Flow Logs to use the key",
            "Effect": "Allow",
            "Principal": {
                "Service": "delivery.logs.amazonaws.com"
            },
            "Action": [
                "kms:ReEncrypt",
                "kms:GenerateDataKey",
                "kms:Encrypt",
                "kms:DescribeKey",
                "kms:Decrypt"
            ],
            "Resource": "*"
        }
    ]
}

```
> 1 votes

---
Tags: terraform, amazon-s3, aws-vpc, amazon-kms
---


# Question
Title: Can I remove a docker image and still keep the container?
Trying to save space on the server, and I've noticed that the running containers are much smaller in size than the images I've had to pull to create the containers.

I was wondering about the dependency of the running containers on the images and if it it's possible to remove the images and only keep the containers.

# Answer
There are 2 really major points for Docker containers.

1. You make sure the app dependencies are versioned and encapsulated inside the container. Container runs smoothly eveywhere, as long as you have the Docker daemon.
2. You have the capability to create/destroy a container on any time, also known as ephemeral property. You could test easily, maintain N of them if needed ( Replica Sets in K8S for example are doing just that, Docker Swarm uses services to maintain this number as well ).

The official Docker docs defines an image as

> An image is a read-only template with instructions for creating a Docker container

You can't remove an image, because the container relies on it.

```
docker image rm ubuntu:20.04
Error response from daemon: conflict: unable to remove repository reference "ubuntu:20.04" (must force) - container ecd06b4eb4bf is using its referenced image

```

Then you might force the process

```
 docker image rm -f ubuntu:20.04
Untagged: ubuntu:20.04
Untagged: ubuntu@sha256:747d2dbbaaee995098c9792d99bd333c6783ce56150d1b11e333bbceed5c54d7
Untagged: ubuntu@sha256:8bce67040cd0ae39e0beb55bcb976a824d9966d2ac8d2e4bf6119b45505cee64

```

But that just untags the image, it's still here.

```
docker image ls
REPOSITORY             TAG                                        IMAGE ID            CREATED             SIZE
<none>                 <none>                                     1d622ef86b13        3 weeks ago         73.9MB

```

So to your question, the best practice is to only remove images that are not being used by any containers. If you maintain 2-5 versions of your app, you could safely remove the others. Also you schould strive for minimum size of each image, even in total that schould not bother your available space that much.
> 7 votes

---
Tags: docker
---


# Question
Title: Jenkins Plugin:Extended Choice Parameter from JSON file
**Extended Choice Parameter plugin query**: How to load params dynamically from a JSON file for jenkins build job. Here is my jenkins script to add normal parameters and Extended Choice parameters as well.

p

```
roperties([
        [$class: 'BuildDiscarderProperty', strategy: [$class: 'LogRotator', numToKeepStr: '500']],
        parameters([extendedChoice(
        defaultValue: 'One,Two,Three,Four',
        description: '',
        multiSelectDelimiter: ',',
        name: 'SAMPLE_EXTENDED_CHOICE',
        quoteValue: false,
        saveJSONParameterToFile: false,
        type: 'PT_CHECKBOX',
        value:'One,Two,Three,Four,Five,Six,Seven,Eight,Nine,Ten',
        visibleItemCount: 10)
        ])
])

```

This is working. But I want to be able to load parameters from a JSON file (using Extended Choice Parameter plugin). How can I modify the above script to accomplish that? PS: Here is the reference for Extended Parameter plugin. https://plugins.jenkins.io/extended-choice-parameter/

# Answer
**EDIT:** Updating my answer to include steps on using the Declarative Directive Generator.

You can experiment with the different available configuration options, and sometimes gets hints as to what certain settings provide. For this specific case:

1. Navigate to `<$JENKINS_URL>/directive-generator/`
2. Select `parameters: Parameters` from the **Sample Directive** drop-down.
3. Click the **Add** button and select `extendedChoice: Extended Choice Parameter`
4. Select `JSON Parameter Type` and then experiment with the available options.
5. When finished, click **Generate Declarative Directive** to generate the code block to paste into your pipeline.

You can add the `javascriptFile` parameter and change the `type` to `PT_JSON`.

Using your example:

```
properties([
    [$class: 'BuildDiscarderProperty', strategy: [$class: 'LogRotator', numToKeepStr: '500']],
    parameters([extendedChoice(
    defaultValue: 'One,Two,Three,Four',
    description: '',
    javascriptFile: '/path/to/file.json',
    multiSelectDelimiter: ',',
    name: 'SAMPLE_EXTENDED_CHOICE',
    quoteValue: false,
    saveJSONParameterToFile: false,
    type: 'PT_JSON',
    value:'One,Two,Three,Four,Five,Six,Seven,Eight,Nine,Ten',
    visibleItemCount: 10)

```
> 1 votes

---
Tags: jenkins, jenkins-pipeline, jenkins-plugins
---


# Question
Title: Good IDE or plugins for typical DevOps functions?
I'm starting up the DevOps initiative at my company and trying to decide what IDE and other development tooling to invest the team in for our various repositories.

Language wise we have a lot of Ruby and Groovy, but that's hardly the entire picture. We really don't even know fully yet all the systems we will be working with in the near future, and there are some DSLs in the mix:

* Akamai DSL
* CloudBees / Jenkins logic in Groovy and Jenkinsfile
* Chef
* Ansible, maybe
* Docker, hopefully
* One of several release automation products, not yet chosen
* I would like to use ServerSpec or something like it
* SaltStack DSL
* Kubernetes DSL
* And so on.

Are there any IDEs with plugins to make working with this stuff a little easier?

* that understand the relationships between these entities
* that can mock out DNS and network connections to make unit tests possible for these DSLs
* that can validate DSL syntax or even detect certain logical errors considering multiple of these?

This reddit thread is very discouraging to me as it seems a lot of folks out there are using basic editors with syntax highlighting plugins, which I view as woefully inadequate.

# Answer
> 6 votes
I personally really like PyCharm (same guys who made IntelliJ and Android Studio).

I mostly use Ansible, Bash, and Python. Has good plugins + syntax highlighting for Bash (doesn't everything at this point?), and Ansible. Since it's a native Python IDE, it's also very convenient when writing or troubleshooting python scripts, especially since you don't even need to exit the IDE to run them.

It also has integrations through plugins:

Chef: https://plugins.jetbrains.com/plugin/7548-chef-integration

Ruby: https://plugins.jetbrains.com/plugin/1293-ruby

Both should in theory allow full IDE-like experience with both languages (i.e. resource completion, syntax highlighting, jumping to references, etc).

Really, you can't go wrong with anything by JetBrains if you choose to use any of their other IDEs like IntelliJ.

# Answer
> 4 votes
At the moment I am using Atom which I think is a decent application that has wide support for all of the DSL's you've mentioned at the trade off of some speed.

# Answer
> 0 votes
Answering my own question after our practice is well established. There seems to be no comprehensive "DevOps IDE" in today's market, because the ecosystem of languages is so huge and none of the IDEs do it all. So whatever you choose is only going to have partial coverage at enterprise scale with multiple teams using different tools.

We settled on IDEA because it covers Groovy and Spock very well and we spend more time on those than anything else. Some of the JetBrains IDEs have no free option, currently RubyMine and WebStorm do not. So for Chef cookbooks, web frontends and Node.js, it is necessary to either pay for seats or fall back to Eclipse.

---
Tags: toolchain
---


# Question
Title: How can I retrieve the execution status of parallel triggered child jobs to a pipeline script
I have a pipeline script that executes child jobs in parallel.

Say I have 5 data (a,b,c,d,e) that has to be executed on 3 jobs (J1, J2, J3)

My pipeline script is in the below format

for (int i = 0; i \< size; i++) { def index = i branches\["branch${i}"\] = { build job: 'SampleJob', parameters: \[ string(name: 'param1', value:'${data}'), string(name:'dummy', value: "${index}")\] } } parallel branches

My problem is, say the execution is happening on Job 1 with the data 1,2,3,4,5 and if the data 3 execution is failed on Job 1 then the data 3 execution should be stopped there itself and should not happen on the subsequent parallel execution on Jobs 2 and 3.

Is there any way that I can read the execution status of parallelly execution job status on the Pipeline script so that I can restrict data 3 execution to block in Jobs 2 and 3.

I am quite blocked here for a long time. Hoping for a solution from my community. Thanks a lot in advance.

# Answer
> 2 votes
Use the `result` property of object returned from the `build` step.

This was already asked and has an answer over on StackOverflow. Additionally, this is hinted at in the official build step documentation in the description for the `propagate` parameter ("use the `result` property of the return value as needed"). I haven't tested it as I don't have a test Jenkins instance available right now, but based on those two pieces of information it looks like something like this should do the trick:

```
def b = build('my-job-name', wait: true)

def r = b.result

```

---
Tags: jenkins, jenkins-pipeline, jenkins-plugins
---


# Question
Title: how to run Docker/Jenkins behind a proxy
For the past few days I’ve been scratching my head trying to get the Jenkins docker image to work

The issue appears to be a proxy issue.

> OS: Ubuntu 16.04 LTS
>
> Running behind work proxy
>
> Docker version 19.03.5, build 633a0ea838
>
> Jenkins:https://hub.docker.com/\_/jenkins/

When I run the following command, I can’t access jenkins on localhost:7000

```
docker run --name myjenkins -p 7000:8080 -p 50000:50000 -v /var/jenkins_home jenkins

```

However I can with the following command

```
docker run --name myjenkins -p 7000:8080 -p 50000:50000 --env HTTP_PROXY="http:// localhost:8080" --env HTTPS_PROXY="https:// localhost:8080" -v /var/jenkins_home jenkins

```

The issue is, with the second command, I can access the Jenkins but I can’t install plugins and it asks me to configure the proxy Setting the proxy in the UI didn’t work

My understanding is that Jenkins look for port 8080 by default and because in the command above I’m using port 7000, this is preventing Jenkins from connecting to the internet (correct me if I’m wrong)

So what I did was download the Dockerfile for Jenkins and changed the port from 8080 to 7000

**Dockerfile**

I am now attempting to build the image with the following command

```
docker build --tag jenkins:1.0 .

```

I get the following error

```
curl: (7) Failed to connect to github. com port 443: Connection timed out
The command '/bin/sh -c curl -fsSL https:// github. com/krallin/tini/releases/download/v${TINI_VERSION}/tini-static-amd64 -o /bin/tini && chmod +x /bin/tini && echo "$TINI_SHA /bin/tini" | sha256sum -c -' returned a non-zero code: 7

```

I suspect, this is a proxy issue and I did put proxy arguments in the Dockerfile but not sure if this is what I need to do or whether I’m doing it correctly

Any idea what could be the solution?

Thank you

# Answer
I believe that the problem lies on your proxy environment variables.

```
$ docker run \
    --name myjenkins \
    -p 7000:8080 \
    -p 50000:50000 \
    --env HTTP_PROXY="http:// localhost:8080" \
    --env HTTPS_PROXY="https:// localhost:8080" \
    -v /var/jenkins_home \
    jenkins

```

The values of `$HTTP_PROXY` and `$HTTP_PROXY` are not supposed to have space characters on them and the protocol should be `http`. Additionally, although setting `$HTTP_PROXY` is useful for compatibility with other applications running inside the container, `curl` expects the HTTP proxy environment variable to be written in lowercase, that is, `$http_proxy`. From `curl (1)` manual page:

> ## Environment
>
> The environment variables can be specified in lower case or upper case. The lower case version has precedence. `http_proxy` is an exception as it is only available in lower case.
>
> `http_proxy [protocol://]<host>[:port]` Sets the proxy server to use for HTTP.
>
> `HTTPS_PROXY [protocol://]<host>[:port]` Sets the proxy server to use for HTTPS.
>
> `FTP_PROXY [protocol://]<host>[:port]` Sets the proxy server to use for FTP.
>
> `ALL_PROXY [protocol://]<host>[:port]` Sets the proxy server to use if no protocol-specific proxy is set.
>
> `NO_PROXY <comma-separated list of hosts>` list of host names that shouldn't go through any proxy. If set to a asterisk '\*' only, it matches all hosts.

Therefore the `docker run` command might be written as below:

```
$ docker run \
    --name myjenkins \
    -p 7000:8080 \
    -p 50000:50000 \
    --env HTTP_PROXY="http://localhost:8080" \
    --env http_proxy="http://localhost:8080" \
    --env HTTPS_PROXY="http://localhost:8080" \
    --env https_proxy="http://localhost:8080" \
    -v /var/jenkins_home \
    jenkins

```

However, my solution will certainly not work because the port 8080 inside the container is being used by Jenkins and not by a HTTP proxy. Please, could you provide more information on how your work proxy is set up so I can edit my answer with more accurate details? Are you running a HTTP proxy such as `squid` or `cntlm` in your computer that receives requests from applications running in your workstation and forwards them to the corporate proxy after authentication? If so, you will need to adjust container's network configuration according to Stack Overflow question From inside of a Docker container, how do I connect to the localhost of the machine?.

---

**EDIT:** According to comments, a `cntlm` instance running locally on port `8080/tcp` provides a HTTP proxy service that local applications must use in order to access internet. I believe that the least complex way to expose it to Jenkins container is by specifying `--network=host` parameter and sharing the root network namespace with the Jenkins container. However, Jenkins is set to listen the port `8080/tcp` by default, therefore it must be instructed to listen to an alternative port, such as `7000/tcp`. That can be achieved by passing `--httpPort=#` command line argument to Jenkins via `JENKINS_OPTS` environment variable.

**EDIT 2:** While analyzing the repository code, I've realized that Jenkins is a Java application and I am not sure that JVM recognizes `$http_proxy` and `$https_proxy` environment variables properly. So, to ensure that Jenkins will use `cntlm` as HTTP proxy, I also suggest passing proxy configuration properties to the JVM via the `JAVA_OPTS` environment variable.

**EDIT 3:** The Docker image `jenkins` is deprecated in favor of the `jenkins/jenkins:lts` one. The former image is currently unmaintained and seems to contain a version that is outdated and incompatible with plugins being downloaded.

To sum up, the `docker run` command have to be written as below for Jenkins to work in your environment:

```
$ docker run \
    --name myjenkins \
    --network=host \
    --env HTTP_PROXY="http://localhost:8080" \
    --env http_proxy="http://localhost:8080" \
    --env HTTPS_PROXY="http://localhost:8080" \
    --env https_proxy="http://localhost:8080" \
    --env JAVA_OPTS="-Dhttp.proxyHost=localhost -Dhttp.proxyPort=8080 -Dhttps.proxyHost=localhost -Dhttps.proxyPort=8080" \
    --env JENKINS_OPTS="--httpPort=7000" \
    -v /var/jenkins_home \
    jenkins/jenkins:lts

```

Passing `--env JAVA_OPTS="-Djava.net.useSystemProxies=true"` instead might work. I am not able to perform a try, though.
> 2 votes

---
Tags: docker, jenkins, jenkins-plugins
---


# Question
Title: amazon ec2 ip whitelisted on a third party service provider
Currently we are running an application in a single amazon ec2 machine. On our application we are using some third party solution. For using those solution, we have asked the third party service provider to whitelist our ec2 machine ip.

```
our EC2 machine: IP - 172.12.32.55
Third party solution like global.bank.com and we have told global.bank.com to whitelist our IP

```

Now we want to scale up our application. We are planing to setup multiple node (our application in multiple ec2 machine) and a load balancer.

So my questions is, how i can manage our third party solution provider? Should I give all our ec2 machine ips to our third party solution providers for whitelist? Or is their any other solution to handle the situation?

# Answer
> 3 votes
A possible approach could be:

* Put your instances in a private subnet
* Route incoming traffic through a load balancer
* Route outgoing traffic through a NAT

Outgoing traffic will then use the elastic IP assigned to your NAT as its source IP.

An added benefit of this approach from a security perspective is that your instances won't be directly reachable from the internet, thus reducing your attack surface.

---
Tags: amazon-web-services, amazon-ec2
---


# Question
Title: Ansible: how to display the name of the currently running included_tasks playbook file?
I wrote an Ansible role which does lots of things, it's called "common" and all Ansible hosts are using it.

The directory structure is as follows:

```
./common
│   ├── ./common/README.md
│   ├── ./common/defaults
│   │   └── ./common/defaults/main.yml
│   ├── ./common/handlers
│   │   └── ./common/handlers/main.yml
│   ├── ./common/meta
│   │   └── ./common/meta/main.yml
│   ├── ./common/tasks
│   │   ├── ./common/tasks/base_packages.yml
│   │   ├── ./common/tasks/iptables.yml
│   │   ├── ./common/tasks/locale.yml
│   │   ├── ./common/tasks/main.yml
│   │   ├── ./common/tasks/pip.yml
│   │   ├── ./common/tasks/pip_packages.yml
│   │   ├── ./common/tasks/python.yml
│   │   ├── ./common/tasks/python2_centos6.yml
│   │   ├── ./common/tasks/python3_centos6.yml
│   │   ├── ./common/tasks/repo.yml
│   │   ├── ./common/tasks/selinux.yml
│   │   ├── ./common/tasks/time.yml
│   │   ├── ./common/tasks/vim.yml
│   │   └── ./common/tasks/yum.yml
│   ├── ./common/templates
│   │   ├── ./common/templates/chrony.conf.j2
│   │   └── ./common/templates/ntp.conf.j2
│   ├── ./common/tests
│   │   ├── ./common/tests/inventory
│   │   └── ./common/tests/test.yml
│   └── ./common/vars
│       └── ./common/vars/main.yml

```

main.yml looks like so:

```
---
# tasks file for common
- name: python installation on centos 6 machines
  block:
  - import_tasks: python2_centos6.yml
  - import_tasks: python3_centos6.yml
  when: ansible_distribution_major_version == "6"
- import_tasks: python.yml
  when: ansible_distribution_major_version == "7"
- import_tasks: pip.yml
- import_tasks: pip_packages.yml
- import_tasks: repo.yml
- import_tasks: yum.yml
- import_tasks: selinux.yml
- import_tasks: base_packages.yml
- import_tasks: iptables.yml
  when: ansible_distribution_major_version == "7"
#- import_tasks: time.yml
- import_tasks: locale.yml
- import_tasks: vim.yml

```

The playbook that I'm currently running:

```
---

# local_slave_playbook_c6.yml
- hosts: localhost
  vars:
    ansible_user_shell: /bin/bash
  gather_facts: true
  roles:
    - common
    - consul-client
    - hosts
    #- node-exporter
    - cron
    - jenkins-slave

```

While I run the "common" role on a server, I want to know which imported\_tasks file is currently running for debug matters. So for example, while Ansible "takes care" for the installation of base packages, I want to add "base\_packages.yml" to the title of the task.

I looked through the list of Ansible special variables but couldn't find the variable that I need.

There is one special variable that looks right "{{ ansible\_play\_name }}" but that gives me "localhost".

# Answer
If you want to "add base\_packages.yml to the title of the task" you'll have to add it to the names of the tasks. For example

```
shell> cat base_packages.yml
---
- name: "base_packages: .... "

```

`sed` might help to automate it.
> 1 votes

---
Tags: ansible
---


# Question
Title: How do we move to a modern CICD approach
This may also be a general pulse; but here lies the issue

We use EKS, we have Jenkins, we have Ansible, we have Terraform, we have BitBucket; but none of these really form a pipeline and I'm stuck with answering; do we need more (FluxCD/AgroCD/drone.io) etc... or what we have is enough and it's simply about stringing these things together.

Currently, Jenkins really just does mostly app deployment stuff of *business app* for our devs; but if I wanted to deploy like Fluentd to our clusters or pod security policies, it'll usually require me to manually deploy it.

Really trying to think of a modern CICD approach with security as the forefront, most of our usescases is deploying stuff to kubernetes; but that is not all of our usecase (some scragglers like firewall automation)

# Answer
You have all the necessary tools for the 2 use cases you mentioned. For application development, it looks like you have something already set up. I'm assuming, for time's sake, you have automated both code quality and security scanning of your application and that is all working. If you haven't already started packaging your applications into helm to deploy into kubernetes I suggest you do that. What your looking to do is automate your deployments of pre-built applications and infrastructure into EKS or cloud (aws).

To deploy applications onto your EKS cluster terraform allows you to deploy via helm. So your Jenkins pipeline can deploy your terraform infrastructure which utilize the providers for kubernetes, eks, helm, AWS and more to provision and configure your infrastructure and applications. There is a separate pipeline usually for custom apps, pre-built apps, and infrastructure. Each has it's own quality, security, and resiliency checks needed so you may need some additional tools as you go forward.

For example here is a link to the Hashicorps website on using terraform in automated pipelines. Note it's considered advanced https://learn.hashicorp.com/terraform/development/running-terraform-in-automation

At this point I'm not sure what you're using Ansible for as that's not necessary for EKS and it's services but if you still need it in parallel to terraform there are plugins to Jenkins https://plugins.jenkins.io/ansible/. There's also one for terraform https://plugins.jenkins.io/terraform//

So in short, no need to start adding tools but exploring the ones you have. The way to look at it is this:

* Task Runner and orchestrator for CI/CD: Jenkins
    + this will orchestrate all CI/CD
* Application deployment: Helm
    + this would deploy and version all applications
    + applications can be deployed via jenkins or terraform
* Infrastructure deployment: Terraform
    + initial testing done manually but Jenkins can run terraform to deploy infrastructure and base services.
> 2 votes

# Answer
If you are looking to combine all your CI\CD activities into one single process\pipeline, i would suggest to go with Azure DevOps which also supports Jenkins, Ansible, Kubernetes and almost all the 3rd party\Open source CI\CD tools. So that you could also use your existing tools like Jenkins and Ansible and connect with Azure Devops to make it connected pipelines. I have used this and i was able to do all my CI\CD activity in a single automated pipeline. Have a try maybe.
> 0 votes

---
Tags: jenkins, kubernetes, ansible, terraform, cicd
---


title: Forwarding email for an entire domain

text:
# Question
Title: Forwarding email for an entire domain
I recently acquired the .com of my company name; previously been using .net domain.

I don’t want to switch emails from .net, is it possible to setup anything at my .com to forward to my .net transparently?

I.E. somebody emails sales@mycompany.com and it forwards the message to sales@mycompany.net keeping the original sender name and address.

# Answer
> 1 votes
depending of the registar and where your email server is you can. For example Ionos (ex 1and1) allow user to redirect any un-existing email to another email. This is done with the wildcard \*@yourdomainname.ext

---
Tags: email
---


# Question
Title: Pass environment variables from pipeline job A to pipeline job B
I passed params to pipeline B and set the environment variables from parameters. However, `"${params.URL}"` is evaluated as null.

Pipeline A is declared in `Jenkinsfile`:

```
pipeline {
    agent any

    stages {
        stage('Run on all repos') {

            steps {
                script {
                    sh 'ls'
                    build job: 'run-on-single-repo'
                    parameters:
                            [string(name:'URL', value:'val')]
}}}}}

```

Pipeline B is declared in `run-on-single-repo.groovy`:

```
pipeline {
    agent any

    stages {

        stage('Find missing dependencies') {

            environment {

                URL = "${params.URL}"

            }

            steps {
...

```

Both pipelines are configured in Jenkins as 'Pipeline script from SCM' and work.

How to pass environment variables from pipeline job A to pipeline job B?

# Answer
> 2 votes
The issue was missing comma between `build job` and `parameters`. Below is the correct syntax:

```
...
                    build job: 'run-on-single-repo',
                    parameters:

...

```

---
Tags: jenkins, jenkins-pipeline, jenkinsfile, environment-variables
---


# Question
Title: Failure to build with docker-compose: Invalid signal: SIGQUIT
```
PS D:\>docker-compose up
Building app
Step 1/7 : FROM php:7.4-fpm-alpine
—> 5ae58b49885c
Step 2/7 : RUN apk add --no-cache autoconf curl dpkg-dev dpkg freetype-dev file g++ gcc giticu-dev jpeg-dev libc-dev libmcrypt-dev libpng-dev libjpeg-turbo-dev libxml2-dev libzip-dev make mariadb-dev postgresql-dev pkgconf php7-dev re2c rsync unzip wget zlib-dev
ERROR: Service ‘app’ failed to build: Invalid signal: SIGQUIT

```

Running:

```
docker run -ti 5ae58b49885c apk add --no-cache autoconf curl dpkg-dev dpkg freetype-dev file g++ gcc git icu-dev jpeg-dev libc-dev libmcrypt-dev libpng-dev libjpeg-turbo-dev libxml2-dev libzip-dev make mariadb-dev postgresql-dev pkgconf php7-dev re2c rsync unzip wget zlib-dev

```

seems to work except for the error at the end:

```
[...]
OK: 392 MiB in 105 packages
time="2020-05-23T11:54:06+05:00" level=error msg="Error waiting for container: failed to
shutdown container: container 4e49e4bb8234f213ed580978bc3a4f4cacff549f91affcf82783a2703f341ddb
encountered an error during hcsshim::System::waitBackground: failure in a Windows system
call: The virtual machine or container with the specified identifier is not running.
(0xc0370110): subsequent terminate failed container 4e49e4bb8234f213ed580978bc3a4f4cacff549f91affcf82783a2703f341ddb
encountered an error during hcsshim::System::waitBackground: failure in a Windows system
call: The virtual machine or container with the specified identifier is not running. (0xc0370110)"

```

Version: 19.03.8
OS/Arch: windows/amd64
Experimental: true

Switching containers doesn’t help. What can I do or how to debug?
Thank you.

# Answer
Latest edge release 2.3.1.0 fixed this and many others problems
> 1 votes

---
Tags: docker, docker-compose
---


# Question
Title: How to buy more Azure DevOps pipelines build minutes?
As mentioned on the Azure DevOps official pricing page, the plan provides 1 Free Microsoft-hosted CD/CI parallel job with up to 1800 minutes per month and 1 Free Self-hosted CD/CI parallel job with unlimited minutes per month.

When we exceed the limit of Microsoft-hosted minutes, what I do is configure a VM as the build agent. How can I buy more build minutes after exceeding this 1800 minutes, rather than launching VMs for agents by myself?

# Answer
You can set up billing in your Organization settings in DevOps

https://dev.azure.com/~your organization name~/\_settings/billing
> 2 votes

---
Tags: azure-devops, azure, azure-pipelines
---


# Question
Title: Install apache on awa ec2 from locally installed ansible server on my laptop
I would like to install apache2 remotely on aws ec2 instance from my locally installed ansible server on my laptop. I have also the. pem key of the instance. Now what Ip need to put in /etc/ansible/hosts file inorder to connect ec2 instance from my laptop?

# Answer
You need to put `public ip address` or `hostname` assigned by aws ec2 service, if your aws ec2 instance is public.

If your aws ec2 instance is located in vpc, you won't be able to connect it directly. Either you need to setup routing through internet gateway or you have to setup ansible server in vpc.
> 1 votes

# Answer
If the instance has *Public IP* address and its *Security Group* permits access from your IP use that to tell Ansible where to connect.
> 0 votes

---
Tags: ansible, amazon-ec2, apache-httpd
---


title: Using tmux to host a server

text:
# Question
Title: Using tmux to host a server
So I have some API server that I need to be up 24/7. Is it a bad idea to open up a tmux window and run the server application? When you run a binary in tmux, the binary stays running even when you close the tmux window. The binary also stays running when I disconnect from the SSH session. I like running the server application in tmux because I can easily just attach to the tmux window and have access to a shell where the executable lives. The server constantly prints log messages to standard output, meaning I like to be able to read those messages in my tmux window. However, there is something that just seems off about this method of running a server application. I would prefer if the server could somehow run in the background, but I also want to be able to see the logs that the terminal is printing out at any given time. So, to summarize, is it a bad idea to run a server application in tmux? If so, what do I do instead so that I can still read the server application's logs?

# Answer
What you want can be done. I've seen it done with GNU screen when Tmux wasn't around yet. *However, I would not consider it good style to do it.*

There are better ways to do this. Daemons are simply "normal" processes being run in a particular way. That particular way usually means to fork "twice" (so that the parent/child relationship gets broken) and plugging stdout, stdin and stderr to some file (or `/dev/null` if no logging is desired).

There are tools aplenty out there to achieve what you want while also ensuring that there is a policy in place if the process unexpectedly goes down. After all a service should be available as close to 100% of the time as possible.

Read up on:

Arguably your system will already have one of these or a very similar tool installed. But you don't give enough information for me to provide more detail.
> 2 votes

---
Tags: servers
---


# Question
Title: Best AWS EC2 instance class for general Kubernetes cluster
Hi We are deploying a fairly standard multiple purpose (data replication workers and some nginx web proxies) in an EKS cluster, and wondering what EC2 instance class people recommend for this workload. Probably would look at a base of 3 x (M5|T3|..).large instances with the ability to scale it up. I wonder if others have experience with comparing the compute types.

# Answer
I think the instance class is going to vary based on the kind of workloads you run. You might start out with X but next thing you know your running a bunch of random stuff. You can always run a cluster of different instance classes and then have the scheduler schedule your workloads to the right instance.

M5 and t3 are Nitro based instances. They have a big list of benefits https://aws.amazon.com/ec2/nitro/ . My favorite being that I can log into those instances using my AWS account with no SSH keys.

My personal workloads are more memory-hungry. Since you can't throttle memory like cpu you should factor that in when sizing your instances. I think fewer denser instances are probably more favorable than lots of small instances.

I'm not sure what your use case is for running nginx rever proxies but you may want to look into kubernetes ingress.
> 2 votes

---
Tags: amazon-web-services, kubernetes
---


title: Get tagging to work on AWS CLI

text:
# Question
Title: Get tagging to work on AWS CLI
I am trying to tag commit hash and app version onto the ECS service to make it easier for developers to identify the service with the code.

But I get this error

```
+ aws ecs tag-resource --resource-arn arn:aws:ecs:ap-southeast-1:12344556677:service/master-web-dev-WebECSServices --tags key=COMMITHASH,value=85491def key=VERSION,value=0.2.4
An error occurred (InvalidParameterException) when calling the TagResource operation: Long arn format must be used for tagging operations

```

Is there any problem with the ARN? Why do I always get this issue?

# Answer
> 3 votes
I think this is because your ECS service was created before tagging was available for ECS services.

See this blog post by AWS on how to migrate to the new ARN format. https://aws.amazon.com/blogs/compute/migrating-your-amazon-ecs-deployment-to-the-new-arn-and-resource-id-format-2/

---
Tags: amazon-web-services, amazon-ecs, aws-cli
---


# Question
Title: Does Buildbot support complicated build/deploy scenarios?
I need a tool/framework for CI/CD which can:

1. Support multi-repository project
2. Build and ship only changed part (frontend/backend) using revison hashes to decide
3. Ship some branches (release candidate brances) to test environment and some to production (master)
4. Get version number from repository based on branch name for RC
5. Get version number from repository based on last merged branch in master for production
6. Create build version in bug tracker and mark issues as fixed in created version based on issue status and planned version
7. Notify about new version uploaded on test/production with issues list from bug tracker
8. Get new database migration scripts (based on diff with previously uploaded version using revison hashes) and notify about them & (maybe) execute them

I wrote a script to do this but it lacks interface to do manual build/ship or to monitor build status (only notification at the end of build/deploy)

I have found Bouldbot but not shure it can handle these requirements.

If you know a framework which can handle these requirements without pain, please, name it.

# Answer
> 1 votes
Jenkins supports scripted pipeline which can handle my every requirement. Buildbot can handle these requirements with custom steps, i suppose, but it's too hard if compared to Jenkins' scripted pipeline.

---
Tags: continuous-integration, continuous-deployment, toolchain
---


# Question
Title: Create pull request for docker dependency package if outdated
Platform: GitHub

CI/CD: GitHub Actions

Additional technologies: Docker with Ubuntu base image

I'm creating a Dockerfile and I need to install a package (such as Terraform). Every time the 'current\_version' within "https://checkpoint-api.hashicorp.com/v1/check/terraform" changes, then I want a pull request made to the repo (to change a JSON file with version info).

I've also looked into options such as Dependabot and Renovate. Maybe create a Cron job?

I would appreciate some advice as to how to do this.

Here's how the code would look on a high-level:

```
FROM ubuntu:18.04

RUN apt-get install # install packages such as curl and wget

wget -O terraform.zip https://releases.hashicorp.com/terraform/${JSON_FILE.TF_VERSION}/terraform_${JSON_FILE.TF_VERSION}_linux_amd64.zip

```

# Answer
Assuming a docker file like this...

```
FROM ubuntu:18.04

ARG TF_VERSION=0.12.25

RUN apt-get install wget -y

RUN wget -O terraform.zip https://releases.hashicorp.com/terraform/${TF_VERSION}/terraform_${TF_VERSION}_linux_amd64.zip

```

In your GitHub action you would need to get the current version from your Dockerfile.

```
export CURRENT_TF_VERSION=$(grep -Eo '[[:digit:]]*\.[[:digit:]]*\.[[:digit:]]*' Dockerfile)

```

Then the latest version of terraform.

```
export LATEST_TF_VERSION=$(curl -s 'https://checkpoint-api.hashicorp.com/v1/check/terraform' | jq -r .current_version)

```

Now in bash or using `if` and `outputs` in GitHub actions compare the two values. If they are not the same then find and replace the current version with the latest version in your Dockerfile.

```
sed -i "s/${CURRENT_TF_VERSION}/${LATEST_TF_VERSION}/" Dockerfile

```

Then you can use the create-pull-request GitHub action to create a pull request.

The create-pull-request documentation doesn't mention what happens if you don't have any changes but I asked here. If it doesn't throw an error when you don't have changes then you could simply get the latest value and run sed.

```
sed "s/[[:digit:]]*\.[[:digit:]]*\.[[:digit:]]*/${LATEST_TF_VERSION}/" Dockerfile

```
> 2 votes

---
Tags: docker, continuous-integration, continuous-deployment, github
---


# Question
Title: Istio Virtual service - enforce specific api version
I am having troubles creating an object within specific api group it gets switced from alpha to beta, why? networking.istio.io/v1beta1 vs networking.istio.io/v1alpha3

Note: I can control in the crd virtualservice.

# Answer
I have managed to solve it myself later that day, I had to disable one of the apigroups within the specific crd that is responsible for the given object. In that case it was editing: kubectl edit crd virtualservices.networking.istio.io And then change:

```
  versions:
  - name: v1alpha3
    served: true
    storage: true
  - name: v1beta1
    served: true
    storage: false

```

to

```
  versions:
  - name: v1alpha3
    served: true
    storage: true
  - name: v1beta1
    served: false
    storage: false

```

Then on redeploy I had the correct apigroup which I wanted.
> 0 votes

---
Tags: kubernetes, istio
---


title: Ansible temp dir change

text:
# Question
Title: Ansible temp dir change
Ansible version 2.9.2

Is it possible to change ansible temp dir for one playbook only? I have VMs with r/o root. So, the playbook fails. Even if I run my playbook under root user on target VM it can't create ansible temp files that it creates in current user's home dir on target VM. I know that I can change it in ansible.cfg file, but I need it only for one VM type.

I need to make changes to other not r/o drives.

# Answer
Q: ***"Is it possible to change ansible temp dir for one playbook only? I have VMs with r/o root."***

A: Yes. It's possible. Configure the variable ansible\_remote\_tmp.

In some cases, a better solution to this problem is ANSIBLE\_PIPELINING. Quoting from Risks and limitations of become:

> Use pipelining. When pipelining is enabled, Ansible does not save the module to a temporary file on the client. Instead, it pipes the module to the remote python interpreter’s stdin. Pipelining does not work for python modules involving file transfer (for example: copy, fetch, template), or for non-python modules.

---

Notes:
> The ANSIBLE\_REMOTE\_TMP environment variable has been added to supplement (and override) ANSIBLE\_REMOTE\_TEMP. This matches the spelling of the config value. ANSIBLE\_REMOTE\_TEMP will be deprecated in the future.
> 8 votes

# Answer
I understood with the help of the documentation above how to replace environment variables.

For example:

```
- hosts: hostnames
  remote_user: root
  gather_facts: no
  vars:
    ansible_remote_tmp: /tmp
  tasks:
  ...

```

Also, it's possible to replace in the separate tasks.

Ansible documentation about playbook environment
> 2 votes

---
Tags: ansible
---


title: Stress testing for docker container

text:
# Question
Title: Stress testing for docker container
I want my container to die under high pressure to check monitoring and alerting as well as compose autorestarts. What is best way to simulate heavy load within docker container. I've tried so far

```
root@docker-vm:/# stress --vm 10 --vm-bytes 2048M --vm-hang 20
stress: info: [238] dispatching hogs: 0 cpu, 0 io, 10 vm, 0 hdd
stress: FAIL: [238] (415) <-- worker 243 got signal 9
stress: WARN: [238] (417) now reaping child worker processes
stress: FAIL: [238] (451) failed run completed in 101s

```

for memory pressure (hoping OOM kill for host machine which has 4Gb of ram) and `stress --cpu 100` which is obviously not really harmful.

Is there some tool to achieve that for docker or preferred approach I should use?

# Answer
Just a quick follow-up on pumba. Installation (for Ubuntu):

```
$ curl -SL https://github.com/alexei-led/pumba/releases/download/0.7.2/pumba_linux_amd64 -O
$ sudo mv pumba_linux_amd64 /usr/bin/pumba
$ sudo chmod +x /usr/bin/pumba
$ pumba --version

```

It uses stress-ng under the hood, so same commands apply. Examples:

1. ```
$ pumba stress -d 1m container_name

```

`-d 1m` \- duration of a stress test (1 minute)

`container_name` \- your target container (from `docker ps` output)
2. ```
$ pumba stress -d 30s --stressors "--vm 10 --vm-bytes 512M --vm-hang 20" container_name

```

`-d 30` \- duration 30 seconds

`-stressors` \- parameters passed to *stress-ng* app

In this particular example we create 10 workers spinning on *malloc()/free()* allocating `512MB` each and sleep 20 seconds before freeing the memory.

Default stressor as in first example is `--cpu 4 --timeout 60s`
> 7 votes

# Answer
As for the tool kind of a thing, I think you're looking for something like pumba
> 5 votes

---
Tags: docker, docker-compose, containers
---


# Question
Title: Jenkins executes Selenium test in headless mode while use `Jenkins as Service`
I have recently installed the Jenkins on Windows 10 Machine (Jenkins as Service) and configured one Maven Project (Selenium project which open chrome browser and executes some testcase). If i build the same, all my testcase gets execute in Chrome headless mode instead of opening actual browser.

I've tried below solution :

1. Checking jenkins service option to **`Allow service to interact with desktop`**
2. Setting **`NoInteractiveServices`** flag to `0` in Registry

But nothing seem to work.

while i use Jenkins by executing Jenkins.war file it runs the testcase as expected (non-headless) but not Jenkins as Service.

Any idea why this is happening ?

# Answer
> 1 votes
1. Following command line if you plan to run the same application in both a headless and a traditional environment: more details

`java -Djava.awt.headless=true`
2. You can add environment variable in Jenkins configurations Manage Jenkins -\> Configure System -\> Global Properties -\> Environment Variables section As: Name : DISPLAY Value: 0
3. Installing Jenkins master as a Windows service. This will run browser in other users account to get rid from this do following things:

3.1 Go to your services and stop Jenkins service. To see all your services run services.msc, from there find Jenkins service and disable\stop it.

3.2 Go to jenkins home directory(`C:\Program Files(x86)\Jenkins`) and start cmd with administrator mode and hit `java -jar jenkins.war`

if still your problem persist go through following threads 1. for as service,

2. for general queries

# Answer
> 0 votes
Although it's too late to answer on this thread but it might help someone going forward.

Only solution worked for me is to switch Jenkins from service to standalone. Unfortunately other solutions didn't work for me may be because latest version of Windows 10 doesn't at all allow running GUI apps from services. However, information given by Dev was very useful in understanding why I was facing the problem.

But while switching Jenkins from service to standalone, you'll have to reconfigure the server completely. So to avoid that I've detailed the steps to backup, safe exit (service), startup (standalone) and restore.

1. While service instance is running, install ThinBackup plugin, do the required settings and take the backup of Jenkins jobs, plugins, user data, configuration, etc.
2. Safely stop the Jenkins service instance using the URL - \[jenkins-url\]/exit (e.g. http://localhost:8080/exit).
3. Launch terminal, navigate to Jenkins installation directory and run the command: `java -jar jenkins.war` and let the server start.
4. Use the same url as earlier, do the initial configuration as per recommended plugins, login by creating new credentials.
5. Install ThinBackup plugin on the standalone instance and restore the backup. This will take few minutes to complete.

Now you'll be able to run the tests by actually launching the browser as well as use Jenkins server as configured earlier.

---

**Note:** The standalone instance won't have the auto startup of the server as was the case with service instance. In order to achieve that create a batch file having Jenkins startup command as above and follow the steps described here.

---
Tags: jenkins, continuous-integration, maven, java, selenium
---


# Question
Title: How to add my own columns to Jenkin's Dashboard
I want to add my custom columns, and I also want to add some popups on those columns in Jenkins dashboard.

How does one add new columns, which are not present in job's configuration page?

# Answer
you can use the Jenkins Dashboard View to add the layout of your choice. The page says:

> The dashboard view comes with a number of portlets that you can configure your view with

so you can add top, bottom, or column portlets as you seee fit.
> 2 votes

# Answer
I wish that was possible, but it seems there's nothing yet. However, I found a decent workaround: you can select column 'Project Description'. Of course, in the project description you can write anything. That includes html code, so for example, you can use that to include links in your dashboard.
> 0 votes

---
Tags: jenkins
---


title: Bitbucket Pipeline & Jenkins & Yocto

text:
# Question
Title: Bitbucket Pipeline & Jenkins & Yocto
I want to setup a CD system for a yocto project which is version controlled in a bitbucket repository. I want to write a bitbucket pipeline which triggers a jenkins job on a build server which starts a yocto build. The reason for this is that yocto builds can take up anywhere to 6h for complete rebuilds and thats pretty costintensive with bitbucket pipelines. My idea to test this setup was to use docker containers so I can test the whole setup locally with ngrok. There are some questions I have regarding this:

* does the containerized setup for this make sense?
* can I mark a bitbucket pipeline build from the outside as in progress/completed/failed?
* what is the nicest way to trigger a jenkins job from bitbucket pipelines? Webhooks?
* how can I make jenkins differentiate between two branches when triggered by bitbucket pipelines?

The setup I am pondering with atm are two docker containers. One is running the jenkins server and the other is the build server with a custom image, which I have verified working. I found that you can instrument multiple containers using docker-compose so I came up with this docker-compose.yml:

```
version: '3'
services:
    yocto-server:
        build: .
        container_name: yocto-server
        networks:
            - build-network

    jenkins-master:
        image: jenkins/jenkins
        privileged: true
        working_dir: /home/jenkins
        depends_on:
            - yocto-server
        ports:
            - 8080:8080
        networks:
            - build-network
        links:
            - yocto-server
networks:
    build-network:
        driver: bridge

```

Can jenkins invoke a command via ssh/telnet/docker magic to the second container with this setup? Because as I see it I have to do the following steps:

* mark the build in bbp as in progress
* if repository is present in the container pull and checkout a specific branch
* if repository is not pressent in the container clone it first run the build with parameters provided by jenkins
* depending on the exit code of the build flag the bbp build as completed/successfull
* upload the artifacts to bitbucket

I am very thankful for any inside on this because I have very little knowledge about docker-compose and jenkins.

For clarification I am using Bitbucket Cloud.

# Answer
I think you're asking too many questions in one post for the StackExchange QA format, but I can address the Bitbucket-related questions.

* For automatically setting up jobs for each branch (and PR) in a Bitbucket project or repository, use the official Bitbucket Branch Source plugin. This will also take care of the build status aspect of your question.
* If you are using Bitbucket Cloud, the Bitbucket Branch Source plugin will also take care of creating webhooks in order to automatically trigger builds.
* If you are using Bitbucket Server, you will need some other way of triggering builds. I personally use the pull request notifier plugin for Bitbucket since our workflow is PR-based, but you can use whatever plugin or tool works for you - there are a ton of them out there.
> 1 votes

---
Tags: jenkins, bitbucket
---


# Question
Title: One NAT attachment for two private subnets
I have two NAT gateway attachments for two private subnets in AWS. Is it possible to have one NAT gateway attachment for two private subnets so that I can save one EIP?

# Answer
> 4 votes
Remember that you should be engineering to your requirements around cost, resilience and performance.

If your constraint is around cost, then yes, a single NAT gateway, with both subnets having routing tables that route public IP traffic to the NAT gateway is a viable approach. If you are this cost sensitive, are you sure that AWS is the right hosting environment?

If your constraint is around having all traffic arrive on the internet from a single elastic IP address, then may i suggest that having two NAT gateways in different subnets in different AZ's is a better choice. You will need one elastic IP address on the active NAT gateway, with routing tables for the private subnets pointing internet traffic to the active NAT gateway. In the event of a network failure, have a script that moves the elastic IP address to the standby NAT, and then update the route tables to send internet traffic to the second NAT, making it now the active NAT.

Do not fall into the false economy of thinking that you can spin up the replacement NAT at failover time, as it only takes a few seconds. When large catastrophic failures happen within AWS, the processes for the creation of new resources into the surviving AZ's become far busier than normal, and new requests seem to be queued. you would have an unpredictable RTO with this approach. A single NAT should be treated as accepting that on rare occasions, a 12 hour outage is acceptable.

# Answer
> 2 votes
This is not a good practice. AWS docs explicitly mentions that

> Each NAT gateway is created in a specific Availability Zone and implemented with redundancy in that zone

That means that if all of your private subnets share traffic to a same NAT gateway, in case the AZ where the NAT is located goes down you are loosing your whole 'private environment'.

---
Tags: amazon-web-services, aws-vpc
---


# Question
Title: Porting a docker container to a different Linux host OS
I'm new to Docker. I have looked at many tutorials, however, this is still not clear to me :

For example, If I create a docker image on a system which is using Ubuntu as the **host** OS. Can I then later port and use this same docker image on a different computer using a different flavour of Linux for the host OS, for example CentOS or any other?

# Answer
Yes, as long as you have the Docker daemon running on the CentOS. Docker heavily utilizes a set of kernel features - namespaces and cgroups. That set comes native for all Linux derivatives.

In terms of 'porting' the image, you have 2 widely used options:

1. docker save on your Ubuntu host. Get the saved archive to your CentOS and load the image with docker load
2. The above option is not so convenient for the standard CI/CD stack, so you might just push your image from your Ubuntu host to a so called registry. Once the image is in your registry, you could pull it from your CentOS host.
> 3 votes

---
Tags: docker, containers, linux, centos, ubuntu
---


# Question
Title: How to properly setup a multi-environment release pipeline in Azure YAML pipelines?
We have just migrated from TFS 2018 to Azure DevOps Services and we're having a hard time trying to figure out the new Azure YAML pipelines.

We want to be able to:

* Run a build/test pipeline when a PR is pushed to `develop`.
We've already accomplished this by creating a dedicated YAML pipeline that only builds and runs the tests (no publish artifact or anything). This pipeline is triggered by a "Build Validation" branch policy on the `develop` branch.
* Run the full CI/CD pipeline that will build, test and deploy to DEV and QA environments when a commit is pushed to the `develop` branch.
We've already accomplished this by using a multi-stage pipeline triggered by any change in the branch. Although we're not entirely happy about the duplicated tasks here (this has exactly the same tasks to build and test the project as the above pipeline).
* Deploy to our production-staging environment when a commit is pushed to the `master` branch.
We're stuck here. We're not sure how to do this without creating yet another new pipeline that will do exactly the same as the previous one, but targeting our staging environment. We want to "build once, deploy many". This was easy using TFS build and release pipelines (I guess this is what Azure DevOps calls "Classic Pipelines", right?)

How we can accomplish this workflow without needing to create several similar YAML pipelines, but also following the "build once, deploy many" pattern?

# Answer
> 2 votes
Assuming that deploy to DEV and QA steps look a lot like your deploy to production-staging environments, put the code that does that work into a yaml template and take advantage of parameters to allow you to pass in differing environment settings for DEV, QA, and Staging-Production.

Then in one pipeline create a `stage` for each environment. The `stage` should have a `job` which references your template. You should use conditions on each stage to ensure your code goes to the environment when you want it. Here's a pseudocode example:

```
stages:
- stage: MyDevStage #No condition so this stage always runs
  jobs:
    - template: '../MyTemplate.Yaml
      parameters:
        ServerName: MyDevServer
- stage: MyProdStage
  condition: and(
    succeeded() # if my dev deploy doesn't go well don't deploy to PRD
    , eq(variables['Build.SourceBranch'], 'refs/heads/master') # confirm this is a commit to master
    )
  jobs:
    - template: '../MyTemplate.yaml'
      parameters:
        ServerName: MyProdServer

```

At the top of your pipeline yaml file you'll need to specify a trigger which may look something like this:

```
trigger:
  batch: true
  branches:
    include:
    - master

```

---
Tags: azure-devops, continuous-deployment, azure-pipelines, yaml
---


# Question
Title: How do I install local application dependencies in a Kubernetes manifest?
In a container, there are RUN statements that you can add to a containerfile (such as a dockerfile) to add further dependencies using the package manager like APT or DNF within a container.

What is the proper equivalent to performing this in Kubernetes with a YAML manifest? For example, I'd like to install git in my Jenkins container so as to be able to use the git plugin in that application.

I tried to do this with "command" and "args" in the yaml, but it looks like it isn't executing as superuser? I get permission errors, and cannot install any packages that way.

# Answer
> 2 votes
You can't add run-time dependencies on an already build image using a yaml manifest. You might attach/exec to an already running pod and install them, but it makes no sence, since you need to make sure that when the pod is re-created you do it again. The much better and right way is simply to describe it in your Dockerfile.

Kubernetes is not created for such kind of hacky moves. It simpy maintains state that you describe. Essentially you need to add an instruction that adds git in your Jenkins dockerfile, build your new image and refer to it inside the manifest.

Commands and args are used if you want override the image's default entrypoint and its arguments.

---
Tags: kubernetes, jenkins-plugins
---


# Question
Title: Cannot configure node group in new EKS cluster due to no Node IAM role found
I've created a new EKS cluster, and am trying to add a new node group to it.

I'm stuck on selecting a Node IAM Role from the dropdown.

I followed the link, and created all options for roles I could see fit, still none appear in the dropdown.

Note: currently I don't really care about restricting access, I'm only trying to run some POC against EKS with a test account.

# Answer
This is actually pretty well described in the docs.

> Before you create worker nodes, you must create an IAM role with the following IAM policies:
>
> AmazonEKSWorkerNodePolicy
>
> AmazonEKS\_CNI\_Policy
>
> AmazonEC2ContainerRegistryReadOnly

Also you need to make sure that this role could be assumed by EKS.

```
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "ec2.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

```
> 6 votes

---
Tags: aws-iam, role-based-access-control, eks
---


# Question
Title: Linking nodes in a cluster to run a single instance of a docker container
In this cluster, there are 11 identical nodes, each with 16gb of memory. So far, I've been able to connect each node of the cluster to a docker swarm hosted on the control node. My end goal is to run a single instance of a container that utilizes the resources of each node so the total memory of that container would be 176gb. How should I accomplish this?

Each worker node is running Ubuntu Server 20.04 and the control node is running Ubuntu Desktop 16.04, all with docker engine 19.03.8.

# Answer
This is not really possible with container orchestration. Containers are bound by host and cannot be larger than host machines.

At the same time, depending on your specific tasks there may exist specialized distributed software that may be suitable for your needs - i.e. Apache Spark covers a lot of computational use cases.
> 1 votes

---
Tags: containers, docker-swarm, ubuntu
---