# Question
Title: How to check the status of Ethernet port using ansible?
I want to check status of Ethernet port whether it is up or down on remote server using Ansible?

# Answer
Assuming you can connect to the machine (say, using a local connection), you can interrogate the host facts.

It should expose a list `ansible_interfaces`, which will include the name of your ethernet interface. Each device will have a key in the facts. If your device is called `eth0`, you will have a dict called `ansible_eth0`.

The keys should include:

```
"ansible_eth0" {
  "active": true,
  "device": "eth0",
    "ipv4": {
      "address": "<address>",
      "broadcast": "<address>",
      "netmask": "<netmask>",
      "network": "<network>"
    },
    "ipv6": [
      {
        "address": "<ipv6_network>",
        "prefix": "64",
        "scope": "link"
      }
    ],
    "macaddress": "<hw_address>",
    "module": "<kernel_module>",
    "mtu": XXX,
    "pciid": "XXX",
    "promisc": false,
    "type": "ether"
}

```

The `active` key there will tell you whether the device is active.

I would do it in the following way:

* discover list of devices
* find device that matches a name
* discover status of the device.

A shorter way might be with a `json_query` filter

If you need to check that the device is ready from a remote execution, you could use a `wait_for` on a specific port that should become available when the device is up.
> 1 votes

---
Tags: ansible, automation
---


# Question
Title: SFTP server with third party user management system?
I understand how to set up SFTP servers on Linux and create users and their passwords manually.

So basically, the users get authenticated as normal Linux users get authenticated through `/etc/passwd`. Is there any way to introduce a third party for the user and credentials management?

I am setting up the server on Google Kubernetes Engine. I see AWS provides SFTP as a service and I wanna achieve something similar on GCP.

Here is what AWS is doing:

AFAIK, everything (1,2 and 4) from the above image is in place except AWS IAM and Amazon Route S3 (point-3).

I have given AWS reference just as an example. I am looking for a way to use a third party to manage my SFTP users and credentials instead of normally using `/etc/passwd`.

There are tools available like okta, gsuite, Google Cloud Identity, etc. But I am not sure how to leverage them to manage users in an SFTP server.

# Answer
On a common linux instances the user are authenticated against `/etc/passwd`, because **YOU SPECIFICALLY ASK** for this behavior.

For example, when I install vsftpd, I can see a new file created under `/etc/pam.d`

```
cat /etc/pam.d/vsftpd

# Standard behaviour for ftpd(8).
auth    required        pam_listfile.so item=user sense=deny file=/etc/ftpusers onerr=succeed

# Note: vsftpd handles anonymous logins on its own. Do not enable pam_ftp.so.

# Standard pam includes
@include common-account
@include common-session
@include common-auth
auth    required        pam_shells.so

```

And in `common-auth` you would find something like

`auth [success=1 default=ignore] pam_unix.so nullok_secure`

`man pam_unix:`

```
DESCRIPTION
       This is the standard Unix authentication module. It uses standard calls from the system's libraries to retrieve and set account information as well as authentication. Usually this is obtained from the /etc/passwd and the
       /etc/shadow file as well if shadow is enabled.

```

You can authenticate unix logins against MySQL if you want to https://github.com/NigelCunningham/pam-MySQL

So either you can use some kind of a standard sftp solution with a custom pam module like AWS Cognito https://gist.github.com/ergo70/ca776d1f49c464c07930d94c6e8b01aa

Or you can use a solution which includes another means of authentication. I don't say that it is the best solution, but you can try to install CrushFTP - there is a free limited version - just to see if this is the way you want to go.
> 1 votes

---
Tags: amazon-web-services, kubernetes, linux, gcp
---


# Question
Title: Artifactory archive deposit doesn't extract
I'm attempting to upload a moderately large file system as a 73 gb zip file using curl to my org's artifactory. The goal is for it to extract and replicate the file system in artifactory.

All I can accomplish is getting the zip file on the archive, but there's no extraction of the archive. Is there a problem with my headers? I've tried both -H 'X-Explode-Archive: true" and 'X-Explode-Archive-Atomic=true'.

'X-Explode-Archive: true' resulted in

```
  "errors" : [ {
  "status" : 404,
  "message" : "Explode archive deployment failed. View log for more details."

```

However, I don't see a log file. Do I need to pipe output to create a log or specify something to get it?

For what it's worth, this is my curl call:

```
curl -H 'X-JFrog-Art-Api:<my api key goes here>' -H 'X-Explode-Archive-Atomic=true'   "https://my.orgs.artifactory.path" -T my.local.file.path.zip

```

# Answer
> 2 votes
Ask your Artifactory admin to increase the upload size in **General Configuration**-\>**File Upload Max Size (MB)**

---
Tags: artifactory
---


# Question
Title: connection refused from webserver in container
I've got a simple python http server running on port 8080.

```
class Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        print(self.path)
        self.wfile.write(b'\n')
        return

class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
    """Handle requests in a separate thread."""

if __name__ == '__main__':
    server = ThreadedHTTPServer(('0.0.0.0', 8080), Handler)
    server.serve_forever()

```

fire up 'python3 balancer.py' and it works fine.

> curl -X GET http://10.203.32.90:8080/?path=/project-2

works every time. from the same machine and others.

Now after creating a docker image from this script and deploy it in k8s cluster on port 32003 it does fire up. But when calling the service with

> curl -X GET http://10.203.32.90:32003/?path=/project-2

a connection refused error **sometimes** is returned if it's called from another machine. and when the curl is done from the same machine more often than not a connection refused is returned. sometimes (1 out of 10) it works though.

```
laurijssen@ub:~/pablo$ curl -X GET http://ub:32003/
curl: (7) Failed to connect to ub port 32003: Connection refused
laurijssen@ub:~/pablo$ curl -X GET http://ub:32003/
{"success": "true", "code": 0}

laurijssen@ub:~/pablo$ curl -X GET http://ub:32003/
curl: (7) Failed to connect to ub port 32003: Connection refused

```

The docker file of the webserver:

```
FROM python:3.8-slim-buster
RUN pip install requests
ADD balancer.py /
CMD ["python3", "balancer.py"]

```

The yaml of the k8s service/deployment:

```
apiVersion: v1
kind: Service
metadata:
 name: loadbalance-svc
spec:
 selector:
   role: app
 ports:
   - protocol: TCP
     port: 8080
     targetPort: 8080
     nodePort: 32003
 type: NodePort
 externalName: lb.nl

---

apiVersion: apps/v1
kind: Deployment
metadata:
 name: loadbalancer-deployment
 labels:
   role: app
spec:
 replicas: 1
 selector:
   matchLabels:
     role: app
 template:
   metadata:
     labels:
       role: app
   spec:
     containers:
     - name: web
       image: "10.203.32.90:5000/loadbalancer"
       env:
       imagePullPolicy: Always

```

Dont know how to fix this. What logs to check? What could cause this sometimes it works, sometimes it doesnt behaviour?

# Answer
> 0 votes
After diving into the rabbit hole of netfilter and flannel routing / firewall rules it turns out to be something really simple.

There were multiple deployments with the same label!

When different pods have the same selector labels k8s selects the wrong one sometimes and then gives connection refused. You have to have unique names for the selectors. Weird that this is not logged anywhere which pod is selected or why it is refused. So the following is wrong because 'app' exists over multiple deployments.

```
apiVersion: v1
kind: Service
metadata:
 name: revproxy-svc
spec:
 selector:
   role: app
 type: NodePort

---

apiVersion: v1
kind: Service
metadata:
 name: loadbalance-svc
spec:
 selector:
   role: app
 type: NodePort

---

apiVersion: apps/v1
kind: Deployment
metadata:
 name: loadbalancer-python
spec:
 replicas: 1
 selector:
   matchLabels:
     role: app
 template:
   metadata:
     labels:
       role: app

---

apiVersion: apps/v1
kind: Deployment
metadata:
 name: revproxy
spec:
 replicas: 2
 selector:
   matchLabels:
     role: app
 template:
   metadata:
     labels:
       role: app

```

---
Tags: docker, kubernetes
---


# Question
Title: Scaling a stateful service which has primary/secondary architecture by setting a property
I have a java web stateful service which connects to mongodb and currently it is being deployed as a jar directly on the VM. This service has a property in properties file i.e. `primary`.

If we set `primary` to true then it is primary instance of the application and on this instance mongodb aggregation will happen.

If we set `primary` to false then it is secondary instance of the application and on this instance no mongodb aggregation will happen but it will serve all read and write queries.

Now we are migrating towards k8s deployment.

Challenges are :

**How do we set the value of primary during the k8s pod deployment having multiple replicas ?**

**How do we handle the failure of primary instance of application pod ?**

I am open for any architectural changes suggestions in order to facilitate better deployment structure.

Thanks in advance.

# Answer
> 1 votes
You are looking for the leader election pattern. This can be implemented in many different ways. It typically means that an instance *take a lock* in some form of persistent storage, e.g. a PostgreSQL database, Redis or in etcd itself.

Also see Java Leader Election Pattern and Leader Election in distributed systems

---
Tags: kubernetes, java, scalability
---


# Question
Title: AWS FARGATE ECS - Exact steps to deploy new Docker images with aws CLI
We run services in AWS FARGATE... our Docker builds are tagged before we push them to AWS with a `Makefile`.

I committed changes to a Docker service, ran `make`, and I see the new Docker image tag when I run `aws ecr --profile staging --region us-east-1 describe-images --repository-name "bind"`:

```
{
    "registryId": "8675309",
    "repositoryName": "bind",
    "imageDigest": "sha256:0d3f8d5341286a8e8eb4ddfff2865638b35f8ff83823f8c2cee4bad359d0ffff",
    "imageTags": [
        "0.0.3-0.20210825120405-81e9cffff200"
    ],
    "imageSizeInBytes": 105228639,
    "imagePushedAt": "2021-08-25T12:05:02-05:00",
    "imageManifestMediaType": "application/vnd.docker.distribution.manifest.v2+json",
    "artifactMediaType": "application/vnd.docker.container.image.v1+json"
},

```

I am rather new to FARGATE, and I can't figure out the `aws` CLI to replace the current (outdated) Docker image. Which commands should I run? I'm open to using `aws` CLI or `boto`.

# Answer
The image versions are defined in task definition, so you'll have to create a new task definition (or update the existing one) changing the Docker image tag.

See AWS docs for a task definition template.
> 1 votes

---
Tags: amazon-web-services, python, aws-cli, fargate
---


# Question
Title: GitLab variable false but job is running
I'm triggering job via API, passing custom variables.

Here is example request: `curl -X POST -F token=${TRIGGER_TOKEN} -F ref=${branch} -F "variables[SHOULD_COPY_ASSETS]=false"`

This how job in the .yml file looks:

```
copy-assets:
    stage: copy
    script:
      - echo -e "//registry.npmjs.org/:_authToken=${NPM_TOKEN}" > ./.npmrc
      - npm ci
      - npm run copy-develop-ci
    artifacts:
        paths:
          - assets-shared-tmp
        expire_in: 1 week
    rules:
      - if: $SHOULD_COPY_ASSETS

```

As you can see $SHOULD\_COPY\_ASSETS was set to false, but this job is running anyway. What do I do wrong?

# Answer
> 2 votes
`if: $SHOULD_COPY_ASSETS` checks for the existence of the variable, not its value. You want something like `if: $SHOULD_COPY_ASSETS != "false"`

---
Tags: continuous-integration, gitlab, cicd, build-pipeline
---


title: k8s pods not load balanced

text:
# Question
Title: k8s pods not load balanced
I've got a service running in an on premise k8s environment. It is basically an nginx image, and a call takes like 10 seconds. replicas is two and of course two pods are running. But almost if not every call goes to one pod, queueing up calls. If there's many calls they just wait until all of them are ready, not a single one goes to the other pod.

```
apiVersion: v1
kind: Service
metadata:
 name: revproxy-svc
spec:
 selector:
   role: app
 ports:
   - protocol: TCP
     port: 80
     targetPort: 80
     nodePort: 32001
 type: LoadBalancer
 externalName: my.domain.nl

---

apiVersion: apps/v1
kind: Deployment
metadata:
 name: proxy-deployment
 labels:
   role: app
spec:
 replicas: 2
 selector:
   matchLabels:
     role: app
 template:
   metadata:
     labels:
       role: app
   spec:
     containers:
     - name: web
       image: "10.203.32.90:5000/myproxy"
       resources:
        requests:
          cpu: 500m
       env:
        - name: NODE_IP
          valueFrom:
            fieldRef:
              fieldPath: status.hostIP
       imagePullPolicy: Always

```

What do we need to config to set up proper loadbalancing?

# Answer
> 1 votes
Suddenly found the answer with ipvs.

All that had to be done was put kube-proxy in ipvs mode with rr (roundrobin) scheduler.

```
kubectl -n kube-system edit cm kube-proxy

scheduler: "rr"
mode: ipvs

```

Then restart kube-proxy and we have proper loadbalancing

# Answer
> 2 votes
If you're trying to use a NodePort, then your Service type should be NodePort.

```
     nodePort: 32001
     type: NodePort      #(not LoadBalancer)

```

It is my understanding that the LoadBalancer type is essentially built for cloud environments where the cloud provider provides a load balancer. You are NOT creating a load balancer by creating a LoadBalancer Service type. The load balancer is a separate thing. The Cluster type is *supposed* to load balance internally, but I haven't been able to produce that behavior reliably. Options for the integrated load balancer to work with LoadBalancer type, on premise, seem to be incredibly limited. You can use a standard load balancer, like HAProxy, outside the cluster environment, with NodePort type on the backend.

---
Tags: docker, kubernetes
---


# Question
Title: AWS EC2: RDP without exposing instance to public
I'm planning to use AWS EC2 instances for the development purpose of my web application. So, I will like to disable all the outbound ports except RDP(3389) port. The RDP port will only be open to a particular AWS Service(host). Users can access the host only after logging into AWS Console.

Can anyone please help me understand if there is any service in AWS that can be used as the host?

For reference, Azure has a Bastion Host, which can be used for this purpose.

# Answer
There're many ways to access its EC2 instances as we know and if you want let the users access the instance **only through AWS Console**, then you might want to use AWS SSM (System manager), which will allow the user access only the authorised EC2 instances without Bstion host by the given IAM policy.

Here's instruction that you can consider as below:

1. Install and verify its SSM agent on the Windows or Linux instance. See Manually install SSM Agent on EC2 instances for Windows Server
2. Set a given IAM policy belong to the users or group. Refer to Additional sample IAM policies for Session Manager.

* You can either restrict access based on instance tags or specific instances-id, Just pick one that you want to.

An example of user IAM policy as following:

```
{
    "Sid": "ssmpolicy",
    "Effect": "Allow",
    "Action": "ssm:StartSession",
    "Resource": "arn:aws:ec2:eu-west-1:xxxxxxxxx:instance/*",
    "Condition": {
        "StringLike": {
            "ssm:resourceTag/Name": [
                 "Instance_app_1",
                 "Instance_app_2"
            ]
         }
     }
}

```
3. Lastly, set EC2 IAM Role for the instance with `AmazonSSMManagedInstanceCore` managed policy by AWS.

Then you're good to go safely. Connecting its Windows instance on AWS Console is pretty easy. Please take a look by visitng Connect to a Windows instance using Session Manager

When a user access the EC2 via AWS console, a prompt will be appears like in the picture, and then can access if IAM Role and policy has been given properly in each instance and the users.
> 2 votes

# Answer
You can define security groups in AWS. later you can attach these security groups to instances.

REFERENCE : https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-security-groups.html
> 1 votes

---
Tags: amazon-web-services, amazon-ec2
---


# Question
Title: What means "Reconciliation" from Argo CD
I m looking a metric in Grafana from a official Dashboard from Argo CD, the panel its named "Reconciliation Activity" and use the following metric:

```
argocd_app_reconcile_count

```

I look for a while in internet what means "reconciliation" for Argo CD but I do not find any result.

What does it mean?

# Answer
> 2 votes
In situation like this I tend to visit the project and dig in some code. For this particular case this is the PR that the metric was introduced in - https://github.com/argoproj/argo-cd/pull/1184. You can go from there and do some more research.

`argocd_app_reconcile_count` is metric related to Argo CD Application Controller, basically each time a *reconcile action* is executed the count is incremented.

Reconcile may broadly mean: "state refresh/re-check".

---
Tags: monitoring, cicd
---


title: Re-running a pipeline job if failed

text:
# Question
Title: Re-running a pipeline job if failed
I've heard about the Naginator plugin, but unfortunately it's only for freestyle job type.

Is there any solution for pipeline jobs to be triggered if they failed? I thought about creating a command in the syntax or maybe triggering a freestyle job on the pipeline.

# Answer
The simplest way to do this would probably be to add a `retry` option to the pipeline.

See https://www.jenkins.io/doc/book/pipeline/syntax/#options
> 1 votes

---
Tags: jenkins, jenkins-pipeline, jenkins-plugins
---


# Question
Title: Building a docker container with multiple conda envs
As a novice to the technology I have only a basic knowledge regarding how to build and deploy a docker so i was hoping someone could provide some help.

The situation is the following:

I need to create a docker container with two conda environments and then I want to call a command line from my native os (linux).

Firs i created two conda envs:

```
env-a.yml:

conda create --name env-a
conda activate env-a
conda install -c bioconda samtools
conda env export > env-a.yml

env-b.yml:

conda create --name env-b
conda activate env-b
conda install -c bioconda samtools
conda env export > env-b.yml

```

Next I build my docker container

Dockerfile:

```
    FROM continuumio/miniconda3
     COPY ./env-a.yml /opt/
     COPY ./env-b.yml /opt/
     RUN conda env create -f /opt/env-a.yml
     RUN conda env create -f /opt/env-b.yml
     RUN echo "source activate env-a" > ~/.bashrc
     ENV PATH /opt/conda/envs/env/bin:$PATH

```

my build :

```
docker build -t test ./

```

and now i run it

```
docker run -it test
(env-a) root@1c44fcc2e83d:/#

```

looks good… now i want to mount a location from which my files are read and execute two pipelines each calling tools from different environment:

```
docker run --init -v /media/data/home/booby/test/:/opt/ test /bin/bash -c "source activate env-a && env view -h /opt/my.bam > /opt/my.sam && source activate env-b && samtools view -b /opt/my.sam > /opt/my.bam"

```

but when i mount a volume (-v) , it seams I lose access to my conda envs.

```
/bin/bash: activate: No such file or directory

```

what would be the proper way to set this up?

Thank you

# Answer
> 0 votes
You're overwriting the contents of `/opt/` in the container with the contents from `/media/data/home/booby/test/`. Mount it in some other directory - for example to a subdirectory of `/opt` as shown below as shown below

```
docker run --init -v /media/data/home/booby/test/:/opt/sam /bin/bash -c "source activate env-a && env view -h /opt/sam/my.bam > /opt/sam/my.sam && source activate env-b && samtools view -b /opt/sam/my.sam > /opt/sam/my.bam"

```

---
Tags: docker, dockerfile
---


# Question
Title: Running a background process in Pipeline job
I am looking for a way to run a java process in background using Jenkins declarative pipeline. Below is the snippet of code

```
stage('Deploy'){
        steps{
            script{
                withEnv(['BUILD_ID=dontkill']) {
                    sh "nohup java -jar test-0.0.1-SNAPSHOT.war &"
                }
            }
        }
}

```

Already aware of ProcessTreeKiller of Jenkins and every answer suggest to do the same. I have tried using environment inside the steps block with no luck.

Jenkins version : 2.60.1

Pipeline plugin : 1.1.7

Any help is greatly appreciated.

# Answer
Any one facing the same problem and using pipeline project, set `JENKINS_NODE_COOKIE` instead of `BUILD_ID`.

Dont waste your time setting `HUDSON_COOKIE`, `HUDSON_SERVER_COOKIE`, `JENKINS_COOKIE` or `JENKINS_SERVER_COOKIE`. None of them work for pipeline project.

Refer to https://issues.jenkins-ci.org/browse/JENKINS-28182 for more details.
> 21 votes

# Answer
Alternative solution. Use the `parallel` pipeline step to run your foreground and background processes as equal partners in, well, parallel.

One advantage is that you have easy access to the stdout of the background process in the Blue Ocean UI.

```
stage("Run") {
    parallel {
        stage('k3s') {
            steps {
                sh 'sudo /home/jenkins/k3s server --write-kubeconfig-mode 664 & echo $! > $WORKSPACE/k3s.pid; wait -n $(cat $WORKSPACE/k3s.pid) || true'
            }
        }
        stage('test') {
            steps {
                sh 'sleep 1; ps -p $(cat $WORKSPACE/k3s.pid)'
                sh 'while ! /home/jenkins/k3s kubectl get node; do sleep 1; done'
            }
            post {
                always {
                    sh 'while ps -p $(cat $WORKSPACE/k3s.pid); do sudo kill -9 $(cat $WORKSPACE/k3s.pid); sleep 5; done'
                }
            }
        }
    }
}

```
> 2 votes

# Answer
With a `bat` pipeline step it works with:

```
            script {
                withEnv ( ['JENKINS_NODE_COOKIE=do_not_kill'] ) {

                   bat """
                       @set prompt=\$G\$S
                       cd ${RUN_DIR}
                       start \"title\" my.bat
                       start \"title\" my.cmd
                       start \"title\" my.exe
                       """
                }
            }

```
> 1 votes

---
Tags: jenkins, jenkins-pipeline, jenkinsfile
---


# Question
Title: How to get public key from cert-manager?
I need to get a public key from cert manager from bash. How to do it properly?

I can see the private key in kubernetes secrets; how to get public key that can be safely shared?

# Answer
> 0 votes
There are three ways, all should return the same output. If they are not, something is wrong.

All of this assumes Unix-y shell, and jq, base64, openssl installed. The secret is in `cert-certname`, the website (assuming this protects a website) is at https://example.com

1.

```
kubectl get secret cert-certname -o json  | jq -r '.data."tls.crt"' | base64 -d > tls.crt
openssl x509 -inform pem -in tls.crt -pubkey -noout

```
2.

```
kubectl get secret cert-certname -o json  | jq -r '.data."tls.key"' | base64 -d > tls.key
openssl rsa -in tls.key -pubout

```
3.

```
openssl s_client -connect example.com:443 | openssl x509 -pubkey -noout

```

Note that 1) and 2) creates files containing secrets on your disk; you should delete them afterwards. And never ever put them to git or similar version control.

---
Tags: kubernetes, kubectl, cert-manager
---


# Question
Title: Continuous integration of an operating system
I want to hire a VPS but my problem is that most if not all of the VPS companies out there expect me to work with it as is and if I want to majorly upgrade it, I must, backup everything I have added on top of it, delete the OS and then install a new version of the OS from the VPS hosting company's own interface (the one I used for installing the old version) and then bring all additional information from the backups. In other words, there is no "continuous integration" of an operating system.

A plausible solution to this problem is shared hosting where the staff takes care for everything OS related but if one still wants a VPS and to have a continuously integrated OS both minorly and majorly, what could be done with todays available software, if at all?

# Answer
> 1 votes
You could use Major Public Cloud Providers (AWS, Azure, GCP) or any other Location (Hetzner Cloud, Linode, digitalOcean etc) \[list\]that is supported by Packer.

With **Packer** you can build and maintain your system image and customize it.

Each time you want to upgrade you can build a new image. By building the new image you can choose the base image version you will build on run your modifications on. Image is saved on the provider.

Finally you deploy your new server with the image you created and finally you dispose the old infrastructure, with the help of Terraform.
Since packer read code you treat it like any other code.

---
Tags: continuous-integration, automation, upgrade, software-recommendation
---


title: How do I measure invisible work?

text:
# Question
Title: How do I measure invisible work?
Full disclosure, I’m not a developer or a DevOps professional so some of the terms I use might not be correct. I’m a product marketer who is trying to learn more about a facet of development work.

I’ve read that up to 50% of developers' time is spent on invisible work - those things that aren’t directly traceable back to code commits or other productivity metrics. Interpersonal components like helping teammates or asking for help or discovery components like experimentation or hunting down solutions. I don’t know how accurate these numbers are, so if anyone has different ones with sources, that would be appreciated.

Given the importance of this type of work and how much of it is invisible, what are the strategies or approaches that you use to make this work visible?

# Answer
There is a brilliant talk and blog on this "invisible work" called Being Glue.

> Every senior person in an organisation should be aware of the less glamorous - and often less-promotable - work that needs to happen to make a team successful. Managed deliberately, glue work demonstrates and builds strong technical leadership skills. Left unconscious, it can be career limiting. It can push people into less technical roles and even out of the industry.

If you don't track it, you can feel like you worked so hard all day, and then you look back and have a hard time figuring out what you ACTUALLY achieved. (You definitely get this feeling in management positions too).

A few ways I've used to track this invisible work and make it more visible:

* **Use your calendar religiously**. For example, you get a ping on Slack asking for help / some pairing. Instead of just jumping on right away, create a calendar invite for it, even if its 15 minutes away from now. This allows you to go back over your calendar and see when you paired with folks throughout the week.
* **Use your teams work tracking system**. For discovery style work, this should be tracked in your teams work tracking system (eg. JIRA). You want to be able to produce an artifact this work is done, eg. describing tradeoffs and what you learned. This not only makes the work visible, but it adds additional value to your team as they can understand your decision and tradeoffs you are making.
* **Look for opportunities to pair with and teach others**. Instead of spending many hours hidden away trying to debug a problem, is there anyone on your team that you could hop on a screen share and work on it together? Drop a Slack message in your team channel "hey i'm about to debug this bug if anyone wants to pair with me". Even if nobody joins, at least your work is visible.
* **Keep a diary of what you are working on**. Set a reminder every hour or so just to make some notes. Refer to this when you have your manager 1:1's or team standups to give people an idea where you have been spending your time. Giving specifics is much better than "I was busy speaking to a lot of people".

There isn't a lot of work that stays invisible if you are using these techniques!
> 2 votes

# Answer
1. Keep track of your time in a spreadsheet--I use LibreOffice Calc or Microsoft Excel to do this. I usually round to the nearest 15 minutes in the Start and End columns. The Time column automatically calculates the delta, and the total at the bottom is automatically updated. Example:
2. Also, include a project in your bug tracking/issue software (eg: Jira) for miscellaneous or ad-hoc requests/problems, and track unexpected events there as tickets.

:\[\]
> 1 votes

---
Tags: culture, metrics
---


# Question
Title: Why does Jenkins' Stage View not show my jobs as ending?
I am using Jenkins' remote API (XML & JSON) to submit Jobs, run them & retrieve the logs. When observing the jobs in Stage View, I noticed that (following a change in my "submission function") the jobs are shown as "processing" while they have completed:

How can one deal with such issues? (I'm not a DevOps person, I am developer that happened to be chosen as the one to deal with Jenkins. I had no courses or so, just received a sample script and googled the rest)

Ah, I mentioned that change in my submission function: not knowing anything, when my submitted jobs were rejected (because a job with that name already existed), I created unique names for my jobs. This created lots of workspaces that IT wasn't happy with, so I learned how I could update existing jobs and changed my code accordingly. That's when I started seeing neverending jobs...

# Answer
> 1 votes
Jenkins stage view is in fact showing that the jobs have ended. More specifically, the jobs have failed, as indicated by the "failed" in the bottom-right corner of the stage which failed.

---
Tags: jenkins, jenkins-pipeline
---


# Question
Title: Point docker container DNS to specific port?
I was wondering if it was possible to assign the DNS server a docker container is querying (via docker run --dns, or otherwise) to be on a specific port, something other than 53? I had built a custom python based DNS server for a specific use case and don't have the option to run the container it resides in on the traditional TCP/UDP port 53 (issue with Apache MESOS Marathon unfortunately). Docker image the server is built on is python:3.6.12-slim-buster.

# Answer
You have to bind server's port to something else when running it's container, using the `--publish` or `-p` flag.
Take a look at examples and documentation here.
> 1 votes

---
Tags: docker, networking, dns
---


# Question
Title: Collaboration tools recommendations like Slack
I started my graduation internship two weeks ago. My assignment is to look for a Slack alternative that'll be able to deploy on-premise. A lot of things need to be automated. I already did some research and found some interesting platforms like Mattermost, Zulip, Rocket.chat.

I was wondering if someone has any recommendations that really worth diving into? Also would like your extensive arguments about topics like user interface, integrations/webhooks/bots, security, scalability and so on.

# Answer
TBH,

I would question the reason for it to be on-premise, since nowadays the trend is to look for SaaS solutions (so, nothing to maintain from an IT / Ops point of view).

Slack has a free plan as long as your team is not concerned about message retention or multi-calls/video.

I don't see the reason to have yet another piece of software to be installed/configured/maintained/patched/secured etc when there are so many players out there which lift all of this out (for a price, of course), so if this is a case of "*my boss doesn't want it to be on the cloud/not under his control*" or b\***shit like that, I would question the real need for it to be on-prem.

Even the old Skype4Business (therefore, Microsoft) dropped its on-prem installation in favour of Teams through Office365...

Again, my thought.
> 1 votes

---
Tags: devsecops, slack
---


# Question
Title: How to configure EC2 inbound rules for GitHub Actions deploy
I have a GitHub Actions workflow to deploy a service to an EC2 instance. I have currently opened the SSH access to all IPs but I want to restrict this to IPs used by GitHub Actions runners.

However, by default AWS limits the inbound rules to 60 IP address ranges or you can request an increase to a maximum of 1000 ranges.

The IP list for GitHub Actions is currently ~1900 ranges, so even increasing the limit to 1000 won't help.

How have others solved this problem? Create multiple security groups with 60 different IP ranges? Create fewer security groups with up to 1000 IP ranges? Dynamically create the inbound rule for the specific runner IP address and remove it after?

# Answer
If the running is aware of it's IP, you could run github action step which takes that as an input var to aws cli or Terraform to update the security group applied to the instance you're targetting, then delete the rule when the run is done.
> 1 votes

---
Tags: amazon-ec2, github-actions
---


# Question
Title: Send command to docker image while start in Jenkins file
In scripted Jenkinsfile if I need to send a command to Docker image, I can do like this:

```
docker.image('mysql --default-authentication-plugin=mysql_native_password')

```

How can I do the same in declarative pipeline? Something like the following:

```
agent {
    docker {
        image 'mysql'
        reuseNode true
        args "-e MYSQL_ROOT_PASSWORD=root"
        command "--default-authentication-plugin=mysql_native_password"
    }
}

```

# Answer
You can use withRun.

```
docker.image("mysql:latest").withRun('-e MYSQL_ROOT_PASSWORD=root)

```

Documentation for this is here.
> 0 votes

# Answer
First of all, I'm not sure if you want to keep the container in the block because the container will be stopped as soon as its body exits if you use `withRun`.

So you can run the COMMAND that specified after the IMAGE can be run within either `Image.run([arg, command])` or `withRun` according to Jenkins pipeline-syntax page.

`$ docker run [OPTIONS] IMAGE [COMMAND] [ARG...]`

> Image.withRun\[(args\[, command\])\] {…}
> Like run but stops the container as soon as its body exits, so you do not need a try-finally block.

> Image.run(\[args, command\])
> Uses docker run to run the image, and returns a Container which you could stop later. Additional args may be added, such as '-p 8080:8080 --memory-swap=-1'. Optional command is equivalent to Docker command specified after the image.

If you see the `Image.run()` by visiting Jenkins pipeline-syntax, it's described that you can run the image with Docker COMMAND.

For instance here's an example as below:

```
stages {
        // docker run -p 3306:3306 -e MYSQL_ROOT_PASSWORD=password -d mysql:latest  --default-authentication-plugin=mysql_native_password
        stage("Run MySQL") {
            steps {
                script {
                    docker.image('mysql:latest').withRun('-p 3306:3306 -e "MYSQL_ROOT_PASSWORD=root"', '--default-authentication-plugin=mysql_native_password') {c ->
                        // Run command
                    }
                }
            }
        }

```

Edit: I tried running the MySQL container using `withRun()` with two parameters, and succeed to run the command. Try the code snippet above, and see the pipeline as shown in the attached file.
> 0 votes

---
Tags: docker, jenkins, jenkins-pipeline, jenkinsfile
---


# Question
Title: kubectl retrieve update time for the resource
I'm wondering whether it's possible to get the time where my resource was updated?

e.g. via

```
$ kubectl describe deployment/myapp

```

I can see `CreationTimestamp` but I want to know the last time new changes were introduced. Also is it possible to get a diff between two rollouts? For example with

```
$ kubectl rollout history deployment/myapp

```

I want to check the configuration difference between current version and the previous one. Is there a way I can do that?

# Answer
> 2 votes
Seems like these commands are sufficient

```
$ kubectl rollout history deployment myapp
$ kubectl rollout history deployment myapp --revision=76

```

and to check the difference (use `diff` or colordiff)

```
colordiff \
    <(kubectl rollout history deployment myapp --revision=76) \
    <(kubectl rollout history deployment myapp --revision=81)

```

---
Tags: kubernetes, helm, kubectl, kubernetes-helm
---


# Question
Title: Helm charts: Can't access defined named templates in subcharts
I am having trouble accessing some named templates defined in the main chart from subcharts. As it is in the documentation that every defined named template is global, I'd like to get help on what I am doing wrong here.

Below is the defined named template in ***\_helper.tpl*** of the main chart and I am trying to access it in the **configMap** from the subchart which is resulting into an error of no template.

**\_helper.tpl** inside chart/templates

```
{{- define "main.postgres"}}
    POSTGRES_URL: 172.30.30.39
    POSTGRES_PORT: 5432
{{- end }}

```

**configMap.yaml** inside chart/subchart/templates

```
apiVerions: v1
kind: ConfigMap
metadata:
  name: subchart-config
  data:
    {{- template "main.postgres" .}}

```

**Error**

> error calling include: template: no template "main.postgres" associated with template "gotpl"

# Answer
> 1 votes
I found the issue in your helm templates.

The charts directory inside the helm chart must be `charts`, not `chart`. I tried to simulate your issue and am able to fix it now.

I refactored the directory structure like below

```
.
├── Chart.yaml
├── charts
│   └── subchart
│       ├── Chart.yaml
│       ├── charts
│       ├── templates
│       │   └── configMap.yaml
│       └── values.yaml
├── templates
│   └── _helper.tpl
└── values.yaml

```

I used your `_helper.tpl` file as it is.

```
{{- define "main.postgres"}}
    POSTGRES_URL: 172.30.30.39
    POSTGRES_PORT: 5432
{{- end }}

```

when I am running `helm template .` command, so I am getting the proper results to hope you expected.

```
helm template .
---
# Source: test/charts/subchart/templates/configMap.yaml
apiVerions: v1
kind: ConfigMap
metadata:
  name: subchart-config
  data:
    POSTGRES_URL: 172.30.30.39
    POSTGRES_PORT: 5432

```

# Answer
> 0 votes
I had the same problem. I double checked my folder structure. Instead of **templates** I named my folder ***template***.

---
Tags: helm, kubernetes-helm
---


# Question
Title: Ansible user_module exit with "rc: -25" if UID is large
Ansible `user_module` exit with `rc: -25` if trying to create user with UID provided from Active Directory (AD) domain controller (DC).

**main.yml**

```
- name: Configure local user for example.com
  user:
    name: example
    password: "{{ lookup('file', '/tmp/example_at_host.example.com') }}"
    system: yes         # Defaults to no
    createhome: yes     # Defaults to yes
    uid: '1234567890'   # Default user id of example
    group: '1234500513' # Default group "example users"
    groups: wheel       # Empty by default, here, provide root rights via sudoers wheel
    shell: /bin/bash    # Defaults to /bin/bash
    state: present
  when: ansible_domain == "example.com"
  tags: install,user

```

**stdout**

```
TASK [Configure local user for example.com] *****************************************************************************
fatal: [host.example.com]: FAILED! => changed=false
  msg: ''
  name: example
  rc: -25

```

Even if trying to create the user manually on CLI

```
useradd --system --create-home --uid 1234567890 --gid 1234500513 --groups wheel --shell /bin/bash --comment "Example User" example

```

it fails.

```
File size limit exceeded (core dumped)

```

How to resolve this and get user created with same ID as in AD?

# Answer
After research it turned out that it will be necessary to set a larger file size limit via Ansible module `pam_limits`. The file size limit should larger than UID * 292 bytes.

```
- name: Add or modify soft fsize limits for root domain (for /var/log/lastlog)
  pam_limits:
    domain: root
    limit_type: soft
    limit_item: fsize
    value: '360493823880' # uid * 292
  when: ansible_domain == "example.com"
  tags: install,user,system

```

**Credits to**
> 0 votes

---
Tags: ansible, linux
---


# Question
Title: How do developers handle developing large scaled dockerized applications locally?
I am currently working on a fairly large scale project (upwards of 250 docker containers), which has complexe dependencies on multiple APIs and front-end applications. The staging and production infrastructures are running on Kubernetes. My team is made up of approximately 50 developers all running macOS on company laptops. We are split up into teams that each work on specific parts of the project. Some are front end developers, some are backend developers. Currently we all develop on our laptops locally using either locally installed dependancies or via docker containers (mongoDb, PHP, RabbitMQ…) but we are limited to installing only the specific part of the project we are currently working on as it does not seem possible to run the whole project at the same time on each developers laptop, but this method is very clunky, and can vary from one developers machine to another. I like the whole idea of using docker-compose to have the whole project running locally, but in my experience as soon as you have a few docker volumes running, the MacBooks grind to a halt. Currently switching over to Linux laptops is not an option.

I am looking for advice on how other large scale companies deal with the problem. I would love to have a similar experience to the production systems, possibly running something like Minikube locally, but I don’t think the performance will be any better and it adds a lot of complexity for new developers. We have also thought about running our dev environments on linux VMs but that can quickly be expensive.

How do people in large companies / projects deal with this ?

# Answer
> 1 votes
TL;DR : you need a shared integration environment

---

There are a lot of moving parts here, so giving explicit advice on what to do might be quite impossible. When you're faced with a complex situation, the trick is to know how to *think*, rather than what to *do*.

This question will invite the criticism of generating opinionated answers, because there are many ways to provide a more stable development environment. A guiding principle may be however the idea of Dev / Prod Parity from the 12factor app which states:

> Keep development, staging, and production as similar as possible

If the development teams on the various components follow independent release cycles (which they should, if the application is properly composed of microservices), then it would make sense to have a shared integration environment to develop against. This could be hosted remotely and provide a single environment for all developers.

Of course, this means you need to have access to that shared environment, and hence the network, so you will lose some offline capabilities. However, it seems like the only way to maintain offline development capability would be upgrading each individual member's hardware, so I'm guessing that you will be forced one way or another to do development connected to the network. If you're going to have network access, you might as well have access to a shared integration environment.

The tricky part of course is ensuring that this environment stays current and correct. This is where the practice of continuous delivery comes in. If you have, say, 4 teams working on 4 separate components of the application, then changes to each of those 4 components should be properly versioned and continuously delivered to the shared environment.

A good starting discussion for for developing robust APIs, I found is in the "APIs" edition of Increment.

# Answer
> 0 votes
The "silver bullet" really is multiple environments. DEV/TEST/UAT/STAGE... and all should be periodically synced with prod. Have you considered spinning up ad-hoc/on-demand development environments via CI/CD?

Another more immediate solution is this https://code.visualstudio.com/docs/remote/ssh

Since you're already using docker, you could build ec2 servers and run containers remotely on the remote development servers, then your dev's SSH onto the ec2 servers.

NOTE: On the remote development servers you can install the docker plugin (or any vscode plugin) to improve the development experience.

---
Tags: docker, kubernetes, macos
---


# Question
Title: How can I get client IP on apache insted of Docker container IP?
I've deployed a Matomo application using the official docker image. I'm trying to activate geolocalization (using a plugin) and it's not working.

**I can see that Matomo (apache) only gets the docker container IP, instead of client's IP.**

I can't figure out if the problem comes from a missing Docker config (yes, I'm a newbie) or from a missing Apache config.

Some network details

Testing it locally (same behavior as in production env) :

1. IP address of external default-gateway : `192.168.253.254`
2. IP address of docker host (from ifconfig command) :  `192.168.253.96` | docker0 `172.17.0.1`
3. IP address of matomo container : `172.28.0.3`

All visitor's IP are tracked as `172.28.0.1` (as far as I understand, locally it should be `127.0.0.1`).

I was wondering if the problem could be related to the docker network config? Matomo app and it's DB are deployed using a same bridge network.

# Answer
> 1 votes
When you deploy something via a Docker-container, you have to keep in mind that it's (normally) not-directly-connected to the rest of the world (the Internet, in your case) and that such a connection (container \<=\> Internet) is mediated by the Docker-host (the PC hosting the docker environment and properly running your container) resulting in a chain that looks like:

`<your container> <=> <docker host> <=> <Internet>`

Actually, even if you haven't provided the details, I expect a real scenario like this:

`<your container> <=> <docker host> <=> <your Internet router> <=> <Internet>`

Note: *this is what happens "commonly". Lots of edges can be tuned by properly configuring Docker networking parameters to adapt them to your needs. But in general, you can assume the above.*

So, back to your case, as your container is receiving the HTTP request from the docker-host, this is exactly the reason why you see what you see (the HTTP request coming from the IP of your HOST, and not the one of the real source, on the Internet).

In technical terms, the docker-host is acting --with respect to your container-- as a (sort-of) REVERSE PROXY. In other words:

1. the real source host send the request to your docker-host (to the public IP address associated to the external interface of your docker host);
2. docker-host receive the request and...
3. ...it "proxy" (actually: "forward") the request to your container.

The above is really common within so-called "cloud environment" and, as such, there are several ways to solve the visibility problem you're facing (knowing the real source IP of the HTTP request you're serving).

In your case, I'd suggest going through the official document to get un understanding about the whole concept and related countermeasures.

But...

...unfortunately the above is **not** strightly suited to your case as the docker-host is **not** strictly running a REVERSE-PROXY "server", but, instead, is acting as a PORT-FORWARDER router. Without entering into details (out of scope of this answer), the problem is that you **DON'T** have the place to properly configure the reverse-proxy actions you need.

So, in the end, you have only two options:

1. change the docker configuration, asking the docker-host to logically connect your matomo container to the *EXTERNAL* network, exposing it exactly like any other host on your LAN. Afterwards, you only need to setup your external networking, sending the HTTP traffic to the container IP, and no more the docker-host IP. In other words, the scenario will be: `<your container> <=> <your Internet router> <=> <Internet>`
2. on the docker-host, install a real REVERSE-PROXY (like NGINX) and configure it accordingly to matomo documentation mentioned above. In other words, the scenario will be: `<your container> <=> <NGINX Reverse-proxy running on the docker host> <=> <your Internet router> <=> <Internet>`

*A final note: I've heavily oversimplified the above description and I'm well aware that is **NOT** 100% correct. But it should be enough to help you solving your problem :-)*

---
Tags: docker, apache-httpd
---


# Question
Title: Run multiple (dynamic list) other jobs from a single job
I have 30+ jobs in jenkins that deploy code. For any release, it could be 1 to 30 of those jobs that need to be run. Each job is standalone and deploys its piece independently. For simplicity let's say I have only 5 jobs...

* job A
* job B
* job C
* job D
* job E

As an example, for one release I need to deploy A, B, C and the next one maybe I need to deploy A and E. I was hoping I could make one job that has all 30 deploy jobs as checkboxes (booleans), then I can just run that job, choose the others to run, and go.

Each job accesses git with a branch parameter, but for a release, the branch is always the same for all projects so I figure I can set the branch in the main job and pass the parameter to each deploy job.

Any ideas of the best way to get this to work?

# Answer
I would use a checkboxParam that has the job names values. Then I would iterate over this param and trigger jobs with it from same groovy file. If you need to get the branch name you can use a git param for it.
> 1 votes

---
Tags: jenkins, jenkins-pipeline
---


# Question
Title: Why my docker container is running on swarm manager node instead of worker?
This is my docker-stackk.yaml file.

```
# cat docker-stack.yml
version: "3.4"
services:

  redis:
    image: redis:alpine
    ports:
      - "6379"
    networks:
      - frontend
    deploy:
      replicas: 1
      update_config:
        parallelism: 2
        delay: 10s
      restart_policy:
        condition: on-failure

networks:
  frontend:

volumes:
  db-data:

```

I did not specified any constraint.

```
# docker node ls
ID                            HOSTNAME   STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
snbsjfcg8eu6w88oyinsaa2q0 *   infinity   Ready     Active         Reachable        20.10.8
bi2iblsfma9dno4thvcpetc29     rockpi     Ready     Active         Leader           20.10.8
d5nk1bgdpmo05u1pdlbtxppem     rockpix    Ready     Active                          19.03.8

```

if you see above rockpix is worker node. and infinity is manager.

but when I started the stack the container started on manager node instead of worker node.

```
# docker stack ps redis-stack
ID             NAME                  IMAGE          NODE       DESIRED STATE   CURRENT STATE           ERROR     PORTS
zvd9dyx10ro9   redis-stack_redis.1   redis:alpine   infinity   Running         Running 2 minutes ago

```

# Answer
A manager is able to run containers just as workers do. To prevent workloads from running on the manager, you need to include a constraint. This is different from the kubernetes model where the managers are tainted and you include a toleration to run on managers.

```
version: "3.4"
services:

  redis:
    image: redis:alpine
    ports:
      - "6379"
    networks:
      - frontend
    deploy:
      replicas: 1
      update_config:
        parallelism: 2
        delay: 10s
      restart_policy:
        condition: on-failure
      # add the below constraint
      placement:
        constraints:
          - "node.role==worker"

networks:
  frontend:

volumes:
  db-data:

```

For more on the placement policies and constraints, see:
> 1 votes

---
Tags: docker, docker-swarm
---


# Question
Title: Kibana kubernetes all containers dashboard
How to create dashboard which will show all running containers in a namespace along with how many different channel (header in route/rest api calls) calling the routes of that container?

# Answer
Kibana is used for logging and Grafana for Monitoring. In order to create a dashboard that shows all running containers one could use Grafana.
> 0 votes

---
Tags: kubernetes, kibana
---


# Question
Title: Why should I tag a commit to create a release?
I work on a new CICD, replacing Jenkins-X 2 by GitHub Actions.

After seeing a lot of actions to create a release + changelog, I found that a lot of them are trigger by a tag on a commit to launch the workflow.

For now, I create a new release on every push on master (merged pr), with a version tag (v1.0.0), via conventional commit.

What is the benefit for me or Lead team to manually create a tag to generate a new release + changelog ?

What is the logic behind that ?

**EDIT:** I would like to know what is the benefit of "Creating a tag manually to run a release pipeline (including the changelog)" instead of automate the whole process (e.g: if a PR is merged on master, so it automaticaly create the release)

# Answer
> 1 votes
A changelog is handy as one will see what changes reside in a certain release. A couple of months ago one of my tools was not working anymore. By consulting the Changelog it was possible to find the issue quite fast. Examples:

By creating the Changelog manually, one really has to think what kind of changes were applied to an app. It is also possible to this automatically, but then the commit messages have to be very clear and lean to prevent unclarity.

The changelog itself makes it possible to click on a version and to compare it with a previous one. It facilitates to debug issues in an App when changes were applied.

If one would create a release for every merge then it is possible that there would be a huge number of releases. I have worked in a company where some developers were merging each individual commit to the main branch. Result: 10 releases a day and an artifact repository disk that had to be increased every week. I spoke to the particular devs and they did not see the issue and my CTO did not see the issue as well. At a certain point we decided to remove the automatic release step from the pipeline. Now the PO had to create a release that contained multiple commits. Once a tag was created manually the release pipeline was started and a release was created. Since then we had to increase the disk size only once every quarter.

---
Tags: cicd, release, github-actions
---


title: Are multiline lists valid in Terraform?

text:
# Question
Title: Are multiline lists valid in Terraform?
I currently have the following.

```
module "module-name" {
  source                              = "./directory"
  environment                         = var.environment
  team_names                          = ["team-a", "team-b", "team-c", "team-d"]

```

This clearly makes the `team_names` line very long. Is it valid to change to this?

```
module "module-name" {
  source                              = "./directory"
  environment                         = var.environment
  team_names                          = [
                                          "team-a",
                                          "team-b",
                                          "team-c",
                                          "team-d"
                                        ]

```

# Answer
Yes. According to Types and Values: Lists/Tuples

> List literals can be split into multiple lines for readability, but always require a comma between values. A comma after the final value is allowed, but not required. Values in a list can be arbitrary expressions.

Source:
> 7 votes

---
Tags: terraform
---


# Question
Title: Why is Terraform not picking up locals.tf?
Here are my files so far.

## main.tf

```
data "my_vars" "vars" {
    certificate_arn = local.certificate_arn
}

```

## variables.tf

```
variable "certificate_arn" {
    description = "The ARN for the certificate"
    type = string
}

```

## locals.tf

```
locals {
    certificate_arn = "foo"
}

```

When I run a plan, Terraform asks me for the value of `certificate_arn`, however as far as I can see it is set in locals. What am I missing here?

# Answer
# Because you made it a variable!

This one confused me for a minute too. Your `locals` are defined correctly. You reference it correctly. So why is terraform prompting for it? Just because you define it as a variable.

Since you're not using `var.certificate_arn` anywhere, defining it is a variable is not needed. But since you did define it as a variable, terraform is going to diligently ask for it. Getting rid of the variable definition should fix your issue.

If later you want to turn it into a variable, fine. But you'll need to get rid of the local and reference `var.` instead of `local.`.
> 2 votes

---
Tags: terraform
---


# Question
Title: Explaining ingress installation with HELM
I was trying to install ingress helm charts but found a couple of tutorials but they just insert the different steps without proper explanation, But I want to know what exactly each step means:

```
kubectl create namespace ingress-nginx
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm install ingress-nginx ingress-nginx/ingress-nginx -n ingress-nginx

```

In a simple words and correct me If I am wrong,
`kubectl create namespace ingress-nginx` -\> this creates a namespace called `ingress-nginx`

`helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx` -\> This add a chart repository called ingress-nginx from the URL mentioned (Am I free to name it to something else!?)

`helm install ingress-nginx ingress-nginx/ingress-nginx -n ingress-nginx` -\> install the chart named `ingress-nginx` (first one) , in the namespace called `ingress-nginx` (last one), To be honest I can't understand it will, can anyone explains it?

and sometimes I see `helm repo update` after adding the chart-repo with `helm repo add` (inginx documentation is doing it) and sometimes not , what is the difference? is there any difference?

# Answer
> 1 votes
You explained what each does pretty well! I strongly suggest reading the documentation, especially for the commands, don't rely on tutorials only.helm commands `helm repo add` =\> This is what it stands for, it adds a new helm chart repository. `helm repo update` =\> Update gets the latest information about charts from the respective chart repositories. When you initially start or want to use a chart you will use `add`, let us say 6 months have passed, the chart has bugs that are now fixed in the latest version, you then use `update` to get the latest version. And you can use different names, see example =\> helm examples

So what you explained yourself is pretty good, really.

1. You create a Kubernetes namespace.
2. You add a new repo for nginx.
3. Then you install that nginx repo in the new namespace you created in point (1). - otherwise, it will probably go in the default namespace. I always specify the namespace btw.

One thing to note, when you add a repo, it gets added locally and from there it gets installed. So when you do installations, helm goes through your local packages that are stored/cached and it installs them where you point them to.

# Answer
> 0 votes
> In a simple words and correct me If I am wrong, kubectl create namespace ingress-nginx -\> this creates a namespace called ingress-nginx

Correct !

> helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx -\> This add a chart repository called ingress-nginx from the URL mentioned (Am I free to name it to something else!?)

Correct too ! Yes, you can name it has you like, it's just the name that you'll use to refer to this repo url (e.g: in a requirement file)

> helm install ingress-nginx ingress-nginx/ingress-nginx -n ingress-nginx -\> install the chart named ingress-nginx (first one) , in the namespace called ingress-nginx (last one), To be honest I can't understand it will, can anyone explains it?

Here, you tell to Helm "Install from a chart named 'ingress-nginx', from the 'ingress-nginx/ingress-nginx' (repository-name/chart-name get from ArtifactHub.io, the official Helm charts repository where you can find a lot of ready-to-use charts) repository, inside 'ingress-nginx' namespace".

> and sometimes I see helm repo update after adding the chart-repo with helm repo add (inginx documentation is doing it) and sometimes not , what is the difference? is there any difference?

After the first "helm install", if you want to update your something like a simple value in your values.yaml file, you need to "helm upgrade" it, but it's only if you want to change the default config.
You can get more informations here : https://helm.sh/docs/helm/helm\_upgrade/

Hope it can help :)

---
Tags: kubernetes, helm, ingress, kubernetes-helm
---


# Question
Title: Should we deploy Storage Class Per Application OR one Storage Class for All applications?
I am using aws-ebs-csi-driver on k8s for persistent volumes. I came across a scenario where I can create ebs storage class once and then use it across all the statefulsets deployments to create PersistentVolumeClaims.

I am trying to weigh in advantages and disadvantages of this approach.

Is using one storage class for all statefulsets fine or Should I create storage class for each individual statefulset ?

# Answer
> 0 votes
You should think about the needs of your applications.

Every storageClass have differents purposes, like the type of storage, the max size, ACLs, CSI, replications...

If you know that your storage can be "cold", you don't need to take a premium storage.

Every providers has it's own level of classes, check the doc of AWS to find the best option for your needs :)

---
Tags: kubernetes, amazon-web-services, eks, persistent-volumes
---


# Question
Title: Windows Docker location of file env.conf
I found that on Linux you can configure some settings inside file: `/etc/systemd/system/docker.service.d/env.conf` .

I am wondering where I can find it in windows wsl?- purpose is to set those two fields: `Environment="BUILDKIT_STEP_LOG_MAX_SIZE=1073741824"`

`Environment="BUILDKIT_STEP_LOG_MAX_SPEED=10240000"`

# Answer
It will be in

```
    C:\Users\<UserName>\. wslconfig

```

if you're using WSL2.

More information here:
https://docs.docker.com/desktop/windows/wsl/
> 1 votes

---
Tags: docker, windows, configuration
---


# Question
Title: Jenkins Groovy to parse console output and mark build failure
What would be a Groovy or pipeline step to parse Jenkins console output, identify a string and mark the build as failure if the string is matched?

# Answer
This question seems similar to another on StackOverflow, but seems to be a close, but not exact duplicate of https://stackoverflow.com/questions/37018509/jenkinsfile-build-log

To quote those :

> if you just want to check, that your log contains string myTestString you can just call manager.logContains('.*myTestString.*')
>
> If you want to get some information from the first matching line you can use manager.getLogMatcher(regexp)

You could use the Groovy flow control to break the build - you just need a step to execute the test:

Following the example on the link :

```
node {
    stage('CheckLog') {
      steps {
        if (manager.logContains('.*myTestString.*')) {
          error("Build failed because of this and that..")
        }
      }
    }

```
> 5 votes

# Answer
I solved this problem by following this answer on StackOverflow.

You can use `currentBuild.rawBuild.getLog(10)` (the `10` specifies that you want the last 10 lines) to get the last few lines of the log stream.

According to this documentation, this method returns a List of Strings (`List<String>`) where each index contains a single line of the log stream.

You can either search line-by-line for what you want, or you can combine the entire List into a single String using the `.join()` method.

To search the actual String, you can use the `.contains()` method or you can use a RegExp
> 1 votes

---
Tags: jenkins-pipeline, salt, groovy
---


# Question
Title: What is the best practise way to expose a service over https in kubernetes without exposing it to the internet?
## Coming from Docker

When I was still using Docker for running my services I had an nginx reverse proxy that handled issuing and signing certificates through letsencrypt automatically.

Using this setup, I was able to host a public site **service.example.com** that did nothing but redirect the user to **example.com**. The benefit of this was that the reverse proxy setup would then issue and get a certificate signed that I could then use internally.

Using a local DNS on my internal network I could then access my internal service **service.example.com** using a trusted and signed certificate. All without exposing it to the public.

## Now in Kubernetes

Now I am slowly learning to use Kubernetes and want to achieve the same result. Is there any best practise/good way to accomplish the same in Kubernetes?

I am currently running an nginx ingress controller paired with cert manager which is working perfectly for all my exposed services.

# Answer
Have you try to do it by Port Forwarding your configured service with TLS to your local machine ?
I've never tried but it seams to be possible with it.

Maybe you can find something here : https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/
> 1 votes

---
Tags: kubernetes, cert-manager
---


# Question
Title: How to track deployments of multiple interconnected repositories
We are working with a ERP system (called Odoo, but that does not matter so much). It consists of many building blocks. Some of this modules/addons are dependent on others, some are standalone, but overall they all makeup the whole system.

Problem is that we would like to track changes not only to code itself but also where it is deployed.

We have many different servers where different instances of this ERP are working. With different set of modules and different versions. At the same time.

Currently we store each module in a separate repository. We also have repositories where we store specific deployments. We are linking modules inside deployment repositories using git submodules.

How would you go about tracking where each module and in what version is deployed? One could just create excel file where all the deployments would be listed. But it is not the way.

I'm not even asking about proper CI\CD automation as I have no idea how to start. I just picked this topic as a starting point.

I've seen *Deployments* tab inside Gitlab but it is working only inside single repository. We would need something like this for the whole group. Assuming we could setup a pipeline that would actually deploy the code.

# Answer
for a short or one liner you need gitops, and argocd is a proper tool to do it

See we were using same with #axelor, we need to maintain the version of each module and environment separately so we need to git these rather the excel file methodology.

Argocd is platform and definition from their website

> Argo CD is a declarative, GitOps continuous delivery tool for Kubernetes.

How you will do it: You have to link all your repos inside the argocd as one application, and than create your deployment pipelines according to env, It have nice UI and can be done via command line too.

If you need further details please don't hesitate to comment here
> 2 votes

# Answer
While I also vouch for Argo CD and GitOps, it is important to mention that Argo CD alone is not enough to track deployments and versioning, especially across multiple environments.

We are building Reliza Hub to bridge that gap - see my write-up here about a sample project and how it allows you track different environments and drifts - https://itnext.io/building-kubernetes-cicd-pipeline-with-github-actions-argocd-and-reliza-hub-e7120b9be870
> 0 votes

---
Tags: git, gitlab, deployment
---


# Question
Title: docker ... invalid mount config for type "bind": bind source path does not exist: but the source path does exist
I have a data set stashed in `/vms1/thoth/new_maplab_shared`, but when I

```
$ docker run -it --mount type=bind,source=/vms1/thoth/new_maplab_shared,target=/new_maplab_shared edd5afa10b5f
docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /vms1/thoth/new_maplab_shared.
See 'docker run --help'.
$ ls -l /vms1/thoth/new_maplab_shared
total 4148912
-rw-rw-r-- 1 thoth thoth 2673031511 Mar 10  2021 MH_01_easy_ordered.bag
-rw-rw-r-- 1 thoth thoth 1571292346 Mar 10  2021 MH_01_easy.zip

```

The directory clearly exists. If I

```
# mount -o bind /vms1/thoth/new_maplab_shared /home/new_maplab_shared
$ docker run -it --mount type=bind,source=/home/new_maplab_shared,target=/new_maplab_shared edd5afa10b5f
root@5d2bbc6202b1:/# ls /new_maplab_shared/
MH_01_easy.zip  MH_01_easy_ordered.bag

```

it works.

What is going on here, and how can I eliminate the bind mount kludge on the host?

# Answer
Snaps are a layer of isolation, similar to containers, that prevent access to the entire host filesystem. User home directories are mounted into the snaps which gives you access to those folders. If you were to install docker outside of a snap, you should have access to mount other host directories into the container.
> 2 votes

# Answer
I went searching for a way to give my `docker` snap access to the directory, but found a couple of other stackexchange questions that lead me to believe it is not currently possible:
> 0 votes

---
Tags: docker
---


# Question
Title: Access docker container by host name instead of localhost:port
Windows 10, docker for windows, simple webapp using nginx and ssl.

Simple thing when using VMs. I have three virtual machines. Every machine has static ip assigned by router. In hosts files I can map ips to host names so I'm able to comunicate between VMs using names: api1.mydev, api2.mydev, application.mydev. Of course all systems have access to other part of network parent. This simple thing took me few days in docker without any progress.

It was super simple to build docker images/containers and they work on custom ports, so localhost:443 works like a harm. I can't find way to:

* use host name to connect to naginx@docker\_container as container's ip changes with every sytem reboot.
* assign host names to many containers as its impossible to define port in hosts file

I've tried nginx-proxy but it gives me 502 errors. I've found traefik but I'm not network guru and configuration looks overhelming for newbie.

The option i thought about is to put all services on one container and use nginx to resolve names but this method gives not true advantages over single VM machine and is against docker philosophy.

Is there is simpler way to achieve that? I have feeling that I've missed something in docker configuration.

# Answer
I've been using traefik for this goal.

```
version: '3.7'

services:

  traefik:
      image: traefik:latest
      container_name: traefik
      command: --api --docker --docker.domain=${DOMAIN:localhost} --logLevel=DEBUG
      ports:
          - "80:80"
          - "8080:8080"
      volumes:
          - /var/run/docker.sock:/var/run/docker.sock
      restart: unless-stopped
      labels:
          - "traefik.enable=true"
          - "traefik.backend=traefik"
          - "traefik.frontend.rule=Host:traefik.${DOMAIN:localhost}"
          - "traefik.port=8080"

  app-account:
    container_name: app-account
    depends_on:
      - traefik
    labels:
      - "traefik.backend=account"
      - "traefik.frontend.rule=Host:account.${DOMAIN:localhost}"
      - "traefik.frontend.headers.customRequestHeaders=Access-Control-Allow-Origin: *"

```

In here `${DOMAIN}` is used only for overriding domain from shell if you'll use this compose file in production, let's say in Docker Swarm. If you'll use it locally only, then just replace ${DOMAIN:localhost} with `localhost`. Launching docker with this docker-compose file you can access this particular `app-account` app via `account.localhost`
> 3 votes

---
Tags: docker
---


# Question
Title: What is the idiomatic way to reload EKS/Kube configs when a new image is pushed to ECR
We are using CircleCI to automate the building of images when merges are detected on our release branch. Those images are then being pushed to ECR, but I am unsure of the best way to refresh some or all of the pods running the outdated images, and additionally how to do it in a way that does not disrupt service, ie: rolling restarts etc.

We are looking into using Flux for this, but I'm not sure if that is the best approach.

I was hoping to get some insight from people running this or similar pipelines in production.

# Answer
The 'best approach' in a theory (coming from the perspective of k8s) would be GitOps: Cloud-native Continuous Deployment. Your question lack some important information, how are you doing your deployments right now ? Do you have some limitations in terms of security and auditability ?

Sharing my personal 2 cents, using GitOps with ArgoCD outsource and guarantees the continious-running of all of your workloads(pods as an example). Argo will take care for them, even if somebody deletes a pod, it will re-store the previous number of pods, essentially maintaining the desired vs real state. Another benefit is that you could really easily track your appps by version, commit, tag ... you could even roll-back(this works only under a specific circumstance). Going in an example, by simply changing the image inside a pod spec in a directory that Argo keeps track of it is enough for the new version to be roll out as safe as possibble.

In our case we have a separate service (it's just a repo with tags) that acts as a wrapper for all the services, once we have a new release we deploy under a pattern of a directory layout and Argo catches it, everything is re/deployed automatically. The UI is great and they even expose a REST API.

Now once you have Argo(or some other tool), you get additional efforts in tracking the releases of the tool, versions, backward-comptability, helm metadata support, security, access, password rotations, best way to drop Argo in the cluster ... but this is another story that comes naturally, once you decide to go this path.
> 2 votes

# Answer
We are using https://github.com/stakater/Reloader The work of the reloader is that when someone update the configmap/secrets then it will keep watch the deployment and it will automatically restart the deployments. This is the best way we have find out till now
> 0 votes

# Answer
Still the most proper way is to update tags of your docker image and update `yourapp.yaml` script , there's no other way and then just `kubectl -f yourapp.yaml`. For this purpose GitOps is used. For me who has small cluster using something as ArgoCD is too much. I've went with Gitlab free service called AutoDevops which deploys read and tested docker container straight to your EKS.
> 0 votes

---
Tags: kubernetes, continuous-deployment, continuous-delivery, eks, aws-ecr
---


# Question
Title: Best practice to update (migrate) e.g. users on database server on several clusters
we have multiple clusters - like 6 - and of course, as a dev makes changes to an application and it is deployed it will migrate the table data and so on by flyway/liquibase.

But what if there is the need for a new user?

Where does the dev "show up" to define the user needed for a specific task in the database?

Or say we need a new wholedatabase or so.

And will this migration work. Let's say we have a single microservice which did not need the user `User` in version 1.7.0. But now, in 1.8.0 it will need the user. So before we deploy 1.8.0 on a certain cluster, we will want to add the user.

How is it done in an Continuous Deployment environemtn?

# Answer
You will add flyway migration as it is, nothing special and than this migration will be applied before the application is opened for service.

There is no specific difference between versions as far as flyway migrations concerns.

If they are master slave than migration can be applied on master instances only.
> 0 votes

---
Tags: build-pipeline, continuous-delivery, microservices
---


# Question
Title: How to structure/combine the installation of helm + deployment files
I have a Wordpress Helm chart, and a seperate ingress definition to go along with it, which I specified in a wordpress-deployment.yaml file.

My workflow for installing would be:

* Install the helm chart using a wordpress-values.yaml file for values
* Apply the wordpress-deployment.yaml deployment file using kubectl

I don't like that these steps are separated (e.g. what if I forget about installing the Helm chart?)

I would like to use K8S in a professional setting. What would be the "best practice" way of going about this kind of file structure? Is there a more structured / less manual way of handling this? Ideally I would like to run 1 command which handles both installing helm charts & the accompanying deployment files & configurations.

Thanks in advance!

# Answer
Helm supports sub charts.

You could make your ingress yaml file into its own helm chart.

You could then make a base chart then put both of your charts in it as subcharts. (Or have one of your charts as the "Main chart" and another as a subchart.)
> 1 votes

# Answer
I would suggest try out Terraform with the helm provider. I work with this every day and all of our helm charts have custom values files for deployment. I guess this can be considered a "best practice" depending on the company. So how the flow goes:

1. We have a custom app all in K8S.
2. We set up a helm\_release with terraform.
3. We specify the custom values file =\> `file("${path.module}/kubewatch-values.yaml")`
4. In the terraform you can specify the chart, its version, and the name(or other settings). You also point it to the values file as above(point 3).
5. You then apply the terraform which deploys the helm chart with your custom values file. And that should be all, its a clean and simple solution and you can have that stored in Git as well.

Example =\> Terraform Examples
> 0 votes

---
Tags: docker, kubernetes, deployment, helm, kubernetes-helm
---


# Question
Title: Kubernetes pod eviction and traffic draining for to be evicted pod
I'm trying understand to Kuberentes pod eviction behaviour and when it decides to terminate a pod how it manages that no request should be in process state when it is going to terminate a pod.

It is based on a recent troubleshooting of incident where 3 requests resulted in 502 bad gateway on the ingress controller Kong.

During the analysis it was found out that at the exact same time K8s terminated a pod because it was hitting ephemeral storage limits set on one of the container of the pod. In addition to this on linkerd container logged that connection was reset by peer and proxy was failed to process the request. This connection reset translated to 502 bad gateway for ingress controller kong.

I understand by improving the limits and controlling the number bytes container writes on disk I can avoid eviction by kubernetes to a great extent.

I'm Interested before terminating a pod does kuberentes makes sure that that no traffic is load balanced on to be terminated pod. Also what are the best practices to fine tune Kubernetes pod eviction.

# Answer
setup a readiness Probe in the pod spec, something like this

```
readinessProbe:
  httpGet:
    path: /
    port: 443

```

readiness probes specifically inform service routing decisions
> 1 votes

# Answer
The pod eviction can happen because of resources outages like CPU, ram, and storage. You can prevent your cluster from the eviction of pod by declaring the `resource quotas`. You can define resource quota over namespace but before that, you have to calculate workload usage and create a resource limit on provided request quota. You have three types of quotas available.

1. Compute Quotas
2. Storage Quotas
3. Object Count Quotas

There are a few more ways to quotas on the kubernetes cluster.

https://kubernetes.io/docs/concepts/policy/resource-quotas/
> 1 votes

# Answer
To answer your two questions:

> I'm Interested before terminating a pod does kuberentes makes sure that that no traffic is load balanced on to be terminated pod.

If you have more than 1 pod running then yes, it will not send traffic to the failed/terminating pod.

> Also what are the best practices to fine tune Kubernetes pod eviction.

The official Kubernetes documentation explains the Pod lifetime cycle in great detail along with suggestions on how to fine tune. While liveness probes and readiness probes can help determine the health of a pod, they will not help in the case of eviction.

Eviction occurs due to a lack of resources (your case), or if the Node is unavailable (e.g., spot instance eviction, Node maintenance). By using something like a ReplicaSet or Deployment you can specify how many pods you want to have running and can fine tune where these run. For instance a DaemonSet will by default deploy to every node. Architecting this way will ensure that if a node goes down, or a pod fails to run due to lack of resources, it will have other nodes available or will be rescheduled to launch on another node.
> 1 votes

---
Tags: kubernetes
---


# Question
Title: How to quickly test if postgres database is accessible?
I whished there were a tool like ping for databases? I just want to ping the database and see if it is accessible from a remote server from Linux command line (bash).

I read about telnet, but I am not sure how it works.

Any suggestions?

# Answer
> 7 votes
Something like:

```
pg_isready --host=192.168.79.8 --username=test --port=5432

```

> 192.168.79.8:5432 - no response

Another solutions is telnet:

```
telnet 192.168.79.8 5432

```

Which returns:

> Trying 192.168.79.8... Connected to 192.168.79.8. Escape character is '^\]'.

If the database is accessible.

---
Tags: postgresql
---


# Question
Title: Jenkins Pipeline - input only if branch matches
I want to ask for input but only if we are on a certain branch. But the input is run before the when. Can this be done with declarative syntax?

For example this will always prompt even when the branch is not master

```
stage('only on master') {
   when {
      branch 'master'
   }
   input { ... }
}

```

# Answer
I see two approaches :

## Save current branch name on a variable and check it in a if statement

```
CURRENT_BRANCH = '...' // some value or parameter

stage('only on master') {
  if (CURRENT_BRANCH == 'master') {
  input {
    ...
  }
}

// be sure to check if input was set or define a default value

```

or

## Run a shell script to know if current folder is a git repo on master

The following command check if current folder is a git repository printing "true" ( or returning a error if does not have repo)

```
git rev-parse --is-inside-work-tree

```

The next, print the current branch name

```
git branch | grep \* | cut -d ' ' -f2

```

Use both to create a script that sets an environment variable. Check the value on your pipeline, to show the input

```
stage('only on master') {

  // Set true on IS_ON_MASTER if current branch is master,
  // otherwise set false

  sh ./check-branch-master.sh

  if (env.IS_ON_MASTER) {
  input {
    ...
  }
}

```
> 4 votes

# Answer
Or you could use the `beforeInput` flag

```
stage('Example Deploy') {
        when {
            beforeInput true
            branch 'production'
        }
        input {
            message "Deploy to production?"
            id "simple-input"
        }
        steps {
            echo 'Deploying'
        }
    }

```

Jenkins pipeline | When
> 2 votes

---
Tags: jenkins-pipeline
---


# Question
Title: What parts of Kubernetes can have IP addresses?
Our network team says there an IP address in a subnet is being claimed and released seemingly at random. They are wondering if Kubernetes could be doing it.

I know that Pods, Services and Nodes can have IP addresses in Kubernetes. I have checked all of those and none of them are doing it.

Is there anything else that can have an IP address in Kubernetes?

I am running an on premises Kuberentes cluster using Istio with node port services. Is there anything else that could be attempting to get an IP address?

# Answer
> 1 votes
The IP addresses that are in your kubernetes cluster are internal to the cluster. There is nothing that would get an IP Address outside of your k8s cluster, like a dhcp from your router/dhcp server. K8s creates virtual adapters depending on the CNI runtime that you have implemented and it also creates domains for the objects inside. Anything inside k8s goes out through forwarding via the NodePort service type, so unless your network team specify which IP it is and which subnet they are tracking, I would suggest that you send them a large tcpdump to go through.

---
Tags: kubernetes, networking
---


# Question
Title: How to determine if a Jenkins build is triggered by a Pull Request merged to Master
I think i've scoured every post on this topic but haven't found my specific scenario. (Maybe that means I'm building the wrong thing OR i've built something unique and new? The former is far more likely)

I have a multibranch pipeline in Jenkins being triggered by webhooks from GitHub. I want to automatically tag the repo being built only when a PR from `develop|release|hotfix` branches are closed and merged to `master`.

What I want to drill in on is how to identify when a `master` branch build is triggered by the merge from a PR, and not from anything else. I *don't* want to tag if `master` builds due to a non-PR merge. (This would allow updating readmes and code comments without generating a new tag.)

I'm getting the tag string from a file in the repo, and I know how to use the PR builder to identify the source and target of a pull request for `when` conditions. I know how to get the JSON build information and pull data out, but what I want doesn't seem to be there.

`when { changeRequest target: 'master' }` only identifies the PR build, not the `master` build that results when the PR is closed and merged.

`when { branch 'master' }` obviously identifies a `master` branch build, but how can I limit it to only build when the build is triggered the merge from a closed PR?

# Answer
> 3 votes
You might try installing the GitHub Integration plugin. Reading their documentation, they provide a number of environment variables you can use for your purpose.

1. `GITHUB_PR_STATE` can be `OPEN`, `CLOSE`
2. `GITHUB_PR_SOURCE_BRANCH` for the source branch (e.g., `hotfix/foo`)
3. `GITHUB_PR_TARGET_BRANCH` or `master`

Using the when condition, you can run a job/stage when it's a closed PR merging into the `master` branch using:

```
when {
  allOf {
    expression { env.GITHUB_PR_STATE == "CLOSE" }
    expression { env.GITHUB_PR_TARGET_BRANCH == "master" }
    expression { env.GITHUB_PR_SOURCE_BRANCH == "hotfix/foo" }
  }
}

```

# Answer
> 2 votes
I've solved this!

I can't find the stackoverflow question that helped me, but they gave me the idea.

`git log --format=format:%s -1`

Specifically, I have a bash file that does the following as part of my jenkins pipeline.

`echo "COMMIT MESSAGE" `git log --format=format:%s -1`;`

In my PR build, it prints the following:

`2021-10-07 12:56:43 COMMIT MESSAGE prints`

In the master build, it prints the following:

`2021-10-07 12:58:11 COMMIT MESSAGE more print statements (#1948)`

And then you can use some bash kung fu to do what you need to do from there!

I noticed this when manually scanning the JSON payload that GitHub sends to Jenkins and noticed that GitHub was adding the PR # at the tail end of the git commit message (but only for master for some reason).

# Answer
> 1 votes
You are not building the wrong thing this is a topic that is coming up more and more frequently as people create more and more advanced pipelines. You can see people discussing the need for it here.

The reason is doesn't come up a ton is most people can just trigger their tag job on any change to master. Which is much simpler to setup. I often want to tear down something like a environment when a new feature is merged.

Pretty much your best bet is too follow this guide. Basically you use the generic-webhook-trigger plugin. It allows you to extract parts of a JSON payload into environment variables injected into the job. Once you have done that you just follow Argyle's answer.

# Answer
> 0 votes
Check if the GIT\_COMMIT is a merge commit using `git rev-parse`:

```
when {
  branch 'master'
  expression {
    isMergeCommit(env.GIT_COMMIT)
  }
}

def isMergeCommit(String commit) {
  isSimpleCommit = sh(returnStdout: true,
      script: "set +e; git rev-parse --verify -q $commit^2 > /dev/null; echo \$?").trim()
  return isSimpleCommit == "0"
}

```

`set +e` is needed because the script returns a non-zero exit code if the commit is a simple (non-merge) commit.

# Answer
> 0 votes
Wow, this has garnered a ton of views. For what it's worth, I've ended up building a much simpler solution since I posted this.

Ultimately, we've stopped using PRs for much of anything pipeline-related other than build success. It's just too complicated to get data out of them reliably, and there's too much hand-waving and magic happening for the average developer who's not inside this stuff all day to really *get* it. In fact, I am able to identify PR branches at a basic level by checking for `env.CHANGE_ID != null`, and just skip wholesale or hardcode a lot of stuff that's not important to us for PRs, like versioning and archiving. I can always come back to it later if we decide we want to actually use the PR builds for anything.

What we have currently is a process implementing basic Semantic Versioning, where we only tag on `develop` builds, `develop` being our de-facto mainline. We don't do a great job of merging to `master` after releases, but we are pretty solid on always only deploying from `develop`. (Why keep `main` around, then? We ask ourselves that often. It comes down to priorities of process changes; it's not really hurting anything to have it, so we focus on other stuff.)

I recently integrated gitversion into the pipe, and have it configured to automatically bump the `PATCH` version on any commit-build to `develop`. If a bump of `MINOR` or `MAJOR` is required, the commit-message phrases can be used.

Gitversion is nice because it removes the need for a file in the repo to be updated manually; it's based entirely on `git tags` and commits. It even updates the `assembly*.cs` etc files in the repo automatically. The downside is it took me quite a while to feel like I understood it well enough to use it; its configurability makes it complicated to grok at first. There are other tools out there that do the same thing, this is the one that was suggested to me and the docs made me comfortable enough to jump into it.

---
Tags: jenkins, github
---


# Question
Title: Deployment Error: getaddrinfo ENOTFOUND <appServiceName>.scm.<appServiceEnvironment>.appserviceenvironment.net
## Issue

We're getting the error:

> ##\[error\]Error: Error: Failed to deploy web package to App Service. Error: getaddrinfo ENOTFOUND {appServiceName}.scm.{appServiceEnvironment}.appserviceenvironment.net {appServiceName}.scm.{appServiceEnvironment}.appserviceenvironment.net:443

When running the below pipeline task:

```
- task: AzureRmWebAppDeployment@4
  enabled: true
  displayName: 'Azure App Service Deploy Zip-file to: $(arm_appServiceFrontend)'
  inputs:
    appType: 'webApp'
    azureSubscription: '$(armServiceConnectorForTestEnvironment)'
    WebAppName: '$(arm_appServiceFrontend)'
    # the deployment step has a hidden step that downloads the published artifacts but it places the
    # artifact in the wrong catalogue. To work around that I have to place my self in the correct
    # path below.
    packageForLinux: './../$(publishedArtifactName)/$(frontEndProjectName).zip'

```

The full output of this task is:

> Azure App Service Deploy Zip-file to: {appServiceName}
>
> View raw log
>
> Starting: Azure App Service Deploy Zip-file to: {appServiceName}
>
> ==============================================================================
>
> Task : Azure App Service deploy
>
> Description : Deploy to Azure App Service a web, mobile, or API app using Docker, Java, .NET, .NET Core, Node.js, PHP, Python, or Ruby
>
> Version : 4.184.4
>
> Author : Microsoft Corporation
>
> Help : https://aka.ms/azureappservicetroubleshooting
>
> ==============================================================================
>
> Got service connection details for Azure App Service:{appServiceName}
>
> Trying to update App Service Application settings. Data: {"WEBSITE\_RUN\_FROM\_PACKAGE":"1"}
>
> App Service Application settings are already present.
>
> Package deployment using ZIP Deploy initiated.
>
> ##\[error\]Failed to deploy web package to App Service.
>
> ##\[error\]Error: Error: Failed to deploy web package to App Service. Error: getaddrinfo ENOTFOUND {appServiceName}.scm.{appServiceEnvironment}.appserviceenvironment.net {appServiceName}.scm.{appServiceEnvironment}.appserviceenvironment.net:443
>
> App Service Application URL: http://{appServiceName}.appserviceenvironment.net
>
> Finishing: Azure App Service Deploy Zip-file to: {appServiceName}

Note: content in curly braces is where I've replaced deployment specific strings with generic names.

## Context

One of our teams has started using Azure DevOps pipelines to deploy code in a GitHub repo to an App Service in an App Service Environment.

We're using the App Service Environment because the app requires access to resources in our private network (internal API and LDAP servers).

The app itself is exposed publicly, though this is done via an Application Gateway, with the app service configured as the backend pool, rather than the app service being public facing.

Since we're deploying to a private ASE, we've created our own Azure Scale Set Release Agent, using an image created from the MS Ubuntu 2004 agent.

A lot of components in this tech stack are new to all members of the team; so it's very likely we've done something wrong somewhere; but our lack of experience with these tools means we're not sure where...

## Question

Does anyone know of a likely cause for the error we're seeing?

Failing that, any tips on useful steps to help us track the source of these issues would be appreciated.

# Answer
> 2 votes
The official solution is documented here: MS Docs: Create ILB ASE - DNS Configuration

You need to have DNS entries for your ASE so that the deployment task can reach it; but if you have an internal only ASE, those entries aren't created in public DNS, and MS don't manage private DNS.

MS propose creating the DNS entries in your internal DNS:

> Create Zone {ASE Name}.appserviceenvironment.net
>
> * Type: Primary Zone
> * Store the zone in AD: True
> * Zone Replication: To all DNS servers in the domain
> * Zone Name: {ASE Name}.appserviceenvironment.net
> * Allow only secure updates: true

> Create A Record under {ASE Name}.appserviceenvironment.net
>
> * Name: *
> * IP: {ASE's Private IP}

> Create A Record under {ASE Name}.appserviceenvironment.net
>
> * Name: *
> * IP: {ASE's Private IP}

> Create Child Zone under {ASE Name}.appserviceenvironment.net
>
> * Name: scm

> Create A Record under scm.{ASE Name}.appserviceenvironment.net
>
> * Name: *
> * IP: {ASE's Private IP}

However, whilst awaiting the DNS creation I tried a workaround which turned out to be sufficient for the pipeline's needs...

Add the below script to your pipeline, ahead of the deployment task:

```
- script: |
    sudo -- sh -c -e "echo '{ASE's Private IP}    {ASE Name}.appserviceenvironment.net' >> /etc/hosts";
    sudo -- sh -c -e "echo '{ASE's Private IP}    scm.{ASE Name}.appserviceenvironment.net' >> /etc/hosts";
    sudo -- sh -c -e "echo '{ASE's Private IP}    {AppName}.scm.{ASE Name}.appserviceenvironment.net' >> /etc/hosts";
  displayName: 'Fake DNS entries required for deployment'

```

HOWEVER; the above fix only fixes the pipeline's deployment.

The FQDN for the app service is still not present; and the IP alone won't work since all apps on the ASE will have the same IP / we need hostname resolution. In our case we had an AppGateway in front of the app service; so you could pick the app service you're after from the list... but that's resolved to an FQDN, which then fails as the AppGateway can't resolve that FQDN to an IP... So you do need your DNS to be updated.

Whilst testing the service / awaiting DNS changes, our developers just updated their local device's hosts files to work around that issue: `{ASE's IP} {appServiceName}.{ASE Name}.appserviceenvironment.net`

# Answer
> 1 votes
Select the Default agent pool in the Releases Run on agent - Agent pool field. It appears this gets set to another value when creating a new release. Once Agent pool was set to Default, the error went away.

---
Tags: azure-devops, azure, azure-pipelines
---


# Question
Title: ansible Unsupported parameters for (command) module:
I'm trying to run a shell command in an ansible task but I keep getting an error.

This is the task item and the shell command I want to run:

```
- name: Set File Watch Limit
  ansible.builtin.shell:
    cmd: "echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p"

```

But this is the error that ansible gives me:

```
fatal: [...]: FAILED! => {"changed": false, "msg": "Unsupported parameters for (command) module: cmd Supported parameters include: _raw_params, _uses_shell, argv, chdir, creates, executable, removes, stdin, stdin_add_newline, strip_empty_ends, warn"}

```

The reason I want to run the command is because it will address a problem that reactjs/nodejs developers are facing when they use `npm run watch` as explained in this related question: https://stackoverflow.com/questions/55763428/react-native-error-enospc-system-limit-for-number-of-file-watchers-reached

How do I get ansible to run the command? THe command works perfectly fine if I login to the server and manually execute it.

# Answer
Got it to work with this

```
- name: Set File Watch Limit
  shell: "echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p"

```
> 1 votes

---
Tags: ansible, bash
---