title: How to read Kubernets secrets values

text:
# Question
Title: How to read Kubernets secrets values
What could be the `kubectl` command to see `k8s` `secret` values

I tried

```
kubectl get secrets/<secrets-name> -n <namespace>

```

It returns

```
NAME            TYPE     DATA   AGE
secrets1        Opaque   1      18h

```

but I want to know what value stored inside the secret

# Answer
> 2 votes
So the answer was so simple I just have to add  `-o jsonpath='{.data}'` at the end of the command

```
kubectl get secrets/<secrets-name> -n <namespace>  -o jsonpath='{.data}'

```

---
Tags: kubernetes, kubectl
---


# Question
Title: kops validate cluster fails with even after providing appropriate state and clustername
I am trying to run validate cluster command as per the documentation kops validate

Strangely it fails with following error

```
$ AWS_ACCESS_KEY_ID=<my_legit_key> AWS_SECRET_ACCESS_KEY=<my_legit_secret_key> kops validate cluster --wait 10m --state="s3://<my_bucket_name>" --name=<my_cluster_name>
Validating cluster <my_cluster_name>

Validation failed: cannot load kubecfg settings for "<my_cluster_name>": context "<my_cluster_name>" does not exist

```

# Answer
> 1 votes
I loaded the `kubeconfig` before and the error disappeared.

```
$ export KUBECONFIG=~/workspace/myrepo/_kube/dev/kubeconfig
$ AWS_ACCESS_KEY_ID=<my_legit_key> AWS_SECRET_ACCESS_KEY=<my_legit_secret_key> kops validate cluster --wait 10m --state="s3://<my_bucket_name>" --name=<my_cluster_name>
## And it worked

```

If the kubeconfig does not exist yet, then we can export it as well.

```
$ kops export kubecfg --admin=87600h0m0s --kubeconfig ~/workspace/myrepo/_kube/dev/kubeconfig --state=s3://my-kops-dev-v4 --name=k8.mydomain.com

```

# Answer
> 0 votes
Try running the `kops get` command to see if kOps can read the configuration stored in the s3 bucket correctly. If you get an error then you probably have missed a required configuration in your environment. I had a similar issue but I was using Named profiles for AWS. I had to set an extra variable `export AWS_SDK_LOAD_CONFIG=1` for kOps to pick up the settings (I think it was AWS\_REGION) correctly. If it's just `kops validate` that is failing then make sure you are using the correct context.

---
Tags: kubernetes, amazon-web-services, kops
---


# Question
Title: Bitbucket: Is there any option to restrict deletion of release branch only after it is merged to master
We are using GitFlow workflow. After merging the release/\* branch to master, we need to close the branch. For this task, we need to provide deletion access to pull request merger but in this case, merger can delete the release/\* branch any time.

Do we have any option to restrict the branch closer only after it is merged to some other branch ?

# Answer
You should not delete the branches at all. Just rename them to stale!

but

As of Bitbucket 7.3 there is an option to delete branches automatically after merge. Just prevent deletion at all, set the other option and you are done.
> 1 votes

---
Tags: git, bitbucket
---


# Question
Title: How to configure single nginx ingress with letsencrypt and multiple bitnami wordpress containers?
I'm using bitnami/wordpress helm chart to create many websites which should run on different domains. I'd like to auto secure them and use a single ingress to route traffic. I'm using this tutorial https://docs.bitnami.com/tutorials/secure-wordpress-kubernetes-managed-database-ssl-upgrades/ to create ingress nginx controller + letsencrypt.

Routing works fine, but the ssl works only for the first created wordpress container/domain and those after it are not secured. I can't figure out why? k8s v1.19.10-gke.1600 helm 3.6.0

Let's say i have sub1.domain.com and sub2.domain.com and i have two sub1.domain.com.values.yaml and sub2.domain.com.values.yamlfiles to customize bitnami/wordpress helm chart

Config is as follows: sub1.domain.com.values.yaml:

```
ingress:
  enabled: true
  certManager: true
  hostname: sub1.domain.com
  extraTls:
  - hosts:
      - sub1.domain.com
    secretName: wordpress.local-tls
  annotations:
      kubernetes.io/ingress.class: nginx
      cert-manager.io/cluster-issuer: letsencrypt-prod

```

sub2.domain.com.values.yaml:

```
ingress:
  enabled: true
  certManager: true
  hostname: sub2.domain.com
  extraTls:
  - hosts:
      - sub2.domain.com
    secretName: wordpress.local-tls
  annotations:
      kubernetes.io/ingress.class: nginx
      cert-manager.io/cluster-issuer: letsencrypt-prod

```

I apply them with:

```
helm install sub1 bitnami/wordpress -f sub1.domain.com.values.yaml
helm install sub2 bitnami/wordpress -f sub2.domain.com.values.yaml

```

Buth are accessible via browser, but only sub1.domain.com is with valid ssl cert, sub2 is with kubernetes fake cert.

What am i doing wrong?

# Answer
Same TLS secret will not be mapped to two ingress at the same time within a namespace.It will map to the first deployed ingress. Create the same secret with a different name and map it to the other ingress for it to work.
> 1 votes

---
Tags: docker, kubernetes, nginx, ingress, wordpress
---


# Question
Title: helm upgrade fail with error: UPGRADE FAILED: "chart-repository" has no deployed releases
I am trying to upgrade a chart but when I run upgrade command, upgrade fails with the following error.

```
nit:~/copy> helm upgrade chart-repository chart-repository/
Error: UPGRADE FAILED: "chart-repository" has no deployed releases

```

but when I do `helm list -A` , it looks to me it is deployed.

```
nit:~/copy> helm ls -A
NAME                NAMESPACE   REVISION    UPDATED                                 STATUS      CHART                   APP VERSION
chart-repository    charts      1           2021-06-08 14:43:05.186875759 +0000 UTC deployed    chart-repository-0.0.16 0.0.14

```

So, why is the upgrade failing? i am using helm 3.6.0.

Thanks

# Answer
You're missing namespace in the upgrade command. Following should work:

```
helm upgrade chart-repository chart-repository/ -n charts

```
> 1 votes

---
Tags: kubernetes, helm
---


# Question
Title: Run docker with bind-mount pointing to non-existent host folder: Where is the data?
I run a docker image with a bind-mount pointing to a non-existing host directory. The docker image then created data into that volume, wherever it may be. The data is accessible on every docker run.

If I then create that missing host directory and re-run the container, the new host folder is used, creating new data. When I remove it again, the old data shows up again.

I have done `docker volume prune` and have nothing coming up with `docker volume ls`.

When I remove the host directory again, then also remove the container and the image, then re-run the docker container, I can see the image getting re-downloaded, but then the old data is still there.

How do I identify the location of the old data and interact with it?

(This is on MacOS)

# Answer
> 1 votes
I have been using Linux and since Docker volume caches the data, it is merely persistent until you remove it manually. In RHEL8, /var/lib/docker/volumes/, this is where all the volume mounts are stored. You can check if the same directory exists in MacOS. I haven't use Mac but I hope this helps.

---
Tags: docker, volume, macos
---


# Question
Title: How to mitigate issues in an IIS on-prem application calling an unreliable cloud vendor API?
We have an IIS 10 (Windows Server) on-prem application that is calling a cloud vendor api and for some reason the api is not reliable. Sometimes the application returns results as expected. Sometimes no data is able to be retrieved.

We checked our code and suspect it's something on the vendor side and the vendor said they are looking into it.

What ways can we mitigate the issue on our side to improve the quality of the call and improve the experience for the end users?

# Answer
I'd make the invocation safe, use some try/catch on the invocation itself. That way, if the API is unreachable or HTTP error code is returned - the app can handle it and present the "error" to the user in a more UX-y way ;)

Second thing, I'd make a returned data validator of some sort. For example data types, structure etc. Especially on key fields. That way we know that nothing that comes from the vendor API is breaking our app and make sure that no data returned won't crash the app on our side.
> 1 votes

---
Tags: windows, servers, api
---


# Question
Title: Code Changes for Implementing Kubernetes Probes
I want to implement Kubernetes Liveliness Probes in my .Net core application with C# language. I understood that first I need to define the liveliness probe in deployment.yaml file of my application. I also created the http service in my application that is responding at 8080 port.

```
apiVersion: v1
kind: Pod
metadata:
  labels:
    test: liveness
  name: liveness-http
spec:
  containers:
  - name: liveness
    image: k8s.gcr.io/liveness
    args:
    - /server
    livenessProbe:
      httpGet:
        path: /healthz
        port: 8080
        httpHeaders:
        - name: Custom-Header
          value: Awesome
      initialDelaySeconds: 3
      periodSeconds: 3

```

Apart from this what changes do I need to do in my C# code ? I am using below Kubernetes nuget in .csproj :

```
<PackageReference Include="KubernetesClient" Version="3.0.7" />

```

# Answer
Apart from @rohatgisanat answer which explains the Kubernetes side, Microsoft did a pretty good job on explaining its own HC middleware on the service side: https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/health-checks?view=aspnetcore-5.0

So the only thing for you to do is combine this knowledge :)
> 0 votes

# Answer
Any code greater than or equal to 200 and less than 400 indicates success. Any other code indicates failure.

More info is found here :- https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#:~:text=The%20kubelet%20uses%20liveness%20probes,application%20more%20available%20despite%20bugs.
> 0 votes

---
Tags: kubernetes, dotnet, healthcheck
---


# Question
Title: Helm - UPGRADE FAILED: cannot patch "..." with kind Deployment: Deployment.apps
I was running the older 2.16.0 version of ChartMuseum Helm Chart. I am trying to update it to use newer 3.1.0. When I try to upgrade using `helm upgrade <release> <name> -n <namespace>`, the upgrade fails with the following error:

```
Error: UPGRADE FAILED: cannot patch "..." with kind Deployment: Deployment.apps "..." is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/instance":"chart-rep", "app.kubernetes.io/name":"chartmuseum"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable

```

I am not sure but I believe this is because of helm v3? I was going through `[this][3]` page where I found helm v3 is a prerequisite. The change from 2.16.0 to 3.1.0 requires helm v3.

I also have a pv bound to older version and ideally I want it to bind to the newer one.I am also using `rolling-update` strategy.

What steps do I need to take so that the upgrade works?

# Answer
> 3 votes
This is not a Helm issue. You are probably trying to update the LabelSelector fields which are immutable. This is expected behaviour as per the kubernetes.

---
Tags: kubernetes, helm, kubernetes-helm
---


# Question
Title: How to combine cloud-init with Azure ARM Template
I'm trying to set up a Ubuntu Server on Azure. None of the images do quite what I want, so I was planning on taking a vanilla Ubuntu image and using cloud-init to modify it. I'd like to be able to deploy this to multiple environments using Azure Pipelines and have an ARM Template that is deploying the vanilla instance quite nicely.

The issue I have is how to get the cloud-init script I have into the custom data parameter of the ARM Template. It seems that because ARM Templates have to be JSON the only way to do this is to have the cloud-init script on one very very long line. Either that or host the cloud-init file somewhere and do something hacky like this: https://www.wintellect.com/arm-templates-and-cloud-init/. The ARM Template task I'm using in Azure Pipelines does not seem to have a way to pass a file's contents as a parameter.

I'd like to have both the ARM Template and the cloud-init script as separate files in my private git repository. Currently I'm considering some kind of pre-processing build step to combine the two. Has anyone already done this and have a code example?

Maybe I'm "doing it wrong", what do other people do/what is the best way to do this?

# Answer
I feel like you are providing the answer yourself. Encode the entire cloud-init file as one-line base64 using a pre-processing script. Something like `cat cloud-init.txt | base64 -w0` would work for Linux based build agents, -w0 will encode everything in one line.
> 1 votes

---
Tags: azure-devops, azure, cloud-init
---


# Question
Title: Retrieving project version from csproj in Azure Pipeline .NET Core CLI task
I have the following tasks in my yaml file in Azure DevOps It publishes my console application and zip it in a file

```
- task: DotNetCoreCLI@2
  inputs:
    command: 'publish'
    publishWebProjects: false
    projects: '**/MyApp.csproj'
    arguments: '-r win-x64 -p:PublishSingleFile=True --self-contained true -o $(Build.ArtifactStagingDirectory)'

- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: '$(Build.ArtifactStagingDirectory)'
    ArtifactName: 'drop'
    publishLocation: 'Container'

```

I set the project version directly on the project file in Visual Studio when I make any changes. Now I'm trying to get that version when I am publishing the project in AzureDevOps and customise the file name, lets say for a project version 1.2.1 the archived file output of the publish in azure should be MyApp-v1.2.1.zip Currently it only output file as MyApp.zip

Not sure if even my approach towards versioning is correct, so appreciate any input!

# Answer
I had a similar need and I did it as below. In my case it is archived and published differently but I think you get the main idea.

```
    - task: PowerShell@2
      displayName: 'Get version number'
      inputs:
        targetType: 'inline'
        script: |
         $VERSION=([xml](Get-Content **\MyApp.csproj)).Project.PropertyGroup.Version
         Write-Host("##vso[task.setvariable variable=VERSION]$VERSION")

    ... // heare I archive the built files.

    - task: PublishPipelineArtifact@1
      inputs:
        targetPath: '$(Build.ArtifactStagingDirectory)/MyApp_$(VERSION).zip'
        artifact: 'MyApp_$(VERSION)'
        publishLocation: 'pipeline'

```

In the first powershell task I get the version number from the project file and set it to a variable called `VERSION`. I learned it from this https://stackoverflow.com/a/56039297/275940 answer which has done the same for maven project.

In the second task I used the variable to make correct file path.
> 1 votes

---
Tags: azure-devops, azure-pipelines, yaml, azure-artifacts
---


# Question
Title: Deploy to Ubuntu server with python script and awsebcli fails
Just yesterday everything worked fine on our AWS ElasticBeanstalk continuous deployment, but today suddenly fails with the following errors:

```
$ pip install awsebcli -q --upgrade
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
$ eb deploy production
Traceback (most recent call last):
  File "/usr/local/bin/eb", line 5, in <module>
    from ebcli.core.ebcore import main
  File "/usr/local/lib/python2.7/dist-packages/ebcli/core/ebcore.py", line 19, in <module>
    from ebcli.core import ebglobals, base, hooks
  File "/usr/local/lib/python2.7/dist-packages/ebcli/core/hooks.py", line 20, in <module>
    from ebcli.core import fileoperations
  File "/usr/local/lib/python2.7/dist-packages/ebcli/core/fileoperations.py", line 32, in <module>
    from json import load, JSONDecodeError
ImportError: cannot import name JSONDecodeError
Cleaning up file based variables
00:00
ERROR: Job failed: exit code 1

```

Our initial deploy script assumed python version 2.7 and installation of awsebcli. Here is the snippet from the script:

```
 - apt-get update -y
 - curl https://bootstrap.pypa.io/pip/2.7/get-pip.py -o get-pip.py
 - python get-pip.py
 - pip --version
 - pip install --upgrade pip setuptools
 - pip install awsebcli -q --upgrade
 - eb deploy production

```

But after receiving this error I decided to swap to python3 and install pip3 instead. Here is the updated script and what it throws:

```
    - apt-get update -y
    - apt-get install -y python3-pip
    - update-alternatives --install /usr/bin/python python /usr/bin/python2.7 1
    - update-alternatives --install /usr/bin/python python /usr/bin/python3.5 2
    - update-alternatives --list python
    - python --version
    - pip3 --version
    - pip3 install awsebcli -q --upgrade
    - eb deploy production

```

```
$ update-alternatives --list python
/usr/bin/python2.7
/usr/bin/python3.5
$ python --version
Python 3.5.3
$ pip3 --version
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.5)
$ pip3 install awsebcli -q --upgrade
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-0a4fa4ab/awsebcli/
Cleaning up file based variables
00:01
ERROR: Job failed: exit code 1

```

There is no explicit error message thrown, so I can't understand why exactly it exits with code 1. Can you please suggest anything?

# Answer
Update:

The import of JSONDecoderError from json module is a valid syntax only starting from Python 3.5, while in previous versions it was a subclass of ValueError class. Apparently the latest version of awsebcli 3.20.0 should launch with python \>= 3.5 instead of python \<= 2.7.

---

Apparently nothing suggested worked for me. Though I used the official aws eb cli setup script and followed the instructions with dependencies and verifying paths to executables. Details here.

Final working script would look like this:

```
$ apt-get update -y
$ apt-get install -y build-essential zlib1g-dev libssl-dev libncurses-dev libffi-dev libsqlite3-dev libreadline-dev libbz2-dev
$ git clone https://github.com/aws/aws-elastic-beanstalk-cli-setup.git
$ ./aws-elastic-beanstalk-cli-setup/scripts/bundled_installer
$ echo 'export PATH="/root/.ebcli-virtual-env/executables:$PATH"' >> ~/.bash_profile && source ~/.bash_profile
$ echo 'export PATH=/root/.pyenv/versions/3.7.2/bin:$PATH' >> /root/.bash_profile && source /root/.bash_profile
$ eb deploy production

```
> 1 votes

---
Tags: gitlab, deployment, python, aws-cli, amazon-ebs
---


# Question
Title: How can I get logs in real time from a docker container hosted on Digital Ocean
I have a docker container in which I am running Flask and Scrapy. I noticed that when I run and build my container on my local machine I get logs in real time. However, on an Ubuntu instance hosted on Digital Ocean, there are almost no updates to the logs.

I am using of course the same parameters for build and run.

Here is my config:

* local dev environement: OS Linux Mint, docker version 20.10.6
* cloud environment: OS Ubuntu 20, docker version 19.03.4

What can I do to update the frequency of the logs?

# Answer
It seems that the solution was related to python than to docker itself. I had to use -U parameter when I run flask .

https://stackoverflow.com/questions/32285782/what-do-u-m-parameters-do
> 1 votes

---
Tags: docker, logging
---


# Question
Title: One or multiple git repositories for independent small parts of one larger project?
We are developing an Azure-based analytics platform that consists of several modules. Among other things, Azure functions, scripts for creating the infrastructure and scripts for initialising the database structure including the creation of stored procedures are used.

Currently, the repository structure is a bit messy and I was asked to clean it up and develop a strategy for deploying and updating individual parts of the platform (We currently redeploy the entire platform with every release).

What is the best way to implement this at repository level? For example, do I create a separate repository for each Azure function and the associated pipelines, or do I put them all in a common repository and try to solve this via individual release branches for each Azure function?

Any other best-practice approaches?

# Answer
> 1 votes
This is something I also struggle with. I can tell you from experience it depends on the type of application and the tech stack being used. The short easy answer is that having everything in one repo is usually not the wrong or hard way. Breaking things out often doesn't offer many benefits but again it will all depend on your CICD engine and what kind of application and what tech is being used.

Here recently I had a project that had 3 repos. 1 for a CLI application. 1 for the infrastructure as code being deployed by the CLI and 1 for a library that was used by the CLI application and other tools.

This all seems straight forward and a good choice but the CICD has been really pain full.

Often a new feature will span across 3 repos. Trying to get this to work in CICD is impossible as far as I can tell. I would much prefer to have the infra and CLI together as most new features affect both codebases.

Then I could at least get a list of the changed files in the CICD pipeline and decide if I need to test the infra or not.

The other issue when you split things up like this is trying to version these different parts and understand what versions work with other versions etc.

I guess to sum it up would be if things depend on each other then keep them in the same repo as it really helps the CICD process. Think an application and its underlying infra. Now would I combine a rest api and its client? probably not, in the past I just required the branches be the same name. So in CICD I would check if the other repo has a branch of X and if so use it and if not use master. If things are mostly independent like a library and a application that uses it, then keep it split up.

# Answer
> 0 votes
There's no easy answer to this, get your repos too big and every branch becomes huge over time, make it too small and you end up doing a lot of administration.

Many smaller repos has my preference, but only after creating some support scripts to do the administration for you.

For example, to pull all repos:

```
ls -d **/ | xargs -P12 -I{} git -C {} pull

```

Also git has a feature built in for repos, submodules.

https://git-scm.com/book/nl/v2/Git-Tools-Submodules

They come in handy when a repo references another but does not to do source control on the other. It comes with its own subset of commands. It's worth looking into this

---
Tags: azure-devops, git, azure, azure-pipelines, repository
---


# Question
Title: Does it make sense to create CloudFormation Stack using Terraform?
I am not very familiar with AWS, but I know the basics. So please forgive me for any newbie mistakes in the question.

I am trying to integrate our AWS accounts with a vendor app. The vendor has provided a CloudFormation template to create a CF stack that will create the required IAM role, S3 bucket, SNS topic etc for the integration. I looked at the CF template, broke down the steps in the CF template, and wrote the resources that need to be created in our AWS for our vendor *(after incorporating changes to suit our environment)*. When I gave the broken down steps along with the CF template to our DevOps team, I was expecting them to Terraformize the CF template and create the IAM role, etc through terraform. But I see that they are trying to write a Terraform code to create the CloudFormation Stack. No one else in our organization would log into AWS account's console and create stuff, only the DevOps team does it and they do everything through Terraform.

Does it make sense to use Terraform to create CF stack since they can create the IAM role, S3 bucket etc using Terraform itself?

# Answer
> 1 votes
Initially, I though no it does not make sense to use Terraform to create the infra structure to run Cloud Formation script to do create resources you can do using Terraform.

If you are experience in Terraform and use it for your other infrastructure, it does not make sense to introduce another way of doing it,

The only reason I could see to create the Cloud Formation stack and run their provide script would to be to remove the possibility of the introductions of errors during the conversion to Terraform.

If their Cloud Formation script is complex this could be an issue, but if it is just creating an IAM role and some policies it is probably not much of a concern.

# Answer
> 0 votes
No sense. As Occam's razor principle says - "entities should not be multiplied without necessity". So, two entities for one task is too much.

# Answer
> 0 votes
If the vendor is going to change the Cloudformation template often or if the Terraform resources don't need to interact with the Cloudformation resource then it's fine to deploy the Cloudformation stack with Terraform.

If both of these things are false you might want to migrate the resources to Terraform. If one is false and the other is true you should consider the trade offs.

---
Tags: terraform, aws-cloudformation
---


# Question
Title: How to run a daemon AND another command in a kubernetes pod
I am trying to setup an automated influxdb instance using kubernetes, nothing too exotic I thought, but : I need to run the "influx setup ...." command for the initial setup, and this needs to be ran after the daemon is started (obviously).

I tried multiple syntax inside the deploy yaml, here is one attempt:

```
    spec:
      containers:
        - name: influxdb-test
          image: influxdb:2.0.7
          command:
            - sh
            - "-c"
            - |
              /bin/bash <<'EOF'
              influxd&
              influx setup --org ORG --bucket dummy --username USER --password USER --force
              EOF

```

But this gives me the following error :

```
Error: failed to determine if instance has been configured: Get "http://localhost:8086/api/v2/setup": dial tcp 127.0.0.1:8086: connect: connection refused
See 'influx setup -h' for help

```

Well I do not need a solution for this specific error, but more a solution about how can I have a daemon running (influxd) and run some command after this daemon (influx setup) and then keep the container up.

I first thought that adding the simple "influx setup" command would be enough, but any command given inside a yaml deploy file overwrite the container initial command (influxd in this case) so I tried to reproduce the whole stuff manually : start the daemon, then run the setup then I am done, but no luck.

Of course I am a k8s newbie.

I am looking at "init container" as this sounds close to fixing my issue ....

# Answer
> 2 votes
I finally found some way to achieve this kind of task: Creating a container only for a single and ephemeral job can be achieved through the .... Job API :)

See this doc.

So I now have a deployment for my long living task (influxdb) + a Job definition for my on-time job (influxdb initial setup). Here is an example (I thought that could be useful for someone) :

```
apiVersion: apps/v1
kind: Deployment
metadata:
  name: influxdb
  namespace: influxdb
spec:
  selector:
    matchLabels:
      app: influxdb
  replicas: 1
  template:
    metadata:
      labels:
        app: influxdb

    spec:
      volumes:
        - name: influxdb-data
          persistentVolumeClaim:
            claimName: nfs-pvc-influx

      containers:
        - name: influxdb-daemon
          image: influxdb:2.0.7
          volumeMounts:
            - mountPath: /home/influxdb/
              name: influxdb-data
          env:
            - name: INFLUXD_CONFIG_PATH
              value: /etc/influxdb2/
          ports:
          - containerPort: 8086
            name: pbe
            protocol: TCP

```

And the job :

```
apiVersion: batch/v1
kind: Job
metadata:
  name: influxdb-init
  namespace: influxdb
spec:
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: influxdb-init
          image: influxdb:2.0.7
          command:
            - sh
            - "-c"
            - |
              sleep 15
              influx setup --host http://influxdb-nodeport.influxdb.svc.cluster.local:8086 --org organistation1 --bucket dummy --username user --password user --token token --force && exit
  backoffLimit: 0

```

# Answer
> 1 votes
Since influxd is running background it will have a job id of 1 (check with the command 'jobs') So I think all you need to do is bring back influxd to the foregroud after the setup with 'fg 1'

```
  command:
        - sh
        - "-c"
        - |
          /bin/bash <<'EOF'
          influxd&
          sleep 15
          influx setup --org org1 --bucket dummy --username user --password passwd --force
          fg 1
          EOF

```

# Answer
> 0 votes
I think your current approach is probably the correct one. You are just missing a sleep to allow `influxd` time to actually get running. See Serve Laurijssen anwser.

You also could instead run some kind of process manager to handle this like s6-overlay but I wouldn't really use it for this example but it is handy in some other situations.

I don't think init container would work for you because it runs before the main container and is supposed to be a short lived command.

Container Lifecycle Hooks would probably also work here but its just more complexity then needed. To use `PostStart` hook you would need to curl `http://localhost:8086/api/v2/setup` in a loop until you got a good reply and then run your command.

# Answer
> 0 votes
Another option is to have multiple entrypoints in a Dockerfile. Example with setting up an nginx reverse proxy.

First create the script that does the configuring.

```
envsubst '${NODE_NAME}' < /etc/nginx/conf.d/default.conf.template > /etc/nginx/conf.d/default.conf

exec "$@"

```

Add the script to the container in the Dockerfile.

```
FROM nginx
ADD docker-entrypoint.sh /
ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["nginx", "-g", "daemon off;"]

```

Set script as ENTRYPOINT, which will create the config inside the container. Afterwards nginx is started with desired config.

Maybe it does not work for influx but it's a clean option for a "config and run" container.

---
Tags: kubernetes
---


# Question
Title: Can I bypass host key checking in Ansible?
I'm running an Ansible playbook with a subtle hack to work on an arbitrary port forwarded port (so I can use one machine with no direct access to lots of machines).

I've got a pre-task to change the `ansible_port` variable, so then when I start running my real tasks and roles, I get prompted to accept the hostkeys for localhost on some random port.

Because I am naive and don't care about security, I would like ssh to auto-accept and redirect to `/dev/null` (or another file for logging).

Is this possible?

# Answer
> 37 votes
This is typically done by setting the following value in `ansible.cfg`:

```
[defaults]
host_key_checking = False

```

If you don't want to modify `ansible.cfg` you can set an environment variable like so:

```
export ANSIBLE_HOST_KEY_CHECKING=False

```

Source: http://docs.ansible.com/ansible/intro\_getting\_started.html#host-key-checking

# Answer
> 27 votes
In your `ansible.cfg` file you need to add the following line:

```
ssh_args = -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no

```

You could also add those options in your `~/.ssh/config` on every machine from which you run it something like this:

```
Host *
   StrictHostKeyChecking no
   UserKnownHostsFile=/dev/null

```

# Answer
> 3 votes
There are boolean variables for this that can be set where they're needed (such as in playbooks) so as not to do this globally, which reduces overall security.

```
    var: ansible_host_key_checking
    var: ansible_ssh_host_key_checking
    var: ansible_paramiko_host_key_checking

```

So doing `ansible_ssh_host_key_checking: False` in a playbook should work.

However, I don't believe they'll work until Ansible 2.12. See Disabling host key checking via variable doesn't seem to work. #49254 for details.

For now, I'd recommend doing this:

```
ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook ...

```

...and not setting this in the global Ansible configuration. It's a good idea to be explicit when disabling security features so that you're aware of what you're doing.

---
Tags: ansible
---


# Question
Title: How to configure Docker tooling on Eclipse?
The Eclipse plugin "Docker tooling" seems to expect the installation of Docker Machine, or a network connection.

But, on Windows 10 things seem different from what this plugin expects so I can't browse for the required executables. Also, how could I find out the local network URL? docker info does not reveal this.

# Answer
Under Windows 10, you'll find the Docker Machine (`docker-machine.exe`) under: `C:\Program Files\Docker\Docker\resources\bin`

This is the value you need to put in the Docker Machine field. You don't need to change anything for the VM Driver field as VirtualBox doesn't need to be installed when you use Docker for Windows.

Then you have to edit some setting of the Docker daemon to allow your Eclipse Docker plugin to connect to it:

1. First of all, check if the Docker daemon is running by clicking on the Task View button on the Windows taskbar.
2. Mouse over the Docker icon and the tooltip should say: Docker is running.
3. If it's running, right-click the Docker icon to open the Docker contextual menu and go to Settings.
4. In the General Tab, which you will land on by default, Make sure the "Expose daemon on tcp://localhost:2375 without TLS" checkbox is checked. (It isn't by default)

You can now edit your Docker Connection in Eclipse and do not need to search for a connection name, simply enter how you want to name it (ie. SomeDockerConnection) and check the "TCP Connection" radio button.

You can enter `tcp://localhost:2375` in the URI and click "Test Connection". You should see a message appear to confirm that the ping was successful.

At this point you'll be able to see your containers and images in Eclipse's Docker Explorer perspective.
> 12 votes

# Answer
I tried the following:

1. Docker Settings: General: Check box enable: "Expose daemon on tcp://localhost:2375 without TLS
2. Eclipse docker tooling: Connect to a docker daemon: TCP Connection: URI: tcp://localhost:2375

I am now able to see images and containers listed in the 'Docker Explorer' view of Eclipse.
> 0 votes

---
Tags: docker, windows, eclipse
---


# Question
Title: GitLab CI Job branch not up to date with origin
I'm new to GitLab CI and don't really understand how Git works within the Jobs. When I'm checking git status on the first running job it is always on detached head state. If I'm switching to another branch (for which job was triggered) and checking git status, it is not up to date with the origin. Pulling the changes, can lead to merge conflics (pull --ff-only or pull --rebase not always help). Can you please explain how to handle this, always being up to date with the triggered branch?

PS. I'm starting jobs using GitLab's trigger API (https://docs.gitlab.com/ee/ci/triggers/)

# Answer
> 1 votes
The way GitLab CI is supposed to work is that you have a `gitlab-ci.yml` file in your repo with pipeline instructions for what GitLab should do when your repo gets pushed or a pull request is submitted to your repo. The pipeline is specific to each branch and will execute only on the branch to which you've pushed changes unless you've configured it otherwise. You can have entirely different pipelines for different branches. You can also manually run the jobs from within GitLab or via API, as you've been doing, but the power of it is that it runs automatically in the background on every code push.

What GitLab does with your repo is that it spins up a Docker container, clones your repo into it, runs whatever instructions you've given it, and then takes the artifacts it's generated, such as a build or test result, and makes them available to the next step in the pipeline, which could be pushing it to your server or your test coverage tool.

From what you're describing, and please correct me if I'm wrong, but it sounds like you're using your pipeline to alter the actual contents of your repo itself. I'm not sure that's the best move. The point of the pipeline process is to run tests, builds, and deployments. You can use it to push a package to NPM, or upload fresh code to your web server, or check your code for vulnerabilities. But I wouldn't use it to actually change code in the repository directly because the unintended side effects could be disastrous.

---
Tags: gitlab
---


title: Diagrams in Azure DevOps

text:
# Question
Title: Diagrams in Azure DevOps
I'd like to include markdown diagrams in my project's source code. I'd also like to see them rendered in Azure DevOps when looking at source/pull-requests when I click on the Preview tab for the file.

md files are supported, but it seems mermaid diagrams are not. Is there a way to enable mermaid diagrams, or is there an alternative I can use that is supported?

Note: Not in the Wiki, in a file in with the source code repository.

# Answer
I had this very same issue. Sadly in the code repos themselves you cannot view the rendered diagrams.

The diagrams (and only a few supported types are rendered) are only rendered in the "Wiki from code"

There is this extension in the VS Marketplace to render the `.mmd` mermaid files but that's not inline documentation with markdown.

There's also the issue that the syntax for mermaid diagrams in the markdown documents breaks traditional spec by using colons instead of backquotes:

```
:::mermaid
  graph TD;
    A-->B
:::

```

v.s.

```mermaid
  graph TD;
    A-->B
```

This wrecks many mermaid plugins in editors like VS Code when it comes to syntax highlighting language support. But thankfully that will actually still render in the markdown preview panes.

* VSCode Mermaid Extensions
    + `bierner.markdown-mermaid`
        - render the diagrams
    + `bpruitt-goddard.mermaid-markdown-syntax-highlighting`
        - Syntax highlighting for mermaid code in code fences (not DevOps Colon syntax)

Some workaround for viewing mermaid diagrams in the markdown documents without being in the code wiki is the use of mermaid INK, with an issue I filed recently, they also updated the supported graph types on mermaid INK so now we can embed the diagrams that way and can get around the outdated version of mermaid that DevOps currently supports.

To use mermaid INK all you need to do is write your mermaid code in the I've editor and then grab the markdown link at the bottom of the editor and embed the diagram into your markdown document. This is a hack because now you lose out on version control of plain text diagram code, but the link to the image will take you back to the editor with an exact copy of the code you used to generate it.
> 2 votes

---
Tags: azure-devops
---


title: How to install Certbot plugins?

text:
# Question
Title: How to install Certbot plugins?
I installed Certbot by following this tutorial:

```
sudo add-apt-repository ppa:certbot/certbot
sudo apt-get update
sudo apt-get install python-certbot-nginx

```

And now I want to set up my cert, but I need the `dns-digitalocean` plugin:

```
# certbot certonly --dns-digitalocean
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Could not choose appropriate plugin: The requested dns-digitalocean plugin does not appear to be installed
The requested dns-digitalocean plugin does not appear to be installed

```

I tried installing it with `pip`:

```
pip install certbot-dns-digitalocean

```

But apparently the apt-get version isn't "seeing" it.

How do I install it properly?

# Answer
First run

```
# type certbot
certbot is hashed (/usr/bin/certbot)

```

To find out where `certbot` is installed to. Or `command -v certbot` if you prefer.

Then run `head /usr/bin/certbot` and note what version of Python it's using:

```
#!/usr/bin/python3

```

In my case, it was using Python 3.

I noticed from my pip output it was trying to install a Python 2.7 package:

```
# pip install certbot-dns-digitalocean
Requirement already satisfied: certbot-dns-digitalocean in /usr/local/lib/python2.7/dist-packages

```

So how do we get pip to install Python 3 packages instead? Just copy the instructions from here:

```
cd /tmp
curl -O https://bootstrap.pypa.io/get-pip.py
python3 get-pip.py
rm get-pip.py

```

Now you should have the `pip3` command, so run this instead:

```
pip3 install certbot-dns-digitalocean

```

And now try again:

```
# certbot plugins
Saving debug log to /var/log/letsencrypt/letsencrypt.log

-------------------------------------------------------------------------------
* dns-digitalocean
Description: Obtain certs using a DNS TXT record (if you are using DigitalOcean
for DNS).
Interfaces: IAuthenticator, IPlugin
Entry point: dns-digitalocean =
certbot_dns_digitalocean.dns_digitalocean:Authenticator

```
> 15 votes

# Answer
Better method, thanks to the others responses for helping me get to this.

Determine what plugins are installed currently:

```
# certbot-auto plugins
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
* apache
Description: Apache Web Server plugin - Beta
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: apache = certbot_apache.entrypoint:ENTRYPOINT

* nginx
Description: Nginx Web Server plugin
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: nginx = certbot_nginx.configurator:NginxConfigurator

* standalone
Description: Spin up a temporary webserver
Interfaces: IAuthenticator, IPlugin
Entry point: standalone = certbot.plugins.standalone:Authenticator

* webroot
Description: Place files in webroot directory
Interfaces: IAuthenticator, IPlugin
Entry point: webroot = certbot.plugins.webroot:Authenticator
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

```

Determine where your certbot (in my case certbot-auto) is installed:

```
# find / -name certbot
/opt/eff.org/certbot
...

```

Get into the Virtual Env and install plugin

```
cd /opt/eff.org/certbot/venv
source bin/activate
pip install certbot-dns-google
deactivate

```

Verify certbot plugins again

```
# certbot-auto plugins
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
* apache
Description: Apache Web Server plugin - Beta
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: apache = certbot_apache.entrypoint:ENTRYPOINT

* dns-google
Description: Obtain certificates using a DNS TXT record (if you are using Google
Cloud DNS for DNS).
Interfaces: IAuthenticator, IPlugin
Entry point: dns-google = certbot_dns_google.dns_google:Authenticator

* nginx
Description: Nginx Web Server plugin
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: nginx = certbot_nginx.configurator:NginxConfigurator

* standalone
Description: Spin up a temporary webserver
Interfaces: IAuthenticator, IPlugin
Entry point: standalone = certbot.plugins.standalone:Authenticator

* webroot
Description: Place files in webroot directory
Interfaces: IAuthenticator, IPlugin
Entry point: webroot = certbot.plugins.webroot:Authenticator
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

```
> 22 votes

# Answer
Now (July 2018), you should be able to use either

```
pip install certbot-dns-digitalocean

```

or

```
git clone https://github.com/certbot/certbot.git
cd certbot/certbot-dns-digitalocean/
python setup.py install

```

You may need `sudo` privileges for either.

After installing you may not be able to see the plugin with `certbot plugins`, but you should be able to `certbot certonly --dns-digitalocean` just fine.
> 8 votes

# Answer
You have to use Docker to use dns plugins. From DNS Plugins:

> These plugins are still in the process of being packaged by many distributions and cannot currently be installed with `certbot-auto`. If, however, you are comfortable installing the certificates yourself, you can run these plugins with Docker.
> 3 votes

# Answer
The way you install certbot plugins depends on how you installed certbot itself. If you installed certbot using some package manager (apt, rpm, brew...), then you should look for compatible certbot plugins in that package manager's repository.

Let's Encrypt also support an alternative installation method: the certbot-auto wrapper. This wrapper creates a private Python virtual installation (generally in `/opt/eff.org/certbot/venv`), and install certbot into that directory. A nice feature of certbot-auto is that it automatically keeps the certbot client up-to-date. A major downside is that it does not *officially* supports plugins installation (that is, aside from four plugins that are installed by default).

It is easy enough to work around this limitation, as described in Ryan G's solution. However, plugins installed through that procedure will be lost every time certbot-auto updates itself, which can result in random renew failures. Here, we have had a few situations where some certificates almost reached expiration because of that issue. Several tickets discuss this issue on certbot's bug tracker, and the team acknowledge the problem, but it seems that it might still be a long way before the issue is actually fixed.

Therefore, if using certbot-auto in an automated setup, it is desirable to either prevent certbot-auto's self updating (by running it with `--no-self-upgrade`), or to implement some strategy to ensure that required plugins are automatically reinstalled every time certbot is updated.

A possible solution to indeed ensure that required plugins are installed is to add a wrapper around certbot-auto. That wrapper could essentially look as follow:

```
#!/bin/bash

# The list of plugins to be installed
CERTBOT_PLUGINS="certbot-dns-route53"

# Force the venv directory to be where we can easily find it
export VENV_PATH="/opt/eff.org/certbot/venv"

# Force certbot-auto to be where we expect it to be
export CERTBOT_AUTO="/usr/local/bin/certbot-auto-upstream"

# Force certbot-auto to bootstrap or upgrade itself, but do no more
"${CERTBOT_AUTO}"  --install-only  "$@"

# Check if required plugins are installed; install them if they are missing
(
    cd ${VENV_PATH}
    source bin/activate

    for plugin in $CERTBOT_PLUGINS ; do
        if ! pip show -q "$plugin" ; then
            pip install "$plugin"
        fi
    done

    deactivate
)

# Execute the actual certbot command
"${VENV_PATH}/bin/letsencrypt" "$@"

```

I have made available a more complete version of that wrapper here; the only differences with the longer version is that it ensures that the wrapper is being run as root, and it properly handle the `--help` argument.

To install that wrapper, download the official `certbot-auto` program to `/usr/local/bin/certbot-auto-upstream`, and copy the wrapper to `/usr/local/bin/certbot-auto`. Make sure both files have proper privileges (`chown root:root /usr/local/bin/certbot-auto*`, then `chmod 755 /usr/local/bin/certbot-auto*`). In the wrapper file, make sure the line `CERTBOT_PLUGINS="..."` includes the list of plugins you actually need. And that's it. Simply use the `certbot-auto` command, as you would have done previously, and forget about the `certbot-auto-upstream` file.
> 2 votes

# Answer
For those who is using **`certbot`** distributed by OS distros *certbot plugins* can be installed by a package manager using provider's name, for example:

**Debian/Ubuntu:**

```
apt-get install python3-certbot-dns-cloudflare
apt-get install python3-certbot-dns-google
apt-get install python3-certbot-dns-route53

```

**RHEL/CentOS/Rocky/AlmaLinux 8:**

```
dnf install python3-certbot-dns-cloudflare
dnf install python3-certbot-dns-google
dnf install python3-certbot-dns-route53

```

**RHEL/CentOS 7:**

```
yum install python2-certbot-dns-cloudflare
yum install python2-certbot-dns-google
yum install python2-certbot-dns-route53

```
> 2 votes

# Answer
If you're on ubuntu or debian, you can grab the following packages from debian testing (buster)

python3-certbot-dns-digitalocean\_0.23.0-2\_all.deb python3-digitalocean\_1.13.2-1\_all.deb
> 1 votes

# Answer
This worked for me on Ubuntu 18.04 LTS

```
apt install certbot
apt install python3-certbot-dns-digitalocean

```

See this certbot issue
> 1 votes

# Answer
I had the same problem, after I updated certbot on OS X I could not get the digitalocean plugin to appear, even after re-installing it with `pip install certbot-dns-digitalocean`.

The solution was to uninstall it and then re-install it. I used sudo just to be safe:

```
sudo pip uninstall certbot-dns-digitalocean
sudo pip install certbot-dns-digitalocean

```

It then appeared ok in the `certbot plugins` list.
> 1 votes

---
Tags: ssl, certbot
---


# Question
Title: Kubernetes etcd bootstrapping - couldn't find local name <name> in the initial cluster configuration
I followed kelsey Hightower's setting up kubernetes the hard way tutorial kubernetes the hard way on AWS

Stuck on bootstrapping etcd on the first control plane node. There are 2 lines in the error message which concerns me:

1. The node is already initialized as member, starting as member..

* i have tried deleting the data directory while reinitializing but still same message

2. couldn't find local name in the intial cluster configuration

Is it picking up old configuration data from somewhere other than the data directory?

# Answer
Well, it's labeled as *hard* :-D

Without knowing more troubleshooting steps that you've already taken, it's hard to know what to suggest for that first item, but the second one is almost certainly that its own name is not present in the `--initial-cluster` list. You'll also want to give etcd the full hostname as AWS's DNS returns it (`ip-10-20-30-40.us-east-2.compute.internal`), to save yourself future troubles when those etcd members attempt to look up their own name. That's almost certainly true of the `Node` names if you choose to use `cloud-provider=aws` when you get to that part, too

If this is the beginning of your kubernetes journey, then this is a great way to establish your future relationship with `etcd`, as it will be the source of unending pain and opaque messages. I would actually also encourage you to pause and go through their docs because having a very firm grasp on what etcd wants, how to debug it, how to *recover from it*, and how to drive `etcdctl` will pay dividends over your future kubernetes lifespan
> 1 votes

# Answer
There was a mismatch in the --initial-cluster list and the local name i was providing. Be sure to check the configuration carefully when following the tutorial.
> 3 votes

---
Tags: kubernetes, amazon-ec2
---


# Question
Title: Create and delete ingress for temporary containers
(I have just stared learning kubernetes, so apologies if my question sounds silly)

In my cluster I am running some pods that execute some batch operations and expose a webUI to keep track of the overall progress.

To access the webUI I am currently using a port-forward, but I was wondering

Is there any way to get a disposable ingress that is tied to the container lifecycle, in such a way that it is automatically created when the pod is started, and deleted whenever the pod terminates?

# Answer
> Is there any way to get a disposable ingress that is tied to the container lifecycle, in such a way that it is automatically created when the pod is started, and deleted whenever the pod terminates?

Is there a way? Absolutely, but be aware that's for sure not how kubernetes thinks about the world, so you are going to be fighting uphill trying to maintain a system like that. With that said:

## Helm

*This part I have done, extensively, and works as advertised*

The way most people manage groups of related resources is via helm, which in your case would create the `Job` as well as the `Ingress` resource for it. When the Job has finished (a process that helm, itself, does not meddle in), you can remove all grouped resources in one shot via `helm uninstall` which will delete the `Job` and its associated `Ingress`.

I believe it may be possible to hook the Job completion events with other machinery, but since this part is confined to things I have actively done, I wouldn't want to speculate on the "best" way for that

## Admission Controller

*this section is speculation, as I have not personally worked with this part*

Kubernetes offers dynamic admission controllers which are *intended* to allow one to have very complicated rules about what workloads one allows in their cluster, but has the pleasing side-effect of being consulted before Pod creation (and I can only presume deletion) takes place. That enables you to run arbitrary code in response to those webhook events, and then just return "Allow" from the validating admission controller
> 1 votes

---
Tags: kubernetes
---


# Question
Title: Docker compose up, Error starting userland proxy: listen tcp 0.0.0.0:22: bind: address already in use
I'm trying to a `gitlab/gitlab-ce` image up and running with docker-compose on macOS (Big Sur) (`docker-compose up -d`). I'm using `docker-machine` with the virtualBox driver.

However, when starting the container I get the following error:

```

ERROR: for web  Cannot start service web: driver failed programming external connectivity on endpoint gitlab-docker_web_1 (236233cf090b94f98e2e82c2fa795bc7f6f5ff38e2ea14f3649e87313a21d196): Error starting userland proxy: listen tcp 0.0.0.0:22: bind: address already in use
ERROR: Encountered errors while bringing up the project.

```

The issues is, I can't find anything running on port 22 on the host.

```
> sudo lsof -i :22
>

```

returns nothing. Running `netstat -l | grep 22` to look for listening ports I don't see anything either:

```
tcp4       0      0  byrons-mac-pro.l.61179 229.26.211.130.b.https ESTABLISHED
tcp4       0      0  byrons-mac-pro.l.60764 108.177.122.108.imaps  ESTABLISHED
tcp4       0      0  byrons-mac-pro.l.60761 108.177.122.108.imaps  ESTABLISHED
tcp4       0      0  byrons-mac-pro.l.60760 108.177.122.108.imaps  ESTABLISHED
tcp4       0      0  byrons-mac-pro.l.60759 108.177.122.108.imaps  ESTABLISHED
tcp4       0      0  byrons-mac-pro.l.60758 108.177.122.108.imaps  ESTABLISHED
tcp4       0      0  byrons-mac-pro.l.60755 108.177.122.108.imaps  ESTABLISHED
tcp4       0      0  byrons-mac-pro.l.60754 108.177.122.108.imaps  ESTABLISHED
tcp4       0      0  byrons-mac-pro.l.54147 yi-in-f188.1e100.5228  ESTABLISHED
tcp4       0      0  byrons-mac-pro.l.49743 104.22.52.136.https    ESTABLISHED
tcp4       0      0  byrons-mac-pro.l.49734 yv-in-f188.1e100.5228  ESTABLISHED
tcp4       0      0  byrons-mac-pro.l.49556 108.177.122.95.https   ESTABLISHED
tcp4       0      0  byrons-mac-pro.l.49173 17.57.144.150.5223     ESTABLISHED
tcp4       0      0  byrons-mac-pro.l.61183 52.96.122.50.https     TIME_WAIT
tcp4       0      0  byrons-mac-pro.l.61182 52.96.122.50.https     TIME_WAIT
udp4       0      0  byrons-mac-pro.l.56826 108.177.122.139.https
c0fd8701c6b3232b stream      0      0                0 c0fd8701c6b32263                0                0
c0fd8701c6b32263 stream      0      0                0 c0fd8701c6b3232b                0                0
c0fd8701b7e36583 stream      0      0 c0fd8701b9bb407b                0                0                0 /var/folders/yt/c5z_l8554mq_fc3f0c5tpw600000gn/T/dotnet-diagnostic-2202-1621045110-socket
c0fd8701add5cb63 stream      0      0                0 c0fd8701add5ca9b                0                0 /var/folders/yt/c5z_l8554mq_fc3f0c5tpw600000gn/T/vscode-ipc-a70bbba5-ec65-4f40-861a-0bc122628a75.sock
c0fd8701a9b0232b dgram       0      0                0 c0fd8701a9b02263 c0fd8701a9b02263                0
c0fd8701a9b02263 dgram       0      0                0 c0fd8701a9b0232b c0fd8701a9b0232b                0

```

I'm a bit stumped. Here's my `docker-compose.yml`:

```
web:
  image: "gitlab/gitlab-ce:latest"
  restart: always
  hostname: "byrons-mac-pro.lan"
  environment:
    GITLAB_OMNIBUS_CONFIG: |
      external_url 'http://byrons-mac-pro.lan'
      gitlab_rails['smtp_enable'] = true
      gitlab_rails['smtp_address'] = 'smtp.gmail.com'
      gitlab_rails['smtp_port'] = 587
      gitlab_rails['smtp_user_name'] = 'remove@cantellyou.com'
      gitlab_rails['smtp_password'] = '.........................'
      gitlab_rails['smtp_domain'] = 'smtp.gmail.com'
      gitlab_rails['smtp_authentication'] = 'login'
      gitlab_rails['smtp_enable_starttls_auto'] = true
      gitlab_rails['smtp_tls'] = false
      gitlab_rails['smtp_openssl_verify_mode'] = 'peer' # Can be: 'none', 'peer', 'client_once', 'fail_if_no_peer_cert', see http://api.rubyonrails.org/classes/ActionMailer/Base.html
      puma['worker_processes'] = 2
      sidekiq['max_concurrency'] = 10
      prometheus_monitoring['enable'] = false
  ports:
    - "80:80"
    - "443:443"
    - "22:22"
  volumes:
    - "$GITLAB_HOME/config:/etc/gitlab"
    - "$GITLAB_HOME/logs:/var/log/gitlab"
    - "$GITLAB_HOME/data:/var/opt/gitlab"

```

# Answer
It is a quirk of the fact that macOS does not have native `docker`, and thus all(?) implementations have a backing virtual machine in which the Linux kernel and dockerd are running (something that I would have thought was obvious from the `docker-machine` you are currently aware of, but ... I guess not). It is almost certainly *that* machine which is listening on 22, given that most folks actually *do* want access to that virtual machine for troubleshooting

I have better than average confidence that you can customize the virtual machine which `docker-machine` starts and ask it to bind ssh to :2222 (akin to this), or "cheat" and `docker-machine ssh systemctl stop sshd.service` type thing, freeing up the ssh port but from that point forward `docker-machine ssh` and maybe even `docker-machine stop` might not work as you expect
> 1 votes

---
Tags: docker, docker-compose, gitlab, macos, docker-machine
---


# Question
Title: Setting up CircleCI to AWS EC2 for a simple node application deployment
I have a simple node application running on the EC2 instance, everything is working just fine and I want to integrate this application with CircleCI for the continuous deployment. I have been following this article where it says to SSH to the EC2 instances and then run your deployment commands.

I've created a basic `deploy.sh` file in the EC2 instance, where I simply change the directory to project directory and then run `npm i` and `pm2 delete all` with `pm2 start server.js`. Now before testing it out my circleCI I thought of checking if it's working in general or not, so

1. When I run this deploy.sh file from within EC2 instance, it works fine and follows all the steps to install `node_modules` and then serving the application with `pm2` correctly. So, no issues here.
2. From my local machine when I tried to run the same deploy.sh with this command `ssh -i <my-pem-file> <ec2-user>@<ec2-ip> \ ./deploy.sh`. it `echoes` the basic stuff but says *`npm` and `pm2` : not found* . The reason this is happening is because that deploy.sh file is being run in context of my local machine (or not?).

So how can I simply run this `deploy.sh` file right after I SSH to my EC2 instance from circleCI, or If there is some better approach to do this all together.

# Answer
I would look at AWS System Manager to do this instead of direct ssh. Create a SSM document to run the commands you want to run, then this can be invoked by an AWS cli command from your CI/CD pipeline.

We do something the same to update configuration files from an s3 bucket, run as part of a Gitlab pipeline.
> 1 votes

---
Tags: amazon-ec2, continuous-deployment, bash, ssh, circleci-2.0
---


# Question
Title: Passing parameters of extendedChoice type to Jenkins Pipeline via jenkins-cli.jar
I have the following declarative pipeline. It defines checkbox multi-choice parameter:

```
pipeline {
    agent any
    options {
        ansiColor('xterm')
    }
    parameters {
        extendedChoice description: '', multiSelectDelimiter: ',', name: 'NAMES', quoteValue: false, saveJSONParameterToFile: false, type: 'PT_CHECKBOX', value:'a,b,c', visibleItemCount: 3
    }
    stages {
      stage('Run on all') {

        steps {
            script {
                  println params.NAMES
            }
        }
      }
    }

}

```

My goal is to call the pipeline via Jenkins CLI where passed values to `NAMES` parameter signifies selected checkboxes.

Running `java -jar jenkins-cli.jar -s http://localhost:8080 -auth u:p -p NAMES='a'` causes empty string to be printed by `println params.NAMES`

Expected `a` to be printed after calling the pipeline via Jenkins cli client.

The only way that worked is adding commas after `a`: `java -jar jenkins-cli.jar -s http://localhost:8080 -auth u:p -p NAMES='a,,'`

Is it the correct syntax to pass checkbox selections of `extendedChoice` type parameter?

# Answer
> 1 votes
Try with tokenize (,) since the options were saved like a string. Save 'NAMES' into a new variable, then tokenize with a comma "," since it was configured the parameters with multiSelectDelimiter: ',' Finally, you can iterate the array with a for-loop and use the elements as desire.

```
  def arr = userInput['NAMES'].tokenize(',')

  arr.each { t ->
    // t is the parameter option
    sh("echo ${t}")
  }

```

---
Tags: jenkins, jenkins-pipeline, jenkins-plugins
---


# Question
Title: Cloudflare page rules for removing multiple trailing slash
Cloudflare page rules for removing multiple trailing slash

I want to set up the below things.

`https://domain.com//` OR `https://domain.com//` OR `https://domain.com///` OR onwards

are redirecting to `https://domain.com/`

How can I set up this using the Cloudflare page rule?

# Answer
> 1 votes
It will be quicker to set that in your webserver(Nginx) virtual host than doing it on Cloudflare.

Add below in your Nginx config:

```
if ($request_uri ~* "\/\/") {
  rewrite ^/(.*)      https://$host/$1    permanent;
}

```

---
Tags: cloudflare
---


# Question
Title: How to filter all roles with specific user in GCP
`gcloud projects get-iam-policy [PROJECT-ID]` lists all users with their roles for specific project.

There are different filters and formatters available but I can't seem to find the right way to just filter only by specific user.

> **HINT**: I reused the example code of DevOps - How to list all users with specific role in GCP

```
$ gcloud projects get-iam-policy MY_PROJECT

bindings:
- members:
  - serviceAccount:12345678-compute@developer.gserviceaccount.com
  - user:alice@foobar.com
  role: roles/editor
- members:
  - serviceAccount:12345678-compute@developer.gserviceaccount.com
  role: roles/pubsub.serviceAgent
- members:
  - user:you@yourdomain.com
  - user:someoneelse@yourdomain.com
  role: roles/owner
etag: ARBITRARY_ETAG_HERE
version: 1

```

What's the `--filter=` expression that I need to use to list only items with the **members** `- serviceAccount:12345678-compute@developer.gserviceaccount.com`?

# Answer
After a bit more googling, I found that post.

Sorry!

Here is the solution which works for me

```
gcloud projects get-iam-policy MY_PROJECT --format='table(bindings.role)' --filter="bindings.members:serviceAccount:12345678-compute@developer.gserviceaccount.com"

```
> 1 votes

---
Tags: google-cloud-platform, gcp
---


# Question
Title: Jenkins Dockerfile declarative pipelines: getting the built image tag
I'm trying to get the image tag generated with the help of the dockerfile agent in Jenkins declarative pipelines.

This Jenkinsfile is incomplete though, because I still need the deployment steps.

Normally, people would then want a step to push to Dockerhub. Although I don't even know how to do that, I want something different: **I want to re-tag the built image locally** to a name of my choice. This is because this is a hobby project, and my Jenkins and production servers are the same. The **problem is that I don't know how to get the dynamically generated image ID** for the `docker tag` command.

(Then what would come next is to swap out the old running image with the new one, but I could probably take care of that myself.)

Here is my Jenkinsfile:

```
pipeline {
    agent {
        dockerfile {
            args '--network szeremi -e DB_HOST=mysql -e APP_ENV=testing'
        }
    }

    stages {
        stage('Run tests') {
            steps {
                sh 'cd /var/www && php artisan migrate:fresh && vendor/bin/phpunit'
            }
        }
    }
}

```

Please help me use Jenkins declarative pipelines to re-tag my built-by-dockerfile image.

# Answer
As I understand you want to add a variable to the docker image tag,

in this case, you have to get an environment variable from Jenkins and use it in your image label.

```
    pipeline {
  environment {
    registry = "docker_hub_account/repository_name"
    registryCredential = 'dockerhub'
  }
  agent any
  stages {
    stage('Building image') {
      steps{
        script {
          docker.build registry + ":$BUILD_NUMBER"
        }
      }
    }
  }
}

```

pleae use this URL for more info
> 1 votes

# Answer
`docker build` *does* output the image ID:

```
Successfully built 6fd2b9d56348
Successfully tagged foo:latest

```

Unfortunately, it doesn't seem like the Docker Build Step plugin exposes it to the pipeline (it *does* write it to the log!). You may want to open an issue with that project.

That said, `docker tag` itself accepts `name:tag`:

```
$ docker help tag
Usage:  docker tag SOURCE_IMAGE[:TAG] TARGET_IMAGE[:TAG]

```

Since you (can) choose image name and first tag for `docker build`, try using those to identify the built image instead of its ID. If that doesn't work with the plugin, you may need a feature request after all.

As a workaround, you can always use `sh script: 'docker ...'` directly.
> 1 votes

# Answer
Probably you need `additionalBuildArgs` to tag your image as `mytag`:

```
    agent {
        dockerfile {
            additionalBuildArgs '-t mytag'
        }
    }

```
> 0 votes

---
Tags: docker, jenkins, jenkins-pipeline, dockerfile
---


# Question
Title: Surprising restarts of Kubernetes pods despite restartPolicy=Never
I have a `cronjob` with `cronjob.spec.jobTemplate.spec.template.spec.restartPolicy=Never`. And I am using `kubectl create job --from=cronjob/my_cronjob my_test` for manually starting an instance outside the schedule.

What I am observing is that subsequent instances of the resulting `pod` start after the first one has resulted in an error. The jobs `job.spec.template.spec.restartPolicy` and the pods' `pod.spec.restartPolicy` are also `Never` (as expected), so what could bring these restarts about?

# Answer
So both `cronjob.spec.jobTemplate.spec.template.spec.restartPolicy` and `cronjob.spec.jobTemplate.spec.backoffLimit` influence such restarts. Setting the 2nd one to 0 (and keeping the 1st one at `Never`) brought about the desired behavior.
> 1 votes

---
Tags: kubernetes
---


# Question
Title: ADO: Able to use pipeline variables in a query?
I would like to have a gate which requires at least one work item of the 'Release' type associated with the pipeline. Is there a way to have the gate query execute `SELECT [System.Id] FROM WorkItems WHERE [System.WorkItemType] = 'Release' AND [Com.Pipeline] = "$(Release.DefinitionName)"` (or equivalent)?

I need it to work through multiple pipelines, which is why I would like to use on of the built-in variables to avoid creating a hundred separate queries. If not possible, is there a better way to stop deployment from moving to the next stage if there isn't a specific associated work item?

# Answer
> 1 votes
So ended up reading about the WIQL API calls, and used an 'Invoke Web Request' task to execute the above query, with the success criteria being `root['workItems'][0]`, which errors out if there isn't an element present. Not quite as elegant as the Query task I was shooting for, but it serves its purpose.

---
Tags: azure-devops
---


# Question
Title: How to determine whether the Jenkins build was started by a user or the scheduler?
I have a Jenkins job which does some unit-testing for some code. It can be started in two possible ways:

1. By the scheduler - it polls the Perforce server, and starts the job when it detects a code change
2. By a user, on demand - it downloads a shelved/stashed code change, and checks whether it breaks the tests

The second way is called "Build Review":

---

If the job discovers problems, I want to send emails conditionally, to avoid spam:

1. If invoked by the scheduler, send mails to everyone who did check-ins since last successful build (called "developers" in Jenkins jargon)
2. If invoked by a specific user, send a mail only to that user

I am using the Any Build Step plugin to add two conditional instances of Editable email Notification into my job settings. But what are the conditions? How can I tell it to send mail only if invoked in a specific way, (1) or (2) above?

# Answer
> 0 votes
The answer is in environment variables. All builds should have the "Environment Variables" link, which shows all the environment variables, which you can easily compare. In my case, the difference is in the `BUILD_CAUSE` and `review` variables:

Build 1 (started by the scheduler):

```
BUILD_CAUSE = SCMTRIGGER

```

Build 2 (started by "build review"):

```
BUILD_CAUSE = MANUALTRIGGER
review = 24564765

```

---

This solution won't work in the general case, because I have some groovy script and/or some plugin which set these environment variables.

# Answer
> 2 votes
I'm not sure if there is a simple way to check if the "scheduler" kicked off the job, but you can check if a user executed the job.

In your Jenkins job, check for the presence of the `BUILD_USER_ID` environment variable. If it is present, retrieve the value and send your email to the specific user(s) (scenario 1). If it is not present, a non-user has executed the job, and you can email the users listed on the last successful check-in (scenario 2).

If you are looking for methods outside of the Jenkins job, you can also use the Jenkins API (http://jenkinsurl/job/\[JobName\]/\[BuildNumber\]/api/json?pretty=true) and parse the `BUILD_USER_ID` from the output.

# Answer
> 0 votes
I found a very useful plugin:

**Environment Injector Plugin** https://plugins.jenkins.io/envinject/

Just configure the job with:

☑ Inject environment variables to the build process

---

## From plugin webpage:

This plugin also exposes the cause of the current build as an environment variable. A build can be triggered by multiple causes at the same time e.g. an SCM Change could have occurred at the same time as a user triggers the build manually.

The build cause is exposed as a comma separated list:

```
BUILD_CAUSE=USERIDCAUSE, SCMTRIGGER, UPSTREAMTRIGGER, MANUALTRIGGER

```

In addition, each cause is exposed as a single environment variable too:

```
BUILD_CAUSE_USERIDCAUSE=true
BUILD_CAUSE_SCMTRIGGER=true
BUILD_CAUSE_UPSTREAMTRIGGER=true
BUILD_CAUSE_MANUALTRIGGER=true

```

---
Tags: jenkins, email
---


title: What is "Push on Green"?

text:
# Question
Title: What is "Push on Green"?
Within Google and other DevOps organisations, they talk about "Push on Green" which I am lead to believe is the practice of continuous delivery based upon a successful run of automated tests, and presumably deployments to pre-live environments.

What exactly does "Push on Green" and what are the implications of applying it?

# Answer
> 10 votes
I Googled "Push on Green" and the first link was:
https://www.usenix.org/conference/lisa14/conference-program/presentation/klein

This was representative of almost the entire first page. It looks like this term originated in Google's SRE group and has been taken up by the industry at large.

You are correct- "push on green" means that deployments are automatically executed when all tests pass. This could be considered synonymous with "Continuous Deployment." The accepted method for automating deployments to environments is to automate testing.

If your automated testing is rigorous enough, you can deploy any code for which the tests pass. In very mature enterprises (the DevOps unicorns of all shapes and sizes) this can lead to Continuous Deployment all the way to the production environment.

My personal experience is that reaching this level of automated testing is a challenge in any enterprise. I am currently in the process of pursuing this level of maturity in my current position. Our first major milestone will be "push on green" deployments to *any* environment beyond "Integrated Development" (our "lowest" environment.)

This challenge is both technical and cultural. As an IT organization, we owe it to our business stakeholders to prove that our testing *is* in fact rigorous enough to serve as the sole gateway to a deployment. Once we have done that to our own satisfaction, we have to convince those stakeholders to let us try Continuous Deployment to the next environment (for us, this is QA.)

# Answer
> 0 votes
Shorter definition:

Push on Green: After each commit, if all tests pass, the new code is pushed to production.

---
Tags: continuous-integration, terminology, continuous-delivery
---


# Question
Title: How to implement custom HAProxy 404 responses?
My goal is to build custom 404 http error code that will depend on a user's language. I thought that I can use header accept language.

I thought about something like this:

**#creating 2 acls that accepts only one accept language.**

```
acl en req.fhdr(accept-language),language(en;pl) -m str en
acl pl req.fhdr(accept-language),language(en;pl) -m str pl

```

**#and here i would like to use backends depends on acl's**

```
use_backend polish404 if pl
use_backend english404 if en

backend polish404
  mode http
  errorfile 503 /path/to/file

backend english404
  mode http
  errorfile 503 /path/to/file

```

Any ideas how to implement this?

@first\_edit 7.07.2021

I've created this using Samit's advice:

```
http-errors pl_ver
  errorfile 404 /path/to/file
http-errors en_ver
  errorfile 404 /path/to/file

acl en req.fhdr(accept-language),language(en;pl) -m str en
acl pl req.fhdr(accept-language),language(en;pl) -m str pl

http-response status 404 errorfiles pl_ver if { status 404 } { pl }
http-response status 404 errorfiles en_ver if { status 404 } { en }

```

Anyone have an idea how to implement this in other way?

@second\_edit 24.07.2021

I've got a better implementation of that problem for future seekers here you go:

```
# Defining HTTP errors
http-errors pl_ver
  errorfile 404 /path/to/file
http-errors en_ver
  errorfile 404 /path/to/file

# Setting variable (under frontend section)
  http-request set-var(txn.lang) req.hdr(Accept-Language)

# Feeding with response
  http-response return status 404 errorfiles pl_ver if { status 404 } { var(txn.lang) pl }
  http-response return status 404 errorfiles en_ver if { status 404 } { var(txn.lang) en }

```

# Answer
You can make use of http-errors for creating different error code and file mappings.

After that, you can easily make use of `http-response` in haproxy config to return based on the error code. An example is below:

```
http-errors site1
    errorfile 404 /etc/haproxy/errors/site1-404.http

http-errors site2
    errorfile 404 /etc/haproxy/errors/site2-404.http

frontend allsites
    bind :80
    default_backend site1-servers
    use_backend site2-servers if { req.hdr(host) site2.com }

    # Store host header in variable
    http-request set-var(txn.host) req.hdr(host)

    # Use site1 error page if site1.com
    http-response return status 404 errorfiles site1 if { status 404 } { var(txn.host) site1.com }

    # Use site2 error page if site2.com
    http-response return status 404 errorfiles site2 if { status 404 } { var(txn.host) site2.com }

```

You can also dynamically generate different 404 error pages from a single template by passing the request parameters from haproxy to the error template. I would recommend to go through this haproxy blog to understand in detail and try it out.
> 0 votes

---
Tags: loadbalancing
---


# Question
Title: How to force lowercase URL in cloudflare
How to force request lowercase in the URL in Cloudflare?

for example, if Cloudflare get requests for `https://example.com/signup/Test-Page-BLOG/` then it should be redirected to `https://example.com/signup/test-page-blog/`

Is that possible to use the Cloudflare page rule or any other method within Cloudflare?

# Answer
> 1 votes
AFAIK, this is not possible in cloudflare. Also, this community thread suggests that it cannot be done in cloudflare.

You can handle this at your webserver configuration itself.

---
Tags: cloudflare
---


# Question
Title: Initial file not found on docker-entrypoint
I want to run a container with docker-compose. My dockerfile is installing some stuff and at the end it looks like this:

```
FROM adoptopenjdk/openjdk16

...

COPY ./myfiles /mydockerfiles

ENTRYPOINT [ "/mydockerfiles/init.sh" ]

```

My docker compose file looks something like this

```
version: '3.3'

services:
  myservice:
    build: .
    environment:
      restart: "no"
    ports:
      - 10000:10000
    volumes:
      - type: bind
        source: ./dockerdata
        target: /mydockerfiles
    deploy:
      resources:
        limits:
          memory: 4G
        reservations:
          memory: 4G

```

Pretty basic stuff so far in my opinion. But for some reason I always get this error:

> ERROR: for ... Cannot start service myservice: OCI runtime create failed: container\_linux.go:380: starting container process caused: exec: "/mydockerfiles/init.sh": stat /mydockerfiles/init.sh: no such file or directory: unknown
>
> ERROR: for myservice Cannot start service myservice: OCI runtime create failed: container\_linux.go:380: starting container process caused: exec: "/mydockerfiles/init.sh": stat /mydockerfiles/init.sh: no such file or directory: unknown ERROR: Encountered errors while bringing up the project.

If however I open up a shell into the container and run the script manually it all works fine. Also during the build process if once tried to do a `chmod +x` on the init.sh-File and no error occured. So why is the file available during building, a manual bash; but not when using `docker-compose up`. What am I missing???

# Answer
> 1 votes
You're overwriting the COPY step in the Dockerfile with the volume mount:

```
    volumes:
      - type: bind
        source: ./dockerdata
        target: /mydockerfiles

```

Volume mounts act like other Linux filesystem mount commands, the newly mounted filesystem shadows any other files or directories that already existed at that location, and they will not be accessible as long as the mount exists. The semi-exception to this is a new named volume, which docker will initialize on first use to include the contents of the image at that location, before performing the mount.

Therefore, if the `init.sh` file is not in the host folder `./dockerdata`, the entrypoint will not be found. Other potential reasons for a shell script to show a "not found" error include:

* referencing an interpreter on the first line that doesn't exist in the container, e.g. `#!/bin/bash` in a container that only provides `/bin/sh`.
* including windows linefeeds in the script, which turn `/bin/sh` into `/bin/sh\r`, and the binary including the trailing carage return does not exist in the container filesystem.

---
Tags: docker, docker-compose
---


# Question
Title: GitLab: Make commit in (shared) CI runner and push to repository/open merge request
I want to run a scheduled pipeline on the shared runners that is able to open a merge request on my repository. I want to run a script that checks if updates for my pinned packages exists and if yes it should update these and open a merge request with a new branch. I am using https://gitlab.com and not a self hosted GitLab instance.

However pushing to my repository failed because whether using ssh or https the authentication is denied.

```
remote: You are not allowed to upload code.
fatal: unable to access 'https://gitlab-ci-token:[MASKED]@gitlab.com/<username>/example.git/': The requested URL returned error: 403

```

or

```
remote: HTTP Basic: Access denied
fatal: Authentication failed for 'https://<username>:@gitlab.com/<username>/example.git/'

```

My gitlab-ci.yml looks something like this

```
doSth:
      image: image

      script:
        - chmod 700 doSth.sh
        - chmod 700 autoUpgrade.sh
        - ./doSth.sh
        - ./autoUpgrade.sh

```

My *autoUpgrade.sh* looks something like this.

```
#!/bin/bash

if [[ `git status --porcelain` ]]; then
    git config --global user.email "email@example.com"
    git config --global user.name "example"
    export BRANCH_NAME=change-sth-`date +"%s"`
    git checkout -b ${BRANCH_NAME}
    git add -u
    git commit -m 'change sth.'
    # I tried using ssh and https
    # chmod 0400 $CI_DEPLOY_KEY
    # GIT_SSH_COMMAND="ssh -i ${CI_USER_TOKEN}" git push --set-upstream origin ${BRANCH_NAME}
    git push https://${CI_USER}:${CI_USER_TOKEN}@gitlab.com/${CI_PROJECT_PATH}.git ${BRANCH_NAME}
else

```

Is it possible to push to a new branch in my repository using the shared runners? And if yes do you know what I am doing wrong? echo "Nothing changed" fi

# Answer
> 2 votes
CI token used for code checkout is not allowed to do writes to the git repository (unlike GitHub Actions). You need to generate a new personal token and add it to secrets.

See our integration project here as a sample: https://gitlab.com/taleodor/sample-helm-cd

Note, if you search through StackOverflow, you may find several other ways to achieve this but in any case you need to explicitly pass valid credentials that are able to write to the repo.

# Answer
> 1 votes
My solution is very similar to @taleodor's, but it 1) uses the predefined CI\_REGISTRY\_USER variable, and 2) just resets the existing origin's url:

```
  before_script:
    - git remote set-url origin https://${CI_REGISTRY_USER}:${API_TOKEN}@${CI_REPOSITORY_URL#*@}

```

I'd also recommend you use an access token for the project, not the user. Create the token by going to {PROJECT} / Settings / Access Tokens. Then create one with read/write repository.

---
Tags: gitlab, gitlab-ci-runner
---


# Question
Title: Build .NET Projects using Visual Studio vs Azure Pipelines
We have many developments going on in.NET and developers use Visual Studio 2019 to Develop / Build / Test applications on their local machine first and then they publish it to Azure App Service Service (i.e Production). We are planning to construct Azure Pipeline for automated build and deployment. Would like to know, whether we should create a couple more App Service and call it as Dev and Staging? And ask the developers to directly commit all the changes to the remote repository (Azure Repo)?

If we do that, then the inbuilt option that is provided in Visual Studio will not be used by Developers. Should we ask the developers to first test it on their machine and then commit it to the remote repository? What would be the right approach?

# Answer
> 1 votes
Ideally yes, you'd want your developers to be able to run the code locally for the sake of unit tests (as one example) and then - when ready - commit to a remote repository on Azure DevOps.

You could then issue a PR, merge into (for example) the 'develop' branch, and have this branch kick off your CICD pipeline in Azure DevOps to deploy to your development environment (or App Service). You could also split your CICD into one build pipeline, which at the end stage hands off to one of three release pipelines (Dev, Test, Production). The flexibility is there.

Our developers used to use Visual Studio to deploy to AWS via the AWS Toolkit before we 'matured' into committing code to Azure DevOps and then letting the CICD pipeline take care of that step for us. Much better audit capability, processes (PR, Approvers etc) and automation.

---
Tags: azure-devops, azure-pipelines
---


title: Blue ocean logging my step commands

text:
# Question
Title: Blue ocean logging my step commands
using a minimal jenkinsfile as an example

```
node("master") {
    stage("Test") {
        sh(
            label: "THIS SHOULD DISPLAY",
            script: "set -e; echo 'this should not be logged it is secret'"
        )
    }
}

```

For some reason the following occurs

I'm not sure what's going on here. Ideally I want the cmand not to be displayed and the label to take the place of the command.

# Answer
> 1 votes
1. If you run a script you have to specify where the output goes. sh(returnStdout: true, script: "echo 'this should not be logged it is secret'") returnStdout: true is standard, set it to false and nothing is shown
2. If you want to capture the output do. def msg = sh(returnStdout: false, script: "echo 'this should not be logged it is secret'")
3. label: "THIS SHOULD DISPLAY" puts a Label on the "sup"step to display on the Jenkins site. What you see is the intended result.

conclusion:

```
node("master") {
    stage("Test") {
      script {
        def msg = sh(
            returnStdout: false,
            label: "THIS SHOULD DISPLAY",
            script: "set -e; echo 'this should not be logged it is secret'"
        )
        // do whatever you want with the output msg
      }
    }
}

```

---
Tags: jenkins, jenkins-blueocean
---


# Question
Title: How to get ConsoleFull Output,changeLog of upstream job along with downstream jobs console full output and changeLog in jenkins pipeline jobs!
Lets say we have Jobs A , B , C ,D and E. Job A triggers Job B ; Job B triggers Job C; Job C triggers Job D;, Job D triggers Job E;

When one of the child job fails for example: Job c fails then email should be sent to recipients of Job A ,Job B and Job c along with console output of A and changeLog ; Console output of Job B and changeLog and Console Output of Job C and changeLog all these info should be sent.!

Is there any way to do this ? Need to use groovy script or how?

# Answer
> 0 votes
If you really want to, you can use

```
${BUILD_URL}/consoleText

```

or even better

```
$JENKINS_HOME/jobs/$JOB_NAME/builds/lastSuccessfulBuild/log

```

there is the log of the last successful build, the others are:

```
lastCompletedBuild
lastFailedBuild
lastStableBuild
lastUnstableBuild
lastUnsuccessfulBuild

```

---
Tags: jenkins, continuous-integration, jenkins-pipeline, jenkinsfile, pipeline
---


# Question
Title: Why is parameter not propagated to post-build parameterized build?
I'm trying to apply the guidance I recently read at how to trigger parameterized builds from a different build in Jenkins and https://plugins.jenkins.io/parameterized-trigger/ \- **but this doesn't seem to work, and I'd like to ask the community for help understanding what I'm doing wrong**.

My jobs are set up thus:

* `foo2` is a non-parameterized build
    + `foo2`'s Build is `Execute shell` with commands:
```
string1="123abc"
export string2="456def"

```

+ `foo2`'s Post-build Action is `Trigger parameterized build on other projects`
        - It triggers `foo3`
        - It *attempts* to pass parameters to `foo3` with `Predefined parameters`:
    ```
    a_string=${string1}
    b_string=${string2}

    ```
* `foo3` is a parameterized build with two string paramters: `a_string` and `b_string`.
    + `foo3`'s Build step is `Execute shell` with commands:
```
echo ${a_string}
echo ${b_string}

```

**Question/Problem: when `foo3` is triggered, its console output are the string literals `${string1}` and `${string2}`, instead of the desired `123abc` and `456def` -- what am I doing wrong? How can I pass variables set during `foo2`'s Build stage to `foo3`?**

Output of `foo3` build:

```
Started by upstream project "foo2" build number 4
originally caused by:
 Started by user unknown or anonymous
Running as SYSTEM
Building remotely on builder-231 (docker-cleanup) in workspace /home/user/workspace/foo3
[foo3] $ /bin/sh -xe /tmp/jenkins2920927890739081404.sh
+ echo ${string1}
${string1}
+ echo ${string2}
${string2}
Finished: SUCCESS

```

---

Screenshots:

`foo2`'s Build and Post-build action to trigger `foo3`:

---

`foo3`'s parameters:

---

`foo3`'s Build

# Answer
Use a Pipeline.

```
pipeline {
  agent any

  stages {

    stage('run job foo3') {
      steps {
        script {

          string1 = "123abc"
          string2 = "456def"

          build job: 'foo3',
            parameters: [
              // both is possible
              string(name: 'a_string', value: "${string1}"),
              string(name: 'b_string', value: string2)
            ]
            wait: true

         }
       }
     }
   }
 }

```
> 0 votes

---
Tags: jenkins, variables
---


title: Jenkins pipeline for restricted repo

text:
# Question
Title: Jenkins pipeline for restricted repo
I have an SVN repo that only a subset of user's are allowed to access. I would like to have Jenkins checkout the repo and build it according the groovy script in the repo. However, when setting this up like my other pipelines BuildSlave fails due to not having permission to checkout the repo. There is an existing set of users with access to the repo but they do not have permission to start JAgent. Is there a way I can set up Jenkins such that there is some way to checkout the repo without unauthorized users being able to see the the pipeline or the SVN?

I have found https://stackoverflow.com/questions/42426804/restricting-others-to-build-the-particular-job-in-jenkins

which pointed me towards https://plugins.jenkins.io/role-strategy/

I think if I created a group in Jenkins that I would be able to define who can see the pipeline but that doesn't solve the SVN access. I am wary of adding BuildSlave to the SVN repo's whitelist. There is already a group account. I have been told adding this group account to Jenkins would allow unauthorized users to see the account. Is there a way to combine these role strategies with the group account to get what I want?

My team is currently thinking of setting up a separate Jenkins server to deal with the restricted project. How does that solution compare to built-in alternatives?

# Answer
In the pipeline post actions you can use this:

```
post {
  always {
    deleteDir() // clean up our workspace
  }
}

```

What way no files in the workspace remain. You have to either copy the file you want to keep or save them as artefacts.
> 1 votes

---
Tags: jenkins, authentication
---


# Question
Title: How to explicitly kill the process with PID 1 from inside a container?
For demonstration purposes, I would like to kill the process with PID 1 from inside a container. But apparently, `kill 1` doesn't work:

```
$ sudo docker run -it centos
[root@3752d3a44f10 /]# ps
  PID TTY          TIME CMD
    1 pts/0    00:00:00 bash
   15 pts/0    00:00:00 ps
[root@3752d3a44f10 /]# kill 1
[root@3752d3a44f10 /]# kill -9 1
[root@3752d3a44f10 /]# kill -SEGV 1
[root@3752d3a44f10 /]# <-- shell is still running

```

Is there a way to explicitly kill the process with PID 1 from inside a container? How?

# Answer
PID 1 is a special PID which will ignore certain signals unless handlers are explicitly created. Docker has some documentation on this:

> Note: A process running as PID 1 inside a container is treated specially by Linux: it ignores any signal with the default action. So, the process will not terminate on SIGINT or SIGTERM unless it is coded to do so.

From here: https://docs.docker.com/engine/reference/run/#foreground

To kill PID 1 you will have to explicitly declare the handler for the SIGTERM signal or, in current versions of Docker, pass the `--init` flag in the `docker run` command to instrument tini.
> 15 votes

# Answer
Also have this problem, solving by use follwing docker-entrypoint.sh

```
#!/bin/sh
set -e
trap "trap - SIGTERM && kill -- -$$" SIGINT SIGTERM EXIT SIGHUP

some_app &
wait

```

You can 'kill 1' somewhere to exit this whole container.
> 2 votes

---
Tags: docker
---


title: EKS NodeAffinity for Services

text:
# Question
Title: EKS NodeAffinity for Services
https://www.eksworkshop.com/beginner/140\_assigning\_pods/affinity/

Links like above usually show node affinity for "kind: Pod".

Isn't it possible to use node affinity for "kind: Service" whereby all pods in a service will have a node affinity?

# Answer
i guess you mean a Deployment because service do not have any pods, and yes, you can use node affinity on a Deployment. you can find it here https://www.howtoforge.com/use-node-affinity-in-kubernetes/
> 1 votes

---
Tags: kubernetes, amazon-web-services, eks
---


# Question
Title: Azure DevOps build pipeline error on copy in post build event
I have built a simple pipeline to build my solution triggered by commits to the master branch using the Azure DevOps Pipelines. I have a problem with the post build copy commands failing and returning errors

**my post build copy command:**

```
copy /Y "$(TargetDir)$(TargetFileName)" "$(MY_PATH)$(TargetFileName)"

```

**I get the following errors:**

```
PostBuildEvent:
  copy /Y "D:\a\1\s\{some_path\file.dll}" "D:\a\1\s\BuildBin\{file.dll}"
The system cannot find the path specified.
              0 file(s) copied.

```

the `MY_PATH` environmental variable is defined in the pipeline variables and using a "CmdLine" `echo %MY_PATH%` in the .yaml it echos the proper path in the console:

```
    Starting: CmdLine
==============================================================================
Task         : Command line
Description  : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
Version      : 2.177.3
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
==============================================================================
Generating script.
Script contents:
echo %MY_PATH%
========================== Starting Command Output ===========================
"C:\windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "D:\a\_temp\1db7603c-e3c0-4e32-924f-c5b1fc2cf96e.cmd""
D:\a\1\s\BuildBin\
Finishing: CmdLine

```

I have defined the `MY_PATH` to be `D:\a\1\s\BuildBin\` as I witnessed that this is the solution folder, the paths in the error are correct.

Also I have tried running a script to create the BuildBin folder as I thought the folder does not exists but its not the reason as I run the following in the .yaml:

```
steps:
- script: echo mkdir D:\a\1\s\BuildBin

```

I cant find any info on the topic any where, just this StackOverflow question and it was not helpful.

I'm running the job in Windows Windows Server 2019 environment. **Needless to say the solution builds on my pc with no problems, it also builds fine in the pipeline, its just the copy that fails.**

# Answer
> 2 votes
I had the same problem and fixed it with this solution:

* for the pipeline yaml I added an additional param, could be any random name, I choose "AzureBuild" and set it to true:

* for the \*.csproj file I added a Condition to Exec with that defined param in mind:

Now this post build events are only fired for local builds and not on Azure DevOps builds.

---
Tags: azure-pipelines
---


title: Kubernetes deploy workflow

text:
# Question
Title: Kubernetes deploy workflow
I have an application (Django web app alongside couple of microservices) and have done deployment previously via Ansible tasks. Playbook was invoked from Ansible Tower and essentially breaks down into following steps

* pull desired docker image
* migrate database
* build js bundle / collect static
* restart application with docker compose

**What is the best practice / preferred solution to have centralized control panel for deploys with k8s?**

Do I need to just rewrite Ansible tasks which will render `kustomization` file with new image tag and the rest will be just invocations of `kubectl`? Appreciate any suggestions or related resources

P.S. this one haven't helped much

# Answer
> 1 votes
You can use kustomization to run kubectl commands/apply files and can achieve the deployment.

However if you really want to get into the k8s ecosystem you can have look into the various tools available for a more controlled deployment workflow.

* **Packaging:-** Manage/package your k8s objects.E.g Helm,jsonnet
* **Helm Repository:-** Centralised helm chart repository. Helm repo is to helm charts what docker is to ECR/GCR/DockerHub. E.g (ChartMuseum)
* **CI:-** Upgrade your CI to publish Helm charts. Apart from this you can run your builds on k8s as well which provide parllelism/scale and cost reduction. E.g Github Action/Jenkins.
* **CD:-** Use A specialized CD tool to manage multi-cluster deployments. FluxCD,ArgoCD,Fleet
* **CI/CD:-** Use a complete delivery tool that fulfills needs of CI/CD. You can use this if you are starting the pipelines from scratch without any existing infra. Also look into the lock-in aspect of using a complete solution. (Might not fulfill all the use cases you may have. Sometimes using bunch of tools is better). E.g JenkinsX/CircleCI
* **Secret Manager:-** You will not want to hard-code senstive/environment related information in your CD pipelines. You can look into Vault,Sealed-Secrets to manage secrets.
* GitOps/Pull/Push formats

This is not an exhaustive list, but will get you started on continuous delivery on k8s.

---
Tags: kubernetes, ansible, deployment, continuous-deployment, awx
---