title: Ansible role depend on other roles

text:
# Question
Title: Ansible role depend on other roles
I have a role, `foo` that installs a file `{{ foo_path }}` that's defined in `roles/defaults/main.yml`.

I have a second role, `bar` that depends on that file being there. I could put `foo` in the `roles/meta/main.yml` `dependencies` but then it will run the entire role which is not ideal. I could potentially tag the task in the role that creates the file, but that's not really scalable, and introduces problems when I'm not the owner of that role's code.

I could just `assert` that the file is there, but the problem is that I don't know where "there" is.

How can I load the `foo_path` var from a different role without the overhead of running the play?

# Answer
> 2 votes
Given the *foo* defaults

```
shell> cat roles/foo/defaults/main.yml
foo_path: /tmp/test.txt

```

There are more options:

1. Include the *foo* defaults into a dictionary and get the variable *foo\_path*

```
shell> cat pb_bar.yml
- hosts: localhost
  pre_tasks:
    - include_vars:
        file: roles/foo/defaults/main.yml
        name: foo_defaults
    - debug:
        var: foo_defaults.foo_path
      when: debug|d(false)|bool
    - set_fact:
        foo_path: "{{ foo_defaults.foo_path }}"
  roles:
    - bar

```

Use the variable in the role *bar*

```
shell> cat roles/bar/tasks/main.yml
- stat:
    path: "{{ foo_path }}"
  register: foo_path_st
- debug:
    var: foo_path_st
  when: debug|d(false)|bool
- assert:
    that: foo_path_st.stat.exists

```

gives

<sup>
```
shell> ansible-playbook pb_bar.yml -e debug=true

PLAY [localhost] *****************************************************************************

TASK [include_vars] **************************************************************************
ok: [localhost]

TASK [debug] *********************************************************************************
ok: [localhost] =>
  foo_defaults.foo_path: /tmp/test.txt

TASK [set_fact] ******************************************************************************
ok: [localhost]

TASK [Assert foo_path] ***********************************************************************
skipping: [localhost]

TASK [bar : stat] ****************************************************************************
ok: [localhost]

TASK [bar : debug] ***************************************************************************
ok: [localhost] =>
  foo_path_st:
    ansible_facts:
      discovered_interpreter_python: /usr/bin/python3
    changed: false
    failed: false
    stat:
      ...
      exists: true
      ...
      path: /tmp/test.txt

TASK [bar : assert] **************************************************************************
ok: [localhost] => changed=false
  msg: All assertions passed

PLAY RECAP ***********************************************************************************
localhost: ok=6    changed=0    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0

```
</sup>

---

2. Structure the roles' tasks

```
shell> tree roles
roles
├── bar
│   └── tasks
│       └── main.yml
└── foo
    ├── defaults
    │   └── main.yml
    └── tasks
        ├── assert_foo_path.yml
        ├── install_foo_path.yml
        └── main.yml

5 directories, 5 files

```

```
shell> cat roles/foo/tasks/assert_foo_path.yml
- stat:
    path: "{{ foo_path }}"
  register: foo_path_st
- debug:
    var: foo_path_st
  when: debug|d(false)|bool
- assert:
    that: foo_path_st.stat.exists

shell> cat roles/foo/tasks/install_foo_path.yml
- file:
    path: "{{ foo_path }}"
    state: touch

shell> cat roles/foo/tasks/main.yml
- name: Install foo_path
  include_tasks: install_foo_path.yml
- name: Assert foo_path
  include_tasks: assert_foo_path.yml

```

and use the module *include\_role*

```
shell> cat roles/bar/tasks/main.yml
- name: Assert foo_path
  include_role:
    name: foo
    tasks_from: assert_foo_path.yml

```

Create playbooks for testing

```
shell> cat pb_foo.yml
- hosts: localhost
  roles:
    - foo

shell> cat pb_bar.yml
- hosts: localhost
  roles:
    - bar

```

The play will fail if the file is missing

<sup>
```
shell> ansible-playbook pb_bar.yml -e debug=false

PLAY [localhost] *****************************************************************************

TASK [Assert foo_path] ***********************************************************************

TASK [foo : stat] ****************************************************************************
ok: [localhost]

TASK [foo : debug] ***************************************************************************
skipping: [localhost]

TASK [foo : assert] **************************************************************************
fatal: [localhost]: FAILED! => changed=false
  assertion: foo_path_st.stat.exists
  evaluated_to: false
  msg: Assertion failed

PLAY RECAP ***********************************************************************************
localhost: ok=1    changed=0    unreachable=0    failed=1    skipped=1    rescued=0    ignored=0

```
</sup>

Create the file

<sup>
```
shell> ansible-playbook pb_foo.yml -e debug=false

PLAY [localhost] *****************************************************************************

TASK [foo : Install foo_path] ****************************************************************
included: /export/scratch/tmp7/test-321/roles/foo/tasks/install_foo_path.yml for localhost

TASK [foo : file] ****************************************************************************
changed: [localhost]

TASK [foo : Assert foo_path] *****************************************************************
included: /export/scratch/tmp7/test-321/roles/foo/tasks/assert_foo_path.yml for localhost

TASK [foo : stat] ****************************************************************************
ok: [localhost]

TASK [foo : debug] ***************************************************************************
skipping: [localhost]

TASK [foo : assert] **************************************************************************
ok: [localhost] => changed=false
  msg: All assertions passed

PLAY RECAP ***********************************************************************************
localhost: ok=5    changed=1    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0

```
</sup>

The assert task will succeed now

```
TASK [foo : assert] **************************************************************************
ok: [localhost] => changed=false
  msg: All assertions passed

```

---
Tags: ansible
---


title: Two server failover in nomad

text:
# Question
Title: Two server failover in nomad
I am just learning about orchestrators and it's my first time using any orchestrator (`Nomad` in this case).

Our software (video stream processing related) is installed on user premises, currently we have been using `docker compose` to deploy our software (bunch of services as docker container).

Now we have a client who needs failover and has provided us two physical server, one is to be used as a main server and other as failover. What we'd like to do is have our services installed on both the servers but only run these services on failover when the main server goes down.

We won't like to make any changes to our code. we have one services which process the video streams and raise events (`VA service`) and another service which handles database, users, events etc. (`Client Service`). The `client service` knows about this `VA service` and it assigns work to it. *We don't want these services to be running on failover server when the main server is up since it will cause streams to be processed twice i.e produce duplicate events.*

We are configuring our database in master-master mode so that data is available on failover when main fails ( trying it on postgres (current database), had previously done it on Mysql ).

I have set up nomad on both machines with nomad server and client both on each machine and tried configuring them such that if the machine with nomad leader server and its client shuts down, nomad server on failover should take charge and run tasks on its client, but i am unable to achieve this. If i shutdown the main server nothing happens on failover machine.

```
# Leader config file
datacenter = "dc1"
data_dir   = "/opt/nomad/data"
bind_addr  = "192.168.1.38"
name = "server1"

server {
  enabled          = true
  bootstrap_expect = 2
  server_join {
    retry_join = [
      "192.168.1.38",
      "192.168.1.20",
    ]
    retry_max      = 0
    retry_interval = "15s"
  }
}

client {
  enabled = true
  servers = [
    "192.168.1.38",
    "192.168.1.20",
  ]
}

# Follower config file
datacenter = "dc1"
data_dir   = "/opt/nomad/data"
bind_addr  = "192.168.1.20"
name = "server2"

server {
  enabled          = true
  bootstrap_expect = 2
  server_join {
    retry_join = [
      "192.168.1.38",
      "192.168.1.20",
    ]
    retry_max      = 0
    retry_interval = "15s"
  }
}

client {
  enabled = true
  servers = [
    "192.168.1.38",
    "192.168.1.20",
  ]
}

```
1. Is nomad even the right tool for this situation ?
2. How can i achieve what i am trying using nomad ?

# Answer
I think you are conflating a few concepts here, let me try to separate them in your mind.

First, separate availability of the platform, from the availability of your applications:

Nomad, like many other distributed systems, uses the raft consensus algorithm to determine who is the leader of the server cluster. This is different to older high-availability strategies which provide multiple endpoints served by a load balancer for example.

If you have two servers, it's impossible to elect a leader since none of the servers has a winning vote. For this reason, there is zero fault tolerance with two servers -- see https://developer.hashicorp.com/nomad/docs/concepts/consensus#deployment-table. If the active server goes down, requests cannot be processed.

So, if you want **fault tolerance**, put another Nomad server in the cluster, and bring it to 3. Else having 1 or 2 servers won't increase fault tolerance. The state is replicated across servers, so you be able to perform read actions, but you won't be able to change the state until a quorum is re-established and a new leader is elected.

---

Now, for you application:

You have Nomad clients defined on two machines, which means that you can submit jobs with *services* and *tasks* to those resources.

> What we'd like to do is have our services installed on both the servers but only run these services on failover when the main server goes down.

What you are looking for here is the `migrate` stanza of the job - when Nomad determines that a node on which a job is running has failed, it (Nomad server) will migrate the job to a node which is up. However, as discussed before, if the *server* is unavailable because it has lost consensus, this won't be possible... So either you have an extremely reliable single machine running Nomad server, or you have a fault-tolerant 3-node cluster running Nomad server, with unreliable clients.

In terms of your VAService and client jobs -- you should write a Nomad job description which defines these tasks. They can expose services and Nomad's service discovery will connect them, so that they can run on different machines, *etc*.

> We don't want these services to be running on failover server when the main server is up since it will cause streams to be processed twice i.e produce duplicate events.

Nomad is the orchestrator, therefore it determines where to place the job. Unless you have a `count` in the group which the task is running in greater than 1 (ie, more than one instance of the application placed in a group), then Nomad will schedule only one instance of it and you won't have to worry about double-processing streams. In terms of availability , you should choose a different strategy than the traditional HA passive-active :

* `reschedule` -- reschedule the group or job if a task fails or dies
* `restart` -- similar to reschedule, but just restarting in place -- see also `check_restart`
* `migrate` -- move the job to a different node if the client is marked for draining

These are definitions of how Nomad should treat failures in the operational lifecycle of the application.

---

Take a scenario where you have one server and clients enabled on it and the other machine. You write a Nomad job description with:

1. 2 groups -- one for the server, one for the client
2. each group has a `reschedule`, `restart` and `check_restart` stanza to deal with failures.

You deploy this job description to Nomad. The *application server* group gets placed on the same node where the Nomad server is running and the application client gets placed on the other machine. Everything works fine.

Now:

Failure scenario 1: the machine running only the client dies

* Nomad will reschedule see that the node is down and reschedule the group on the remaining machine.
* you have resilience, but there is a disruption in availability.

Failure scenario 2: the machine running the Nomad server dies

* Nomad cannot reschedule the job because quorum has been lost
* Application server is lost, but application client keeps running

Clearly neither of these is a good situation, so:

I would go with a resilience scenario such as: Use Nomad servers in a different failure domain to the Nomad clients.

If you are able to add a third node to the cluster, you will have fault tolerance at the cluster level. By using the built-in mechanisms of Nomad for rescheduling, restarting, *etc* job as discussed above, you will have a resilient application. You cannot eliminate downtime however without changing the application itself, either by making the server idempotent (ie, multiple processing requests against the same event do not change the event result), or by including some mechanism for the server itself to elect a leader of that tier of the application. There are several approaches here, but I think they'd fall outside of the scope of the question.

Finally, I would suggest opening this topic at the Hashicorp discussion forum so that you can get some deeper context and better feedback :)
> 2 votes

---
Tags: orchestration
---


# Question
Title: What is the point of starting pipeline when MR creating?
Can you explain what the point of starting pipeline for source branch again when I do MR in default branch? I mean, when I did git push before in my feature branch which I want to merge with master, pipeline was successful in that case. Why should we start it again in MR? Is it correctly if I configure except MR for stages in pipeline?

# Answer
> 0 votes
Running CI pipelines in branches is a foundational practice in continuous integration -- Check whether the application can build at every change.

By running a pipeline when a Merge Request is opened, you are preparing for the actual merge event, which is precisely what you aim to achieve by opening the Merge Request in the first place. Correct me if I'm wrong, but your question is related to the merge commit not the merge request, right?

What you are asking however seems to be why that same change, once merged, needs to run again through the same pipeline. The answer lies in whether the branch pipeline and the mainline pipeline are identical. If the exact same tests and tasks are run in the branch pipeline as in the mainline pipeline once merged, I can understand your perplexity.

However, there are very often extra or different stages which get run in a mainline pipeline. For example, a branch pipeline might perform continuous delivery to a testing or staging environment, while the mainline pipeline might deploy to the production environment.

Different secrets may be consumed, changes in the production environment state may have occured, or simply a release may need to be cut and delivered to the artifactory, *etc*, *etc*. Delivering and testing changes is very different to actually deploying them into a user-facing environment.

If it seems like a trivial duplication of work, this may be a signal that the pipelines need to be re-looked at and refined. At the very least, you can agree that every change to the mainline branch should be tested, therefore every commit should be tested, therefore also merge commits should be tested.

---
Tags: gitlab, cicd
---


# Question
Title: View all AWS S3 buckets and list each buckets storage used
I have multiple **AWS** accounts and I need to list all **S3** buckets per account and then view each buckets total size.

Currently, I can only view the storage size of a single **S3** bucket with:

```
aws s3 ls s3://mybucket --recursive --human-readable --summarize

```

# Answer
> 20 votes
**Resolution 1**

So I solved this with the following script. I originally posted the question just in case there was an easier way that I was not aware of.

```
#!/bin/bash
aws_profile=('profile1' 'profile2' 'profile3');

#loop AWS profiles
for i in "${aws_profile[@]}"; do
  echo "${i}"
  buckets=($(aws s3 ls s3:// --recursive --profile "${i}" --region your_region | awk '{print $3}'))

  #loop S3 buckets
  for j in "${buckets[@]}"; do
  echo "${j}"
  aws s3 ls s3://"${j}" --recursive --human-readable --summarize --profile "${i}" --region your_region | awk END'{print}'
  done

done

```

**Resolution 2**

Using **Dashboards** in **CloudWatch** in the AWS console.

You can then simply specify all S3 buckets and add the numbers stats to show the storage size metrics.

This won't cost you plenty of API calls and can be significantly faster depending on the size of the s3 buckets(takes quite awhile to get the size on very large buckets).

**Verdict**

Creating the Dashboard (**Resolution 2**) on each AWS account was the most efficient option **for me** cause it is way quicker for me to log in and grab the metrics manually from each AWS account than to wait for the scripts API calls to finish. :(

# Answer
> 5 votes
You will need to write a script that parses and queries this data because as far as I'm aware, there is not a tool or cli function that performs this. Luckily, you can gather all of this information with the CLI.

1. List and parse all of the accounts in your org.

```
aws organizations list-accounts

```
2. For each account, list and parse all of the buckets.

```
aws s3api list-buckets --query "Buckets[].Name"

```
3. Finally, get the size of each bucket within each account. You can use the same cli command you were before, but be warned that you are going to be listing the individual size of each item within the bucket. You can also use this cli command to get bucket size.

```
aws s3api list-objects --bucket BUCKETNAME --output json --query "
[sum(Contents[].Size), length(Contents[])]"

```

# Answer
> 3 votes
Easiest way to see this information nowadays is using S3 Storage Lens.

S3 dashboard \> S3 storage lens (left navigation pane) \> default-account-dashboard \> Buckets tab

Scroll down to the "Buckets" section and sort by total storage.

# Answer
> 2 votes
You're going to have to write a script to do this unless there's a tool to do it that I can't find.

If you're working with multiple aws accounts you'll need to be switching between accounts to get the bucket sizes.

To get all of the buckets and their total size you can use 'aws s3api'. Using `list-buckets` you can get all of the buckets in one account and then you'll be able to list their sizes and try the next account.

# Answer
> 1 votes
None of these show the size of the bucket if Versioning is turned on and massive amounts of data have been "deleted". In the S3 GUI console, the Metrics tab shows what you are being billed for.

Scraped the following command out of a script found at https://serverfault.com/questions/84815/how-can-i-get-the-size-of-an-amazon-s3-bucket . It works well on Windows. Substitute `yourregion` and `yourbucketname`:

`aws cloudwatch get-metric-statistics --namespace AWS/S3 --start-time 2019-05-09T00:01:00Z --end-time 2019-05-11T23:59:00Z --period 86400 --statistics Average --region yourregion --metric-name BucketSizeBytes --dimensions Name=BucketName,Value=yourbucketname Name=StorageType,Value=StandardStorage`

This gives the average size in bytes over the days bounded by the start and end times. **The result can shock you** if your bucket has Versioning Enabled (active) *and* you think you have "deleted" massive amounts of data.

To clean up Versioning-enabled buckets, you must set a LifeCycle rule. You can choose to permanently delete Previous Versions. You may omit the checkboxes at the Transitions step.

# Answer
> 0 votes
I just had to go through this and wrote/uploaded a script for it in GitHub. https://github.com/johnwhumphreys/s3-buckets-by-size/blob/main/get-bucket-sizes.py

It's a single python script that will quickly grab the cloudwatch metrics for the sizes for your buckets and rank them by order, showing the top 10. Runs in a minute or so on our 200 bucket account.

(no way I could have listed the files we had in any reasonable time frame, so had to find a faster way)

This also deals w/ **Intelligent Tiering** as of 2023.

---
Tags: amazon-s3, aws-cli
---


# Question
Title: Jenkins try-catch the actual error instead of hudson.AbortException
how do I get the actual exception of a failed pipeline step/plugin? I have the following pipeline code

```
try {
    checkout(
        scm: [ ... ]
    )
} catch (error) {
    echo error.getClass()
    echo error.getMessage()
}

```

which always results in

```
class hudson.AbortException
Error fetching remote repo 'origin'

```

But running without try/catch it would result in:

```
hudson.plugins.git.GitException: Failed to fetch from https://git.repo.git
    at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:1003)
    at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1245)
    at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1309)
    at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:129)
    at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:97)
    at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:84)
    at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --force --progress --prune --depth=1 -- https://git.repo.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout:
stderr: fatal: Unable to create '/var/jenkins_home/workspace/some_space/.git/shallow.lock': File exists.

Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.

    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2734)
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2111)
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:623)
    at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:158)
    at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:151)
    at hudson.remoting.UserRequest.perform(UserRequest.java:211)
    at hudson.remoting.UserRequest.perform(UserRequest.java:54)
    at hudson.remoting.Request$2.run(Request.java:377)
    at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
    ... 4 more
    Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to slave-lpdkwpci13vm
        at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1784)
        at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
        at hudson.remoting.Channel.call(Channel.java:1000)
        at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:143)
        at jdk.internal.reflect.GeneratedMethodAccessor1484.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:129)
        at com.sun.proxy.$Proxy209.execute(Unknown Source)
        at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:1001)
        at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1245)
        at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1309)
        at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:129)
        at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:97)
        at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:84)
        at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        ... 4 more

```

so the real exception is:

```
hudson.plugins.git.GitException: Command "git fetch --tags --force --progress --prune --depth=1 -- https://git.repo.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout:
stderr: fatal: Unable to create '/var/jenkins_home/workspace/some_space/.git/shallow.lock': File exists.

```

**Is there a possibility to catch a `GitException` instead of `hudson.AbortException` or get the message from `stderr`.**

I already tried things like `error.getCause()`

# Answer
> 1 votes
What you are referring to here is exception chaining. I don't think you can catch a specific exception in the chain. You can walk the chain to see all the exceptions. To get the last exception, the `GitException`, you can use this code.

# Answer
> 0 votes
Pipelines use regular Groovy, so you would use the normal Groovy approach for this:

```
try {
   checkout()
} catch(hudson.plugins.git.GitException e) {
   // handle a GitException
} catch(hudson.AbortException e) {
   // handle an AbortException
} finally {
   // cleanup
}

```

---
Tags: jenkins, jenkins-pipeline, jenkins-plugins
---


# Question
Title: Way to detect base container version for pods
I am trying to understand how companies detect that a Kuberentes pod is running a container that is based on a container image that is out of date.

So, for example, say I have an application called WidgetOrdering. In my `dockerfile` I setup my WidgetOrdering container, first by having a `from` clause that that references the .Net 6 runtime container, then I add in additional instructions specific to my application. Then I deploy my WidgetOrdering container to Kubernetes.

After it has run for several months, .Net 6 releases a new version of their runtime container with some security patches in it. I need to know what versions of the .Net 6 container I have running in my cluster so I can target them for updating.

How do people check Kubernetes to see what "base" containers are running in their cluster?

# Answer
> 1 votes
Typically you don't do this in Kubernetes. Instead you do this in your CI pipeline or by scanning images on the registry. In the CI pipeline, the easy option is to periodically rebuild and redeploy the image on a schedule. This not only picks up changes in the base image, but any other external dependencies that may have changed since the previous build.

To check an image in the registry, this requires metadata like annotations to be defined on the image. OCI has documented standard annotations that can be used specifically for this:

> * `org.opencontainers.image.base.digest`: Digest of the image this image is based on (string)
> * `org.opencontainers.image.base.name`: Image reference of the image this image is based on (string)

As an example, using one of my own projects:

```
$ regctl manifest get ghcr.io/regclient/regctl:alpine --format '{{jsonPretty .Annotations}}'
{
  "oci.opencontainers.image.base.digest": "sha256:124c7d2707904eea7431fffe91522a01e5a861a624ee31d03372cc1d138a3126",
  "oci.opencontainers.image.base.name": "alpine:3",
  "oci.opencontainers.image.created": "2023-04-14T23:41:20Z",
  "oci.opencontainers.image.revision": "847254c7ac7d6f027dcdfb196a9aa4c11eb61ed9",
  "oci.opencontainers.image.source": "https://github.com/regclient/regclient.git"
}

$ regctl image digest alpine:3
sha256:124c7d2707904eea7431fffe91522a01e5a861a624ee31d03372cc1d138a3126

```

You can see the base name `alpine:3` has the same digest from when this image was last built. Note that this metadata is not automatically added by most image build tooling. The steps used to add this in my own project can be seen in the github actions.

---
Tags: docker, kubernetes, containers
---


# Question
Title: Is there a way to migrate from HCL with the Terraform provider to YAML?
Currently I have a deployment of an applicationn done with HCL in Terraform. Is it possible to have the terraform provider for Kubernetes output to a directory or in some form the YAML that it is sending to kubernetes?

# Answer
I don't know of a way to get Terraform to output to YAML but if you have access to the k8 cluster you can easily get the YAML for a deployment/pod. See this existing SO answer.
> 0 votes

# Answer
> Is it possible to have the terraform provider for Kubernetes output to a directory or in some form the YAML that it is sending to kubernetes?

You can do this with a combination of a file provider and the `yamlencode()` function.

Let's say you have a `kubernetes_deployment_v1` and you want to write the yaml description of it to a file:

```
resource "kubernetes_deployment_v1" "this" {
  ...
}

resource "local_file" "that" {
   content = yamlencode(kubernetes_deployment_v1.this.spec)
   filename = "${path.module}/deployment_spec.yaml"
}

```
> 0 votes

---
Tags: kubernetes, terraform
---


# Question
Title: Install OpenSSL into Red Hat image lacking yum
I have a container (running image `quay.io/keycloak/keycloak:21.1.1`) where `cat /etc/os-release` reports the following (excerpt):

```
NAME="Red Hat Enterprise Linux"
VERSION="9.1 (Plow)"
CPE_NAME="cpe:/o:redhat:enterprise_linux:9::baseos"

```

`openssl` is not installed, neither is `yum`. (`yum install openssl` fails with "command not found".) So how can I install OpenSSL into such an image (perhaps it is a UBI micro-image)?

# Answer
The Keycloak image `quay.io/keycloak/keycloak:21.1.1` is currently based on the Red Hat Universal Base Image `ubi9-micro`, which lacks any package manager (`dnf`, `rpm`, `yum`) in order to minimize size.

I came across a couple of tricks for installing packages regardless, e.g. one involving `buildah mount` and another one involving multi-stage build plus `yum install --installroot /mnt/rootfs`. The second now serves well in my scenario.
> 0 votes

---
Tags: docker, oci
---


# Question
Title: Docker push getting error EOF to private repository
I'm trying to push images to repo private, but it's getting error below. Before, it was working, but idk what happened...

```
PS C:\Users\user\aplication> docker push registry.apprepo.app/app:1.22.3
The push refers to repository [registry.apprepo.app/app]
f4ca70ea751f: Layer already exists
05e4cd1b24c1: Pushed
1ed92f7e8e52: Pushed
f9cb3f1f1d3d: Layer already exists
f0fb842dea41: Layer already exists
c1cd5c8c68ef: Layer already exists
1d54586a1706: Layer already exists
1003ff723696: Layer already exists
f1417ff83b31: Layer already exists
Patch "http://registry.apprepo.app/v2/app/blobs/uploads/f3666007-df22-456e-b15d-094d0317e166": EOF

```

Dockerfile

```
FROM node:14 as build-stage

WORKDIR /app

COPY . .

ARG target_env

RUN npm install -g gulp-cli \
    && npm install \
    && npm run build:$target_env

FROM nginx:1.23.4-alpine-slim as deploy-stage

COPY --from=build-stage /app/dist/ /usr/share/nginx/html

COPY --from=build-stage /app/nginx/default.conf /etc/nginx/conf.d/default.conf

COPY --from=build-stage /app/scripts/config.env.sh ./config.env.sh

CMD  chmod +x config.env.sh && ./config.env.sh

```

Docker Compose

```
version: '3'

services:

  app:
    image: registry.apprepo.app/app:${APP_VERSION}
    build:
      context: ./
      args:
        target_env: docker
    env_file:
      - docker/.env
    ports:
      - "8081:80"

```

I've try:

```
docker builder prune -af

```

but it not pushing yet. I'll appreciate for your help!

# Answer
I found out where the problem might be, actually the problem is in NGINX, using the alpine image.

I decided to move the image to Ubuntu, using NGINX 18.04. That worked.
> 0 votes

---
Tags: docker, docker-compose, dockerfile
---


# Question
Title: Gitlab Runner setup issues on Mac Mini with M2 processor
I have a pair of Mac Minis with M2 processors that I am trying to setup as runners on a locally hosted gitlab setup. For a variety of other systems I simply followed the instructions on the runners page. When I select Mac and Arm64 here are the instructions I see:

> Download and install binary

```
# Download the binary for your system
sudo curl --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-darwin-arm64`

# Give it permission to execute
sudo chmod +x /usr/local/bin/gitlab-runner

# The rest of the commands execute as the user who will run the runner
# Register the runner (steps below), then run
cd ~
gitlab-runner install
gitlab-runner start

```

> Command to register runner

```
gitlab-runner register --url https://gitlab.blacktoppstudios.com/ --registration-token Fj3n5ngFakeToken

```

I needed to create the output folder for the download from AWS, but that seemed to work. But when I try to do `gitlab-runner start` I get the following error:

> FATAL: Failed to start gitlab-runner: exit status 134

I couldn't see any results for exist status 134. A number of people had issues with the log files not being writable. I tried setting permissions on the log files and get the same error. If I change the log file location there is no apparent effect also.

I tried to perform the registration step and this appeared to work and create the toml file but didn't change the error message.

If I copy the command out of the file plist file at `~/Library/LaunchAgents/gitlab-runner.plist` and add `--debug` I get this command:

> % /usr/local/bin/gitlab-runner --debug run --working-directory /Users/cisadmin --config /Users/cisadmin/.gitlab-runner/config.toml --service gitlab-runner --syslog

and that gives me this:

```
Runtime platform                                    arch=arm64 os=darwin pid=2562 revision=436955cb version=15.11.0
Starting multi-runner from /Users/cisadmin/.gitlab-runner/config.toml...  builds=0
Checking runtime mode                               GOOS=darwin uid=501
WARNING: Running in user-mode.
WARNING: Use sudo for system-mode:
WARNING: $ sudo gitlab-runner...

Configuration loaded                                builds=0
listenaddress: ""
sessionserver:
  listenaddress: ""
  advertiseaddress: ""
  sessiontimeout: 1800
concurrent: 1
checkinterval: 0
loglevel: null
logformat: null
user: ""
runners: []
sentrydsn: null
modtime: 2023-05-09T16:43:47.632488987-05:00
loaded: true
shutdowntimeout: 0
  builds=0
Waiting for stop signal                             builds=0
listen_address not defined, metrics & debug endpoints disabled  builds=0
[session_server].listen_address not defined, session endpoints disabled  builds=0
Initializing executor providers                     builds=0
Feeding runners to channel                          builds=0
Starting worker                                     builds=0 worker=0
Feeding runners to channel                          builds=0

```

The `listen_address not defined` has me concerned but I have no further insight into what this means or how to troubleshoot it. I would appreciate any insight on how to approach `Failed to start gitlab-runner: exit status 134`.

EDIT - Some Docs suggested that local terminal have different permissions somehow than ssh connections. Well I tried following the steps from a local prompt and rebooted and now `gitlab-runner status` says that it is running, but it still isn't showing as connected from the runners listing on the gitlab web ui.

I looked into the config.toml and it is missing the \[\[runners\]\] section. I tried creating one by copying and adapting from a

# Answer
There were several errors.

1. The log files weren't writable. Either set permissions or change to some writable place. I set it `/Users/$USERNAME/.gitlab-runner/gitlab-runner.{out,err}.log`
2. Ran the bootstrap commands the local terminal suggested that ssh never presented even with identical commands. (I hate this, I have no clue why it is different, but it is).
3. Copied the \[\[runners\]\] section from the `/etc/gitlab-runner/config.toml` to `/Users/$USERNAME/.gitlab-runner/config.toml`.

After that I issude `gitlab-runner stop` and `gitlab-runner start` and checked the GitLab web UI and the runners were connected.
> 1 votes

---
Tags: gitlab, gitlab-ci-runner, macos
---


title: What regex engine does GitLab use?

text:
# Question
Title: What regex engine does GitLab use?
Let's say I have a rule like this,

```
- if: '$CI_COMMIT_TITLE =~ /^(fix|feat|perf|docs|build|test|ci|refactor)\S*:/'

```

It occurs to me that `^` does not match the start of any line in a multiline regex, only the start of the first line. That raises the question,

* Does anything match the newline?
* Is there a multi-line regex mode?

Where is the GitLab Regex documented? What Regex implementation do they use?

# Answer
> 1 votes
Updated answer, according to these docs:

> In GitLab 11.9.4, GitLab began internally converting the regexp used in only and except keywords to RE2.
>
> RE2 limits the set of available features due to computational complexity, and some features, like negative lookaheads, became unavailable. Only a subset of features provided by Ruby Regexp are now supported.
>
> From GitLab 11.9.7 to GitLab 14.9, GitLab provided a feature flag to let you use unsafe regexp syntax. We’ve fully migrated to RE2 now, and that feature flag is no longer available.

The same is true for `rules` and `if`, like you are using.

TL;DR: GitLab regexps now use Google's RE2 syntax, not Ruby.

# Answer
> 1 votes
GitLab uses Ruby's core regex implementation which is Onigmo (Oniguruma-mod). I found that from looking at the source code in Ruby's `regex.c`.

Official documentation is mostly here: https://github.com/k-takata/Onigmo/blob/master/doc/RE

---
Tags: gitlab, gitlab-ci-runner
---


# Question
Title: How can we get the jenkins console output in a text file?
How can i get the Jenkins console output in a text file? I want to share it with someone, is there any way to do it?

# Answer
> 41 votes
if you want just access the log and download it as a txt file to your workspace from the job's URL:

```
${BUILD_URL}/consoleText

```

On Linux, you can use `wget` to download it to your workspace

```
wget ${BUILD_URL}/consoleText

```

The actual log file on the file system is in the Master machine. You can find it under:

`$JENKINS_HOME/jobs/$JOB_NAME/builds/lastSuccessfulBuild/log`

# Answer
> 4 votes
For Windows you could use `curl` in a powershell prompt:

```
curl  ${BUILD_URL}\consoleText -OutFile C:\SomeLocation\SomeFile.txt

```

For MacOS:

```
curl  ${BUILD_URL}/consoleText -o /SomeLocation/SomeFile.txt

```

# Answer
> 1 votes
To get the Jenkins console log without curl or wget:

```
writeFile file: "jenkins_console_output.txt", text: currentBuild.rawBuild.logFile.text
sh 'sed -ri "s/\\x1b\\[8m.*?\\x1b\\[0m//g" jenkins_console_output.txt'

```

Explanation:

* `currentBuild.rawBuild.logFile.text` will return annotated log.
* `\x1b\[8m.*?\x1b\[0m` is a regex expression to match AnsiColorer
* `sed -ri` use regexp-extended and in-place to update the file

---
Tags: jenkins
---


# Question
Title: kube-prometheus-stack/grafana - Getting "too many redirects" error
I have a Kubernetes cluster with the `kube-prometheus-stack` Helm chart (https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack) installed. NGINX is used as the reverse proxy. We are using Kubernetes 1.24.

I am running into a problem when I attempt to access the Grafana dashboard. I am getting the "too many redirects" error. It can even be seen in the browser URL:

I have tried to find what could be causing this and a solution to correct the problem. I found some info at the following site that seems to be very much related, but from what I can tell, what is discussed is specific to a more direct installation, rather than within the context of Kubernetes, Helm charts and container abstractions.

https://github.com/grafana/grafana/blob/main/docs/sources/tutorials/run-grafana-behind-a-proxy/index.md

and

https://grafana.com/tutorials/run-grafana-behind-a-proxy/

I think I can see a parallel between some of what's mentioned in the above articles (they have a lot of the same information) and my Helm configuration (`values.yaml`). However, I have no idea what is being referred to regarding NGINX configuration. I'm not sure where to look to see the NGINX config or how to modify it. There is mention of setting up an NGINX rewrite directive, but I have no idea what that is, how it can be done, or if I even need it.

Here is my Helm chart `values.yaml` if it's of any help.

```
kube-prometheus-stack:
  defaultRules:
    create: true
    rules:
      alertmanager: true
      etcd: true
      configReloaders: true
      general: true
      k8s: true
      kubeApiserverAvailability: true
      kubeApiserverBurnrate: true
      kubeApiserverHistogram: true
      kubeApiserverSlos: true
      kubeControllerManager: true
      kubelet: true
      kubeProxy: true
      kubePrometheusGeneral: true
      kubePrometheusNodeRecording: true
      kubernetesApps: true
      kubernetesResources: true
      kubernetesStorage: true
      kubernetesSystem: true
      kubeSchedulerAlerting: true
      kubeSchedulerRecording: true
      kubeStateMetrics: true
      network: true
      node: true
      nodeExporterAlerting: true
      nodeExporterRecording: true
      prometheus: true
      prometheusOperator: true

  prometheus:
    enabled: true
    ingress:
      enabled: true
      ingressClassName: nginx
      hosts:
        - prometheus.nnpi.beast-code.com
      paths:
        - /
      pathType: ImplementationSpecific

  grafana:
    enabled: true
    ingress:
      enabled: true
      ingressClassName: nginx
      hosts:
        - grafana.nnpi.beast-code.com
      path: /
    persistence:
      enabled: true
      # size: 10Gi
    grafana.ini:
      server:
        root_url: grafana.nnpi.beast-code.com

  alertmanager:
    enabled: true
    ingress:
      enabled: true
      ingressClassName: nginx
      hosts:
        - alerts.nnpi.beast-code.com
      paths:
        - /
      pathType: ImplementationSpecific
    config:
      global:
        slack_api_url: <path:arc/data/alertmanager#SLACK_API_URL>
      route:
        receiver: "slack-default"
        group_by:
          - alertname
          - cluster
          - service
        group_wait: 30s # 30s
        group_interval: 5m # 5m
        repeat_interval: 1h # 4h
        routes:
          - receiver: "null"
            matchers:
              - alertname =~ "InfoInhibitor|Watchdog"
          - receiver: "slack-warn-critical"
            matchers:
              - severity =~ "warning|critical"
            continue: true
      receivers:
        - name: "null"
        - name: "slack-default"
          slack_configs:
            - send_resolved: true # false
              channel: "#notify-arc-alerts"
              text: |-
                {{range .Alerts}}
                Alert: {{.Labels.alertname}}, Severity: {{.Labels.severity}}
                Summary: {{.Annotations.summary}}
                Description: {{.Annotations.description}}
                {{end}}
        - name: "slack-warn-critical"
          slack_configs:
            - send_resolved: true # false
              channel: "#notify-arc-alerts"
              text: |-
                {{range .Alerts}}
                Alert: {{.Labels.alertname}}, Severity: {{.Labels.severity}}
                Summary: {{.Annotations.summary}}
                Description: {{.Annotations.description}}
                {{end}}

```

Here is the generated ingress:

```
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: prometheus-grafana
  namespace: prometheus
  uid: df852ca5-897e-4d5c-bd6b-09b4a4a45910
  resourceVersion: '111566747'
  generation: 1
  creationTimestamp: '2022-11-09T19:27:17Z'
  labels:
    app.kubernetes.io/instance: prometheus
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: grafana
    app.kubernetes.io/version: 9.3.1
    helm.sh/chart: grafana-6.48.2
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: >
      {"apiVersion":"networking.k8s.io/v1","kind":"Ingress","metadata":{"annotations":{},"labels":{"app.kubernetes.io/instance":"prometheus","app.kubernetes.io/managed-by":"Helm","app.kubernetes.io/name":"grafana","app.kubernetes.io/version":"9.3.1","helm.sh/chart":"grafana-6.48.2"},"name":"prometheus-grafana","namespace":"prometheus"},"spec":{"ingressClassName":"nginx","rules":[{"host":"grafana.nnpi.beast-code.com","http":{"paths":[{"backend":{"service":{"name":"prometheus-grafana","port":{"number":80}}},"path":"/","pathType":"Prefix"}]}}]}}
  managedFields:
    - manager: argocd-application-controller
      operation: Update
      apiVersion: networking.k8s.io/v1
      time: '2022-11-09T19:27:17Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:metadata:
          f:annotations:
            .: {}
            f:kubectl.kubernetes.io/last-applied-configuration: {}
          f:labels:
            .: {}
            f:app.kubernetes.io/instance: {}
            f:app.kubernetes.io/managed-by: {}
            f:app.kubernetes.io/name: {}
            f:app.kubernetes.io/version: {}
            f:helm.sh/chart: {}
        f:spec:
          f:ingressClassName: {}
          f:rules: {}
    - manager: nginx-ingress-controller
      operation: Update
      apiVersion: networking.k8s.io/v1
      time: '2022-11-09T19:27:56Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:status:
          f:loadBalancer:
            f:ingress: {}
      subresource: status
  selfLink: >-
    /apis/networking.k8s.io/v1/namespaces/prometheus/ingresses/prometheus-grafana
status:
  loadBalancer:
    ingress:
      - ip: 10.107.160.141
spec:
  ingressClassName: nginx
  rules:
    - host: grafana.nnpi.beast-code.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: prometheus-grafana
                port:
                  number: 80

```

I have also included a section of the `nginx.conf` file from one of the nginx controller pods running in the cluster. I do not understand this at all, but I'm guessing that a combination of ArgoCD, Helm and Kubernetes is taking care of setting this up.

```
## start server grafana.nnpi.beast-code.com
server {
        server_name grafana.nnpi.beast-code.com ;

        listen 80  ;
        listen [::]:80  ;
        listen 443  ssl http2 ;
        listen [::]:443  ssl http2 ;

        set $proxy_upstream_name "-";

        ssl_certificate_by_lua_block {
                certificate.call()
        }

        location / {

                set $namespace      "prometheus";
                set $ingress_name   "prometheus-grafana";
                set $service_name   "prometheus-grafana";
                set $service_port   "80";
                set $location_path  "/";
                set $global_rate_limit_exceeding n;

                rewrite_by_lua_block {
                        lua_ingress.rewrite({
                                force_ssl_redirect = true,
                                ssl_redirect = true,
                                force_no_ssl_redirect = false,
                                preserve_trailing_slash = false,
                                use_port_in_redirects = false,
                                global_throttle = { namespace = "", limit = 0, window_size = 0, key = { }, ignored_cidrs = { } },
                        })
                        balancer.rewrite()
                        plugins.run()
                }

                # be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any
                # will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)`
                # other authentication method such as basic auth or external auth useless - all requests will be allowed.
                #access_by_lua_block {
                #}

                header_filter_by_lua_block {
                        lua_ingress.header()
                        plugins.run()
                }

                body_filter_by_lua_block {
                        plugins.run()
                }

                log_by_lua_block {
                        balancer.log()

                        monitor.call()

                        plugins.run()
                }

                port_in_redirect off;

                set $balancer_ewma_score -1;
                set $proxy_upstream_name "prometheus-prometheus-grafana-80";
                set $proxy_host          $proxy_upstream_name;
                set $pass_access_scheme  $scheme;

                set $pass_server_port    $server_port;

                set $best_http_host      $http_host;
                set $pass_port           $pass_server_port;

                set $proxy_alternative_upstream_name "";

                client_max_body_size                    0;

                proxy_set_header Host                   $best_http_host;

                # Pass the extracted client certificate to the backend

                # Allow websocket connections
                proxy_set_header                        Upgrade           $http_upgrade;

                proxy_set_header                        Connection        $connection_upgrade;

                proxy_set_header X-Request-ID           $req_id;
                proxy_set_header X-Real-IP              $remote_addr;

                proxy_set_header X-Forwarded-For        $remote_addr;

                proxy_set_header X-Forwarded-Host       $best_http_host;
                proxy_set_header X-Forwarded-Port       $pass_port;
                proxy_set_header X-Forwarded-Proto      $pass_access_scheme;
                proxy_set_header X-Forwarded-Scheme     $pass_access_scheme;

                proxy_set_header X-Scheme               $pass_access_scheme;

                # Pass the original X-Forwarded-For
                proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;

                # mitigate HTTPoxy Vulnerability
                # https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
                proxy_set_header Proxy                  "";

                # Custom headers to proxied server

                proxy_connect_timeout                   5s;
                proxy_send_timeout                      60s;
                proxy_read_timeout                      60s;

                proxy_buffering                         off;
                proxy_buffer_size                       64k;
                proxy_buffers                           4 64k;

                proxy_max_temp_file_size                1024m;

                proxy_request_buffering                 on;
                proxy_http_version                      1.1;

                proxy_cookie_domain                     off;
                proxy_cookie_path                       off;

                # In case of errors try the next upstream server before returning an error
                proxy_next_upstream                     error timeout;
                proxy_next_upstream_timeout             0;
                proxy_next_upstream_tries               3;

                proxy_pass http://upstream_balancer;

                proxy_redirect                          off;

        }

}
## end server grafana.nnpi.beast-code.com

```

I would assume (and hope) that it would not be necessary to somehow modify this directly and that instead, modifying some upstream config (probably in the Helm chart `values.yaml`) would be propagated downward.

# Answer
A responder on the Grafana/Helm charts GitHub site (https://github.com/grafana/helm-charts/issues/2383) pointed out what was wrong with my config. Turns out I do not need the `root_url` for the grafana.ini. Once that was removed the issue has been resolved.
> 0 votes

---
Tags: kubernetes, helm, nginx, grafana
---


# Question
Title: Configure Azure Terraform provider to only allow SSH access to my current IP address
After following this quick start guide https://learn.microsoft.com/en-us/azure/virtual-machines/linux/quick-create-terraform, running `lastb` on the provisioned host shows login attempts from many IP addresses.

The included security rule

```
  security_rule {
    name                       = "SSH"
    priority                   = 1001
    direction                  = "Inbound"
    access                     = "Allow"
    protocol                   = "Tcp"
    source_port_range          = "*"
    destination_port_range     = "22"
    source_address_prefix      = "*"
    destination_address_prefix = "*"
  }

```

is too permissive. Ideally, it would restrict the source\_address to my IP address.

What's the best way to restrict ssh access? Some ideas:

1. Statically configure my current IP address in `main.tf`
2. Dynamically configure the security rule using my current IP address at provision time
3. Run a post-provisioning script to allow only my current IP address
4. Something else

Any suggestions?

# Answer
> 0 votes
This provisions the host with a more restrictive ssh rule.

Add this to `variables.tf`:

```
variable "source_address_prefix" {
  type        = string
  description = "Source IP address allowed to make ssh connections."
}

```

Change the security rule to this:

```
  security_rule {
    name                       = "ssh"
    priority                   = 1001
    direction                  = "Inbound"
    access                     = "Allow"
    protocol                   = "Tcp"
    source_port_range          = "*"
    destination_port_range     = "22"
    source_address_prefix      = var.source_address_prefix
    destination_address_prefix = "*"
  }

```

Set this environment variable (use the actual public IP address you use to connect):

```
export TF_VAR_source_address_prefix=192.168.1.1

```

Provision:

```
terraform apply

```

Verify no more connection attempts to the newly provisioned Azure VM.

---
Tags: terraform, azure
---


# Question
Title: Best practice multiple pipelines into production
Working with a larger corporate. They are planning on opening a second pipeline for a hybrid mobile app. This second pipeline will be for hotfixes straight into production. My gut feeling is that this is a bad smell and what should really happen is more shift-left, better code quality/testing and feature flagging hiding the code/features that are not yet ready. This keeps one pipeline and means its the same process to release for features as it is for hotfixes.

Their concern with this approach is that releasing half finished code (even hidden) into production could open them to liability/bad pr from people who find it and interpret it the wrong way. I am thinking we run the compilation process so that the feature flags stops code even going into the production version of the app.

My question is, do other pragmatic people with real experience in a large (~10k employees) corporate use a second pipeline? If you don't have a second pipeline, what is the best evidence to build a case for change (I tried looking in the devops handbook and doing some googling but no really authoritative statements I could find)?

# Answer
It's pretty common to have a difference at the code level, for example building off a branch of the code. However, the build and deployment pipeline ought to be the same.

The issue around what code goes live should be handled pre-build with a branch, or as part of the build process. There should't be any differences after this (i.e. it shouldn't be used as an opportunity to expedite a fix faster as that's where the trouble starts).

Ideally, you wouldn't even branch for this - you'd trust your feature flags. However, a certain amount of pragmatism helps at the 10,000 employee level.
> 1 votes

---
Tags: continuous-integration, continuous-deployment, build-pipeline, pipeline, operations
---


title: User-level secrets in GitHub Actions

text:
# Question
Title: User-level secrets in GitHub Actions
I would like to create a GitHub workflow for running tests against a REST API. The API needs a key that is always tied to a user, there is no global "testing" key.

Is there a way for each contributor to store their API key in a way that GitHub Actions can access it? I understand there is GitHub Secrets, but these are at best defined at a repo-level. I have also seen that this option exists for Codespaces, but my organization is not using those.

# Answer
> 1 votes
I haven't seen this, since the GHA only has the person who triggered and commits context available.

For testing, it's better to have a user assigned to this purposes and then store its secrets in the repo/org level. Although if the REST API is the project itself, setting up the project (API), creating the test user (with pre-defined secrets) and testing with it should be part of the pipeline as possible.

As a last resource (wouldn't recommend, reconsider strategy), I'd go ahead and use a separate vault for secrets (like Hashicorp Vault), login with project credentials and based on an input or action actor, fetch per-user secrets.

---
Tags: continuous-integration, github, testing, github-actions
---


# Question
Title: On AWS console, how to view ALL tags on a resource?
In the AWS console, for some resources (say for an ECR repo) there is a page for "tags". This page purports to list all tags, however it seems to list only tags without hyphens or underscores in their names.

Why is this, is it documented, and how do you view a list of *all* tags for some resource in the web console?

# Answer
> 0 votes
If you click the "edit" button, you finally do get a complete list of tags on that resource.

Alternatively, if you type a hyphen or underscore into the filter box, more tags appear (if tags containing such characters in their names actually exist).

It's not clear to me why the initial list does not display *all* tags (although it surely purports to be doing so). Regardless of whether this behaviour may be documented somewhere, it is surely a source of unnecessary confusion.

---
Tags: amazon-web-services
---


# Question
Title: Should we build our application inside a docker container, or copy it in?
I am dockerizing our applications to have the environments recreatable. As a process of dockerizing, we will be required to do the below steps:

* Build the application.
* Prepare the environment as the container.
* Deploy the application to the container.

Is it fine to have the application build, inside the docker, as part of docker building? In that way we need not install the build dependencies in local.

But the trade-off is, we have to build the complete docker for the even a small code change that we build. I understand this can be optimized to a level with multi-staged build.

# Answer
> I understand this can be optimized to a level with multi-staged build.

This is an understatement - you do not "optimize" that, but that's just the way you do it.

To answer your question: yes, by all means run the build inside a docker container. How else would you be sure that at build time every library or dependency is the same? There is no reason whatsoever not to do it.

Of course you need to pay a bit of attention when writing the Dockerfile. Ideally, whatever you change in your source code day-to-day, the only thing that happens when you re-run your build is the very last step of actually compiling your code and doing whatever it is you do with it afterwards (creating your deliverable archive...).

You *need* to use multi-staged builds afterwards to get rid of all the intermediate steps, and especially all the compile-time-only components (and it's pretty easy as well).

Note that there's usually some parts of your setup which are used by both the compile-time and run-time (unless you are just creating a .war file and sending that off to a webserver). I.e., if you have a ruby-on-rails application, the "ruby-on-rails" part could well be shared between compile-time and run-time. You want to make sure to put everything that is shared in this way *very early* in the process, and have a clean cut-off point that you can use as a `FROM` later on, copying the compiled files in as needed.
> 2 votes

# Answer
There are significant performance gains to be had in some projects by not building inside the docker containers. If you are building multiple images you can benefit from shared package caches and shared build artefacts which is not possible when you are building every app in isolation inside it's own container.

In a previous project I worked on, we were building the same shared libraries 5 times in 5 different images. Building and running for example tests outside docker really helped speed up our CI/CD.

This is especially true when you are using cloud pipelines like Github actions where downloading and re-hydrating a full docker cache can take as much time as building it from scratch.
> 1 votes

# Answer
The answer depends on your deployment. It sounds like you are wanting to create a generic "environment image" and then do something like mount whatever you want to run with a volume. In this case a multi stage build like doesn't much for you but it may still benefit you.

If you are going to containerize each application then a multi staged build makes the most sense I think. It does keep your final images "clean" since the build was done in a separate container.

The "trade-off" you mentioned should not be an issue. You should work to get your infrastructure to a point where deploying a new image isn't such a big deal therefore negating this problem.
> 0 votes

# Answer
> But the trade-off is, we have to build the complete docker for the even a small code change that we build.

No, not usually. Generally in your Dockerfile the application build process will be after all of the system configuration (installing libraries and such), and so Docker will reuse cached layers for those.

If your concern is that you'll have to rebuild the entire application each time, rather than use some sort of build object caching in your build system, then yes, that does happen. In development you can often mitigate this by using live-reload auto-rebuild systems that watch the filesystem for changes, combined with a Docker directory mount of the code from the local development machine. This way when a developer saves a file, the changes are propagated to the running container, where the autobuild notices and rebuilds and restarts the application.

The other thing you should be careful of are bloated images. If you install a bunch of development tools in the container, then even if you uninstall them later in the Dockerfile, the resultant image will include layers with them, which can increase the overall size of the image.
> 0 votes

---
Tags: docker
---


# Question
Title: Is the forking git workflow used outside of open source projects?
Here is the Forking workflow if you're not familiar with it.

The company I work for is mostly comprised of historically open source developers. This has caused them to be stuck on the forking workflow and they're not extremely willing to move to the github flow. Since there's no support for Jenkins to pick up forks of a repo on stash, build and test automation is very awkward and doesn't really work. My question is: Is there a standard way of supporting the forking workflow, or is it just not realistic for a team to develop this way?

I would really like to move away from the forking workflow, but I have yet to see other companies utilize it. If there's a good way to support it then that would be fine. I just don't know what that would look like from a DevOps perspective. Separating into isolated forks doesn't seem to be as productive and working on a single repo together.

# Answer
> Is there a standard way of supporting the forking workflow, or is it just not realistic for a team to develop this way?

That's the only question I see in your text, so I'll try and address that with my knowledge.

# 'forking' around

`git` was designed to solve the problem of a single monolythic repository everyone are tied to by 'forking'. Compared to Subversion or CVS (or even Visual SourceSafe) a local repo cloned from a remote `origin` is in all practicallity a fork of the one-true-source (`origin`) repo.

This has evolved in centrally-hosted Git SaaS providers to also fork projects to either:

* Fork into a new project - pick up where another project stopped.
* Temporary forking - For easier Pull Request management. (Easier only if it's within the same provider)

## back to the question

* Is there a standard way of supporting the forking workflow

No, as there is no inherent benefit to manage complete ecosystems in separation.

* Is it realistic for a team to develop this way ?

It depends. For developing code and software, there is no benefit, so I would personally answer - no. For developing DevOps solutions - that's another case, as you need a complete envrionment separated from the others to test your devops code.
> 0 votes

# Answer
Maybe you can use a Webhook to read information from the repository. So that way you can access dynamic information such as the repository URL where to look the code, and even create or loaunch specifics Jobs for each Repo.

https://plugins.jenkins.io/generic-webhook-trigger/ Support passing parameters.

I guess it's not the best solution, because it involves configuring each Git repository, but it's quite powerful and dynamic.

I hope you can find a good solution \<3
> 0 votes

# Answer
> Is the forking git workflow used outside of open-source projects?

I have run across it a few times but every time I have seen it, it was being used in a situation where it shouldn't have been used.

> The company I work for is mostly comprised of historically open source developers.

I doubt this. When I have run into teams that used forking it was because they were not experienced open-source developers. Experienced open-source developers would understand the security and isolation that forking provides and why outsiders of a project have to fork in order to contribute. They would also understand that team members of the same team working on the same codebase would be much better off working on the same repository.

When I typically see forking its either because someone set it up that way and then all the people who came afterward thought that was the correct way. I have seen this more than once working with ML teams where the ML researchers are not real developers and thought everyone needed to fork all the repos.

The other reason I have seen forking is because the company did not have any kinda CICD and they had issues with people pushing bad code to the main branch, skipping the PR process, the review process or overwriting each others code.

> Is there a standard way of supporting the forking workflow, or is it just not realistic for a team to develop this way?

I had Jenkins working fine with forks? I don't remember having issues with developers creating PR's on the "main" repo and then running automation on that one repo. Maybe its the SCM you are using. I was using a private GitHub enterprise server and the Github pipeline plugin to run multibranch jobs on new PR's.

> Separating into isolated forks doesn't seem to be as productive and working on a single repo together.

You have nailed it. It's not as good for collaboration as just working on a single repo.
> 0 votes

---
Tags: jenkins, git, build-pipeline
---


# Question
Title: Kubernetes fails to do do garbage collection on images
For quiet some time every once in a while I get an alert, that the disk of one of my Kubernetes cluster nodes is filling up. I pretty quickly found out, that it were the Docker images. As I did not have time to deeper analyze the issue, I just used `docker system prune` to get rid of everything not running (yeah, I know this is highly discouraged).

Now I started to look into this issue and it seems, that the garbage collector has issues with the images:

```
max@nb [~]
-> % k get events -owide
LAST SEEN   TYPE      REASON          OBJECT          SUBOBJECT   SOURCE                MESSAGE                                                             FIRST SEEN   COUNT   NAME
3m15s       Warning   ImageGCFailed   node/cluster00              kubelet, cluster00    failed to get imageFs info: non-existent label "docker-images"      69d          20073   cluster00.16fa70fb01b64a69
4m15s       Warning   ImageGCFailed   node/cluster02              kubelet, cluster02    failed to get imageFs info: non-existent label "docker-images"      69d          20085   cluster02.16fa6da5f5b30c31
54s         Warning   ImageGCFailed   node/cluster04              kubelet, cluster04    failed to get imageFs info: non-existent label "docker-images"      69d          20082   cluster04.16fa6ea64df694c5
3m57s       Warning   ImageGCFailed   node/cluster05              kubelet, cluster05    failed to get imageFs info: non-existent label "docker-images"      69d          20087   cluster05.16fa6d1f167fe7f4
48s         Warning   ImageGCFailed   node/cluster06              kubelet, cluster06    failed to get imageFs info: non-existent label "docker-images"      69d          20077   cluster06.16fa700540142542
2m21s       Warning   ImageGCFailed   node/cluster11              kubelet, cluster11    failed to get imageFs info: non-existent label "docker-images"      69d          20074   cluster11.16fa70c17857fef8
max@nb [~]
-> %

```

I have two other nodes in the cluster, that are newer than the others and do not seem to have the same issue:

```
max@nb [~]
-> % kubectl get nodes
NAME        STATUS   ROLES                  AGE      VERSION
cluster00   Ready    control-plane,master   4y107d   v1.23.8
cluster02   Ready    <none>                 2y270d   v1.23.8
cluster04   Ready    <none>                 3y267d   v1.23.8
cluster05   Ready    <none>                 4y107d   v1.23.8
cluster06   Ready    <none>                 3y171d   v1.23.8
cluster11   Ready    <none>                 592d     v1.23.8
cluster12   Ready    <none>                 139d     v1.23.8
cluster13   Ready    <none>                 139d     v1.23.8
max@nb [~]
-> %

```

The cluster was initially setup more than four years ago manually with `kubeadm`. If I remember correctly, the version was 1.9.6 back then. And I updated it from there up to 1.23.8 right now. When the last two nodes (the working ones) were added, the cluster was on 1.19.8.

It is very much possible, that I forgot some configuration change in any of the upgrades. 69 days ago (the time the first appeared as of `get events`) I did the upgrade from 1.19.8 to 1.23.8 (and all versions in-between, of course). Could be, that the issue started then. Could also be, that the upgrade *cleared* the events log. Not sure if the issue was already there before, to be honest.

Has anyone any idea where I could start looking for the reason of this issue?

Thanks in advance!

# Answer
> 1 votes
I think I fixed the issue.

It seems it comes up, when the `kubelet` service starts before the `docker` service is up and running. So I added the directive `After=docker.service` to the `[Unit]` block of the `kubelet` service and restarted it. The issue seems to be gone now.

# Answer
> 0 votes
Quick fix by below works for me:

```
docker system prune
systemctl stop kubelet
systemctl stop docker
systemctl start docker
systemctl start kubelet

```

---
Tags: docker, kubernetes
---


# Question
Title: no kind "KubeSchedulerConfiguration" is registered for version "kubescheduler.config.k8s.io/v1beta3"
I'm trying to apply KubeSchedulerConfiguration but getting below error, Can anybody give direction?

K8s version: 1.21.14
Bootstrap tool: kind

scheduler-config.yaml

```
apiVersion: kubescheduler.config.k8s.io/v1beta3
kind: KubeSchedulerConfiguration

profiles:
  - schedulerName: default-scheduler
    pluginConfig:
      - name: PodTopologySpread
        args:
          defaultConstraints:
            - maxSkew: 1
              topologyKey: kubernetes.io/hostname
              whenUnsatisfiable: ScheduleAnyway
          defaultingType: List

```

Error log

```
root@kind-control-plane:/etc/kubernetes/manifests# cat /var/log/containers/kube-scheduler-kind-control-plane_kube-system_kube-scheduler-31d58a4a4df5ae2d71afa2508797e9026987ca61c82cf047007ae642cfb3a688.log
2022-10-31T09:10:41.6436638Z stderr F I1031 09:10:41.643531       1 serving.go:347] Generated self-signed cert in-memory
2022-10-31T09:10:41.643702592Z stderr F no kind "KubeSchedulerConfiguration" is registered for version "kubescheduler.config.k8s.io/v1beta3" in scheme "k8s.io/kubernetes/pkg/scheduler/apis/config/scheme/scheme.go:30"
root@kind-control-plane:/etc/kubernetes/manifests#

```

# Answer
According to the Scheduler Configuration docs:

> Note: KubeSchedulerConfiguration v1beta2 is deprecated in v1.25 and will be removed in v1.28. KubeSchedulerConfiguration **v1beta3** is deprecated in v1.26 and will be removed in v1.29. **Please migrate KubeSchedulerConfiguration to v1**.
> 1 votes

---
Tags: kubernetes
---


# Question
Title: What is this "rootlessport", and how to get it to release the port hold?
We're trying to improve on the Mailman3 Docker project, converting it to podman(4.4.1) pod. I've seen several "address in use" errors. I believe the way to debug them is to find what's holding onto the port and stop it, or else use a different port. But on this VM, I cannot get the port to release. I've issued the following commands as the user that's been starting the pod. None of them seem to reset networking to release the usage of the port:

```
» podman pod stop <pod>
» podman pod rm <pod>
» podman system prune --all --force
» podman system reset

```

I only see this rootlessport service using the port, which I thought was related to podman itself?

```
» netstat -plan | grep :8000
tcp6       0      0 :::8000                 :::*                    LISTEN      40976/rootlessport
»

```

What is rootlessport, and how do I get it to release the port? I don't believe it's a systemd service:

```
»root ~» systemctl -l | grep -i rootlessport
»root ~1»

```

Possible related bug: podman issue 9083

# Answer
> 1 votes
I found that rootlessport is part of the `podman` package.

I was able to kill the process as root:

```
kill $( ps aux | grep rootlessport )

```

After that, the port was released, and I could spin up new containers.

I'm still not sure what went wrong. I found nothing in podman logs, and also nothing from this:

```
» journalctl -u rootlessport
-- No entries --

```

---
Tags: ports, podman
---


# Question
Title: Passing variables between scripts in a Jenkins pipeline
I have a declarative Jenkins pipeline that looks like the following:

```
pipeline{
    agent {
        ...
    }
    stages {
        stage("Provision EC2 Instance and Set ENV Variable"){
            steps {
                dir("docker-image-builder"){
                    sh 'python ./provisonInstance.py'
                }
            }
        }
        stage("Configure EC2 Instance with Ansible"){
            steps {
                ansiblePlaybook('./ansible.playbook.yml') {
                    ansibleName('1.9.4')
                }
            }
        }
    }
}

```

In my first stage, I have a Python script that provisions an Amazon EC2 instance that will be used to create an AMI. In the second stage, I will then configure configure my instance before finally creating an AMI and terminating the instance. In order to do this, I need the instance-id of the EC2 instance created in the first stage. **Is there an elegant way to pass values generated by a script between stages, and if so how?**

The most obvious method that I can think of is to write the value to a file on the agent in the first stage, and read it in the second stage, but this does not seem like an elegant solution.

# Answer
For Declarative Pipeline:

This question already has a highly-upvoted answer on StackOverflow:

> If you want to use a file (since a script is the thing generating the value you need):
>
> ```
> pipeline {
>   agent { label 'docker' }
>   stages {
>     stage('one') {
>       steps {
>         sh 'echo hotness > myfile.txt'
>         script {
>           // trim removes leading and trailing whitespace from the string
>           myVar = readFile('myfile.txt').trim()
>         }
>         echo "${myVar}" // prints 'hotness'
>       }
>     }
>     stage('two') {
>       steps {
>         echo "${myVar}" // prints 'hotness'
>       }
>     }
>     // this stage is skipped due to the when expression, so nothing is printed
>     stage('three') {
>       when {
>         expression { myVar != 'hotness' }
>       }
>       steps {
>         echo "three: ${myVar}"
>       }
>     }
>   }
> }
>
> ```

A second solution would be to write out output from your script directly or with `writeFile` and then later read it back in with `readFile`. However, this comes with the significant caveat that both the reading and writing must take place on the same node, so you can't use this to send data across nodes.

You can also can archive files centrally with `archiveArtifact` and retrieve them with `copyArtifact` (and then read in the file with `readFile`).

Finally, there are some answers to a near-identical question on StackOverflow that offer alternative solutions.

---

However, if you can, I would suggest using Scripted Pipeline instead, which allows a much simpler solution:

```
def String myVar

stage('my-first-stage') {
  myVar = sh(script: 'my-command', returnStdout: true)
}

stage('my-second-stage') {
  sh("my-other-command --var='${myVar}'")
}

```

(I also already answered this question for Scripted Pipeline over on ServerFault.)

---

It looks like some of these solutions have already been proposed in the comments, but you are concerned that they are not elegant enough. I'm not really sure what kind of elegance you're looking for, because I can't think of any way to simplify these answers in Jenkins-Pipeline-land.
> 9 votes

# Answer
There is a simple solution for declarative pipeline which doesn't involve having to write to files - define the variable-to-be-shared *outside* the `pipeline` block. This is much neater and means you should still be able to restart the job from the later stage if you need.

Example:

```
def variable = "initial value"
pipeline {
    agent {
        ...
    }
    stages {
        stage("Override variable") {
            steps {
                script {
                    variable = sh(script: 'some command', returnStdout: true).trim()
                }
            }
        }
        stage("Read new value of variable") {
            steps {
                script {
                    echo variable
                }
            }
        }
    }
}

```

Further explanation in video form
> 1 votes

---
Tags: jenkins, ansible, jenkins-pipeline, python
---


# Question
Title: Set the base path of an Apache proxy handler
I'm developing my application with 2 containers: **apache** and **php-fpm**.

Apache serves requests, and delegates PHP execution to the php-fpm container in its vhost:

```
    <FilesMatch .php$>
        SetHandler "proxy:fcgi://myapp-php-fpm:9000"
    </FilesMatch>

```

The two containers share a single volume that contains the application code.

The location of that volume is different in each of the containers:

* apache mounts the app code volume at `/home/wwwdata/myapp`
* php-fpm mounts the app code volume at `/var/task`

How can I change the script path that Apache sends to the php-fpm container, so that it matches where the volume is mounted in the php-fpm container?

---

I can do this in nginx by setting `SCRIPT_FILENAME` and `DOCUMENT_ROOT`:

```
        location ~ ^/index\.php(/|$) {
          fastcgi_pass myapp-php-fpm:9000;
          fastcgi_split_path_info ^(.+\.php)(/.*)$;
          include fastcgi_params;

          fastcgi_param SCRIPT_FILENAME /var/task/$fastcgi_script_name;
          fastcgi_param DOCUMENT_ROOT /var/task;

          internal;
        }

```

but I haven't been able to replicate this behavior in Apache.

# Answer
> 0 votes
Use `ProxyFCGISetEnvIf` to set the `SCRIPT_FILENAME`:

```
    <FilesMatch .php$>
        ProxyFCGISetEnvIf "true" SCRIPT_FILENAME "/var/task%{reqenv:SCRIPT_NAME}"
        SetHandler "proxy:fcgi://myapp-php-fpm:9000"
    </FilesMatch>

```

---
Tags: docker, php, apache-httpd
---


# Question
Title: Automate starting and stopping of multiple containers on single production server
I am doing a project for my university where I have the following requirements:

1. I want to start and stop (docker) containers with different apps
2. I have a singe (university) server to my disposal
3. I want the starting and stopping to be somewhat automated using the universities GitLab

My current idea is the following:
Users create a new branch on an existing GitLab project. If the branch has some specific format, i.e. `Deploy-0x-name` a new container is deployed on the server and accessible through the network. If the branch is deleted, the container is also removed from the server. Now I am unsure what tools to use. I was thinking about using Kubernetes, but it seems that is not ideal for a single server setup. At least that's what I read. Also, I don't really need the load balancing between different container instances as there will only be one instance of every application running. Still, I have to manage multiple applications.

Is there a better alternative for K8s in this situation, or should I use a completely different approach?

Thank you

# Answer
Jasper -

I think you want to establish some form of communication between GitLab and your server, in particular, GitLab should be able to initiate events on the server in response to events (changes) in the GitLab repository.

A common pattern for a solution is to use a webhook -- a feature of GitLab (I assume, I use GitHub) that allows you to instruct GitLab to perform a given action when something happens in GitLab. You can choose which things, for example, you might choose to have GitLab fire the webhook whenever there's a push.

What happens when the push occurs? You need to pull from the repo, build a new container, upload it to the container repo, then run docker using the new container.

The webhook is expected to be a URI that refers to a remote server ... in particular, the remove server needs to be listening on the port specified by the URI. The most common port is either the HTTP port 80, or the HTTPS port 443 -- and the reason they are popular is that many systems don't block them, as they do with most other ports. So you need something on your server listening on port 80/443 that will do these steps.

So the steps needed (pull, build, upload, run) often fall into two phases: "build" and "run", where build includes all the steps needed to deploy a workable application. GitHub offers "github actions" -- I suspect GitLab has something similar -- that give you access to a temporary compute resource for things like running commands e.g. "pull", "compile", and "docker". Or, you could build a basic web server app that ran these commands. Either way, something on the server will need to run docker (or maybe ask it to restart) using the newly built image.

Could you use Kubernetes for this ... sure ... but as you imply, it's too much for the single task. Learning how to deploy kubernetes is a lot of work. Instead, Google "xyx web server" where xyz is the programing language or environment to find a suitable installable server. Most offer a simple way to integrate system commands and respond to webhooks.
> 0 votes

# Answer
The problem with not using k8s is that because k8s exists and is so successful you are unlikely to find robust and easy to use alternatives. The problem you have correctly identified is that k8s is designed to run containers at scale so it runs across many machines. And that makes it quite complex to set up. You also need many machines and you have one.

Yet you can run kubernetes on a single server with “MiniKube”. You can run it on your laptop. It isn’t seen as “production ready” only “development mode” because no-one expects to run a business critical system on a single server. Yet if all you have is a single server then this is fine to use MiniKube.

If you don’t run k8s you will have to create at lot of “application infrastructure” yourself. You will have to write software, from scratch, that runs all the time, exposes a Webhook, that GitLab hits, that then does a lot of “from scratch” container management.

While this is completely possibly I wouldn't attempt it. You have to worry about containers crashing, containers starting after server restarts, everything working after the server operating system is patched.

Once you have “MiniKube” running (start on your own laptop!) your problem becomes a whole load easier. It will remember what containers should be running and restart them when the server is restarted (you obviously have to start up MiniKube when the server reboots). It also exposes the standard k8s API that is designed to make management of containers easy. That makes solving your problem a lot easier.

You said you want to run a container per branch with a certain naming convention. This is a slightly unusual. Typically we would make “master” the state of the server and use a folder on master per “service”. Nevertheless an application per branch it is workable. Regardless of whether it is GitLabs, GitHub Actions, or BitBucket Pipeline’s I would use Helm as the tool to manage the containers.

I would create a docker image with Helm in it and have GitLab pass in a credential (perhaps a username and password?) via environment variables and also pass the branch name as a parameter. Then when the build runs you can have a script in the container build the application then run Helm asking to install or upgrade the application. You can use the branch name as the Helm application name. Helm will create or update the application using whatever it is that is inside of the git repo on that exact branch.

To start out install Helm and MiniKube on your laptop. Then follow the solid Helm documentation on how to have it package and install applications into the MiniKube on your laptop. Once you have a prototype solution install MiniKube on the server and try out managing applications by running a Docker image on your laptop that has Helm in it. Finally have GitLab run the tested Helm Docker image.
> 0 votes

# Answer
Since you only have a single server and a few services, I'm not sure K8's is a great choice. It is very powerful and it's worth learning so maybe give it a shot.

A better approach in this situation is using Ansible. It's a configuration as code (CasC) and it can be automated in your Gitlab pipeline. Another nice thing about Ansbile is it will kinda force you to learn more about systems, services and networking than using something like K8's.

Another approach that I'm not very familiar with would be something like Hashicorps Nomad. Similar to K8's? Maybe kinda? But much simpler. Not sure how it compares to Ansible.

The last thing, that I wouldn't really recommend, is just SSHing into the server and doing it with bash and/or docker compose. This is like doing it the hard way but without the benefit of learning a skill like Ansible.

Also, there are Ansible alternatives like Chef and Puppet but I really think Ansible is where it's at.
> 0 votes

# Answer
I would suggest - although not idea either to install GitLab with the Omnibus Installer method and then on the same machine host a GitLab Runner. Since the latter uses Docker already you could deploy your applications via GitLab CI on the very same machine the code is initially hosted on.
> 0 votes

---
Tags: docker, kubernetes, gitlab
---


# Question
Title: Is there a way to increase Rate limits on Azure DevOps?
We were using an on premise Jenkins against an on premise GitBlit. This worked fine. Last week we migrated from GitBlit to Azure DevOps.

Our Jenkins Jobs now run with the PAT of a 'fake' Azure Account (not the 'Build Service Account' because that one can't be used by an on premise Jenkins). So for Azure DevOps this is a regular user. We found no other way to set this up. Using the PAT of an employee was not really a good option.

Problem now is that we are constantly hitting the Rate Limites with following error:

> Request was blocked due to exceeding usage of resource 'ATCPU' in namespace 'VSID'. For more information on why your request was blocked, see the topic "Rate limits" on the Microsoft Web site (https://go.microsoft.com/fwlink/?LinkId=823950).

After which the Jenkins Job fails.

This is a game stopper in our migration to Azure DevOps because there seems to be no way to adjust those Rate limits for a given user...

Do you have recommendations? Is there a way to increase those limits?

In the screenshot, what does the Count and TSTUs exactly mean? We have the idea that it's not about how many bytes are transferred but rather the number of Git Operations.

Our Jenkins Jobs mainly clone Git Repositories with the usage of local mirrors.

# Answer
> 1 votes
A Jenkins build has the same 200 TPU limit as an Azure DevOps build, in that they both get the standard user limit.

Azure builds treat each pipeline as a separate user, so if you have many pipelines using the same account it would mean sharing the limit across all builds.

200 TPU is *a lot*, so it's worth checking to see if you are doing something super inefficient in the build that may have gone unnoticed when things had no limit. For example, are you calling a git operation in a loop in there?

If you are only running one build, it just be doing something huge. These "TPU" values are very close to being equivalent to Azure database DTUs. That's a lot of activity... more than I'd expect one user to be able to generate unless your build churns git operations, or the repo has tons of large files, or some other "unusual" condition.

---
Tags: jenkins, azure-devops, azure, git
---


# Question
Title: Is there a way to avoid deleting a branch in Azure DevOps without obliging pull requests?
In Azure DevOps, configuring a branch policy ensures the branch cannot be deleted. However this also obliges to use pul requests to make changes to that branch.

Currently we are not using pull requests yet, but we still would like to prevent certain branches from getting deleted.

Is there a way to do that other than branch policy?

# Answer
If you disable the "Force Push" permission, it will block users from deleting the branch.

The "Force Push" permission...

> Can force an update to a branch, delete a branch, and modify the commit history of a branch. Can delete tags and notes.

See more on these permissions on Microsoft Learn.
> 1 votes

---
Tags: azure-devops, git
---


title: redirect url/ to url/test using ingress

text:
# Question
Title: redirect url/ to url/test using ingress
I'm running an application on kubernetes, when i use this url: https://myapp.test.com/ it redirects me to ressource not found (problem in the application). what i need now is to avoid this page by redirecting the url/ to url/test where i have the home page of the application. Is this possible using Ingress? If you can show me please an example of a configuration that does this redirection. Thank you.

# Answer
Add an annotation nginx.ingress.kubernetes.io/app-root: /test as per the documentation kubernetes.github.io/ingress-nginx/examples/rewrite/#app-root

Quoted from the doc and adapted to your exemple, your ingress yaml should look like this (adapt the target service and port):

```
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/app-root: /test
  name: myapp
  namespace: default
spec:
  ingressClassName: nginx
  rules:
  - host: myapp.test.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: http-svc
            port:
              number: 80

```
> 1 votes

---
Tags: nginx, ingress, ingress-nginx
---


# Question
Title: What is the corrrect way to install remote Helm applications via Argo CD?
I need to install loki-stack to my k8s cluster, previously I need to change `retention_deletes_enabled` and `retention_period` in `charts/loki/values.yaml`.

What should I do to achieve this?

The problem is that there are several `values.yaml` files and I can't change loki values (`charts/loki/values.yaml`), I only can change the main `loki-stack/values.yaml` of this app when I create new application in Argo CD.

In general is it the correct way to clone the repo or doing `helm pull` then add this code to local git repository and create app from it?

# Answer
If I understood your question correctly, you want to configure the "Loki" Helm chart when installing the "Loki-Stack" Helm chart.

The Loki-Stack Helm chart has several sub-charts (dependencies), each of which is available with its own name through the `values.yaml`: https://github.com/grafana/helm-charts/blob/main/charts/loki-stack/values.yaml

So for example if you want to configure where Loki stores the logs, you would do that as follows **in the loki-stack Helm Helm chart**:

```
loki: # <-- extra nesting required to access values of sub-chart
  loki:
    storage:
      type: 'filesystem'

```

Note that you need the extra "loki" entry to correctly pass the values to the sub-chart. Read me more here: https://helm.sh/docs/chart\_template\_guide/subcharts\_and\_globals/
> 1 votes

---
Tags: helm, argocd
---