# Question
Title: Server error 502 - problems with .net core web app deploy made on GCloud's GKE using OAuth 2.0 Authentication
**I have an MVC .net core 6 web application that uses Google's OAuth2.0 authentication and I cannot publish it with Kubernetes.**

**Project context:**

***Locally it compiles** and runs normally, including creating a docker image and running a container, but to run the docker container locally and have an https port with a valid certificate (I need this for Google's OAuth to work) I need to run the following procedure.*

Sequence of commands and procedures to make the local certificate trustworthy (you only have to do this to run in a docker container, running the project directly is not necessary):

1. create the local certificate pfx: dotnet dev-certs https -ep $env:USERPROFILE\\.aspnet\https\Apresentacao.Web.pfx -p pa55w0rd!
2. Create a UserScretsId configuration line in the application's .csproj: {SomeGuidIdHere}
3. Enter the certificate password in the local user-secrets: dotnet user-secrets set "Kestrel:Certificates:Development:Password" "pa55w0rd!"
4. make the certificate 'trusted' dotnet dev-certs https --trust
5. Run the docker container, but passing through some necessary parameters (run in powershell or bash outside the project folder):

```
docker run -p 8080:80 -p 8081:443 -e ASPNETCORE\_URLS="https://+;http://+" -e ASPNETCORE\_HTTPS\_PORT="8081" -e ASPNETCORE\_ENVIRONMENT=Development - v $env:APPDATA\microsoft\UserSecrets\\:/root/.microsoft/usersecrets -v $env:USERPROFILE\\.aspnet\https:/root/.aspnet/https/ admin

```

If I do not perform the above procedure, or execute 'Docker run' without passing the necessary parameters to use the https port and certificate, I receive the following return when running the container:

`warn: Microsoft.AspNetCore.HttpsPolicy.HttpsRedirectionMiddleware[3] Failed to determine the https port for redirect.`

**My problem is:** Locally I found the solution to deal with the docker container, the https port and the ssl certificate for localhost, however I am publishing this same application on Google Cloud using Git Hub actions to run my deployment pipeline and in the cloud I am saving the image docker in a bucket and creating a service with kubernetes. We already have other applications running on this Google cloud infrastructure with Kubernetes and we had no problems exposing the URLs with Ingress and using the certificate provisioned by Google.

**I did several tests** publishing with different configurations until I suspected that it could be something in the application, I decided to do a test by publishing without the Google authentication configuration and to my surprise the application had health status in Kubernetes and was made available on URL with https correctly. When I include authentication with Google again in the project's Startup.cs, my problem returns in the Kuberntes container and when accessing the URL the return I get is a 502.

**Here's how I'm configuring authentication with Google:** On Startup.cs

ConfigureServices Method:

```
public void ConfigureServices(IServiceCollection services)
        {
            //Auth config google
            services.AddAuthentication(options =>
            {
                options.DefaultScheme = CookieAuthenticationDefaults.AuthenticationScheme;
                options.DefaultChallengeScheme = GoogleDefaults.AuthenticationScheme;
            })
            .AddCookie()
            .AddGoogle(options =>
            {
                options.ClientId = "{my_ClientId}";
                options.ClientSecret = "{my_ClientSecret}";
            });
            ...
}

```

Configure Method:

```
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }
            else
            {
                app.UseExceptionHandler("/Home/PaginaNaoEncontrada");
                app.UseHsts();
            }
            app.Use(async (context, next) =>
            {
                await next();
                if (context.Response.StatusCode == 404)
                {
                    context.Request.Path = "/Home/PaginaNaoEncontrada";
                    await next();
                }
            });
            app.UseHttpsRedirection();

            app.UseStaticFiles(new StaticFileOptions
            {
                OnPrepareResponse = o => { o.Context.Response.Headers.Append("Cache-Control", $"public, max-age=10800"); }
            });
            app.UseUnobtrusiveAjax();
            app.UseRouting();
            app.UseAuthorization();
            app.UseAuthentication();            //Important
}

```

In my HomeController.cs (where my first request hits when accessing the application for the first time and where I validate that the User is authenticated) the \[Authorize\] annotation above the Controller is strictly necessary for what I added in Startup to work .cs:

```
[Authorize]
public class HomeController : Controller
{ ...}

```

My Action for the Home Index:

```
[HttpGet]
return View();

```

LoginGoogle called by Index:

```
public ActionResult LoginGoogle(){
    return Challenge(new AuthenticationProperties { RedirectUri = "/" }, "Google");}

```

To publish to Kubernetes I have the following files (I highlight three of them because I consider that the most important settings for the site are in them):

My deploymet looks like this:

`apiVersion: apps/v1`

My Ingress looks like this:

`apiVersion: networking.k8s.io/v1`

And my service looks like this:

`apiVersion: v1`

With the recent changes I made to my .yaml files for Kubernetes I started receiving more logs from the service, similar to what I would receive locally if I ran the 'Docker run' command without parameters to use the local certificate: Latest container logs: https://drive.google.com/file/d/1VcmiBe0LBuMr5B0VC5YivrvZdygZtEf5/view?usp=drive\_link

Before these changes (mainly in the deployment file) I was receiving the following return in the logs: Previous logs: https://drive.google.com/file/d/16NoN1fhhmoGk2bZRWTY0oxqFBY3xH4Kj/view?usp=drive\_link

Below the logs illustrated above was just an infinite loop of information about ResponseCoockies and the information about the https port was not repeated until a new deployment was made..

How the service responds when accessing the URL when we publish WITH Google authentication configured: https://drive.google.com/file/d/1jDCmZNFPvzRvnq2VeOr7yC79A9Cba1rf/view?usp=drive\_link

**My question is** if I am currently configuring Kubernetes or is something missing for the certificate to be trusted? Why does the deployment happen normally when I remove the Google authentication configuration from my application? (I remove the Startup config and the Controller annotation)

It is important to note that my OAuth Client Id credentials are correct as I was able to test by adding the https localhost URL as authorized, so this would not be the problem either. I added the URL that I am provisioning in Kubernetes to the list of URLs authorized to use the Client Id OAuth, as follows: https://drive.google.com/file/d/1Hzi74jJ6sZ7za5D3rFZcyDQRaIoW4jOS/view?usp=drive\_link

**Other information.**

A screenshot of when I publish the application without Google authentication configuration and it stays UP and the certificate appears to be linked to the URL and secure:

https://drive.google.com/file/d/1RLocACa-TaCsmZo1S74sQoKrzvFgcfyx/view?usp=drive\_link

When deploying with Ingress.yaml, it creates a Load balance in the infrastructure and automatically configures a front end endpoint linked to a static-IP:

https://drive.google.com/file/d/1f1LyIzcDqUbEUEpPTh6VQqdwyshMeUVA/view?usp=drive\_link

We use the AWS service to purchase and host domains and so that when accessing the URL that I configured, my DNS would be resolved to the IP of my front end, I configured a 'Record name' to be redirected to the IP. This has already been done for other applications and it worked, it even works when we publish this application without authentication...

Below is the registration I made in the AWS hosting zone: https://drive.google.com/file/d/1y8BIeK7RYywP95jPzpCGYkgZ7Lv-w7KG/view?usp=drive\_link

And the details of the certificate provisioned by Google and which only became 'Active' when communication with AWS was established due to the mapping I did with the IP, as mentioned above. https://drive.google.com/file/d/1Y9wLH\_DDX-93iMY5eNwBN0VcDFbId5I1/view?usp=drive\_link

Thank you in advance for your help

# Answer
> 0 votes
We found the solution to the problem described above. Basically, the container within Kubernetes needs that, in addition to the health check end-points, the base url of the application is also up, that is, I could not be trying to add the \[Authorize\] annotation directly to my HomeController as it was the first to be displayed by the application. I was defining the "default" End-point of my application as follows:

```
app.UseEndpoints(endpoints =>
            {
                endpoints.MapControllerRoute(
                    name: "default",
                    pattern: "{controller=Home}/{action=Index}");
            });

```

However, HomeController was completely covered by \[Authorize\] and the application was being considered unhealth in the kubernetes backend. Therefore, what solved the key problem was creating a Login screen (without applying \[Authorize\]) to call Google's login and changing the default endpoint for it:

```
app.UseEndpoints(endpoints =>
            {
                endpoints.MapControllerRoute(
                    name: "default",
                    pattern: "{controller=Login}/{action=Index}/");
            });

```

So the backend became heath and the application went up, first calling the Login screen and then clicking on "Login with Google" we called Google's OAuth 2.0, here's another problem... **the application was redirecting to OAuth using HTTP, but we needed to correct this so that it redirected to HTTPS.** We tried all the suggested solutions, but what worked in our case was:

```
public void ConfigureServices(IServiceCollection services)
        {
 services.Configure<ForwardedHeadersOptions>(options =>
            {
                options.ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto;
                // Only loopback proxies are allowed by default. Clear that restriction because forwarders are
                // being enabled by explicit configuration.
                options.KnownNetworks.Clear();
                options.KnownProxies.Clear();
            });
....
}

```

The Microsoft documentation we used as a basis was: https://learn.microsoft.com/en-us/aspnet/core/host-and-deploy/proxy-load-balancer?view=aspnetcore-6.0 However this doc. it became confusing for us, we tried all the suggestions and nothing was working in our case... until the Clear settings for KnownNetworks and KnownProxies made our application finally use our redirect for HTPPS.

---
Tags: kubernetes, google-cloud-platform, gcp, authentication, gcloud
---


# Question
Title: How to run multiple identical docker containers using jumbo frames?
I am attempting to run multiple (we'll say 2) instances of software in docker. This software is proprietary and 3rd party, so modifying it is not an option, and neither is using docker exec or attach to make changes from inside the containers.

The software in question launches a web GUI at port 80. It also receives traffic on port 50000. The traffic we expect to send to port 50000 is extremely latency sensitive and at 10G.

The host has multiple interfaces:

```
bond0 172.16.2.10
bond0:0 172.16.2.20
bond0:1 172.16.2.21
em3 172.16.3.10
em4 172.16.4.10

```

Bond0 is the host's primary connection to the LAN. The aliases 0 and 1 are there because we're running 2 copies of the software, and for simplicity it made sense if they each have their own IP. This might be the wrong approach, so I can remove these aliases if needed.

EM3 and 4 are fiber connections directly to an appliance that will be sending the 10G latency sensitive traffic on port 50000. EM3 is designated for the first instance and EM4 for the second. These assignments are physical and cannot be changed, but if I need to change the configuration of these connections that is fine.

Originally I used the default bridge network for all of this, mapping specific ports and IPs to each container. This worked excellently for connecting at first, the web UIs were accessible and the fiber connections were up. However, I ran into issues soon after: Docker's bridge network driver is limited to 1500 MTU, and the traffic headed over EM3 and EM4 requires an MTU of 9000. Attempts to set the bridge network at MTU 9000 using daemon.json or spinning up a new bridge network have proven unsuccessful.

I really would like to use `net=host` for these containers, but port conflicts would immediately occur and not all of the ports that the software requires are customizable. There is no way in the software to bind to specific network interfaces. I also explored `macvlan_mode=passthru`, setting up a macvlan for each network interface and assigning the relevant ones to containers, but I am not sure if this is the right path.

Anyone have any ideas?

# Answer
I gave up on finding a particularly clean solution and enabled host networking. In order to prevent port conflicts, I set up iptables forwarding and masquerade rules that switch the ports around behind the scenes depending on what IP address and port traffic is sent to or from.

This works for accessing the software GUI (it defaults to typical HTTPS port but can be reassigned with an environment variable) but not for all other features. Thankfully I can update the latency sensitive traffic ports as well. I wish there was a better way, but this does resolve my MTU issue, even if it is messy and imperfect.
> 0 votes

---
Tags: docker, networking
---


# Question
Title: How to implement simple distributed configuration system
I have multiple servers running in data centers around the world. Each server has its own configuraiton database, and cli command needs to be executed to update in-memory cache.

I want to automate this system. I have following requirements:

1. Single source of truth (i.e. git repository)
2. No drift in the system (inserts/updates/deletes are applied to databases as required)
3. Feedback that everything was executed correctly (i.e. if some db replication of cli commands fails, I want to know that)

Please note I would like to store configuration as "Desired state", not as migrations to database which insert additional records.

Right now process if following:

```
1. Create commmit to source-of-truth repository
2. Connect to database
3. Update configuration in database (this is usually lots of records)
4. Ensure configuration is applied correctly (select and verify)
5. Connect to service via ssh
6. Execute command to reload configuration
7. Verify if configuration is reloaded (i.e. run command to dump config and grep new configuraiton entries)
8. Repeat steps 2-7 for multiple datacenters.

```

I've evaluated some existing tooling, and I cannot find anything good:

* Infrastructure management tools like terraform are not good for managing contents of databases, and they cannot really call scripts
* Configuration management tools like ansible are more focused on executing scripts, rather than synchronizing state - i.e. I can apply DB migraiton, but this is very cumbersome (btw, I don't want to configure server instances, but software running on these server instances).

Can you suggest how to automate this system?

# Answer
You can use Flyway for database migrations and you could wrap actual migrations as releases.

So something like you have a Flyway container running as a Cron job and if there is new release - it would be applying migrations from the new releases, otherwise it would just use existing migrations - meaning not change the state.

Then you can use Reliza Hub - the tool I am developing - to manage rollout of these migrations to different instances - see a sample here on how promotion logic may work - https://worklifenotes.com/2023/05/12/how-to-promote-software-to-production-with-reliza-hub/

Reliza Hub can also give you Slack / MS Teams notifications of the releases in progress and if there were errors.

You could also add your additional verification steps in the end of each cronjob, although Flyway is already capable to provide feedback if migrations passed or failed.

I hope this gives basic idea on how to automate your process - but feel free to reach out to me with follow up questions.
> 0 votes

# Answer
Unfortunately "diffing" logical contents of relational databases seems to be a largely unexplored problem. I'd assume because SQL itself is a strictly imperative language and that makes it difficult to act on a difference even if calculating it is technically feasible.

Based on your comment which implies that you require a full view of all the data at the single source of truth at all times, I'd approach the issue using complete "rewrite" migrations instead of incremental ones, which is what tools like Flyway usually expect you to do.

1. Have a pure-text representation of your **complete** desired configuration in a git repository. Maybe YAML, maybe TOML, maybe some other format that's more appropriate for your domain, wouldn't really matter as long as it can be parsed programatically.
2. Write "glue code", for example in Python, which takes the above config and converts it to SQL `INSERT` statements, then executes them against a target DB server.
3. On every script run, the script drops all existing configuration tables and completely recreates them. Naturally, all of these operations need to be done in one transaction to avoid downtime.

All this can then be automated with Ansible.
> 0 votes

---
Tags: configuration-management, mysql, distributed-systems
---


# Question
Title: SSH from one docker container to another that is using --network flag?
I am attempting to ssh from a docker container into another, but the container I am trying to ssh into needs the --network host flag to be used, and therefore has no IP address.

I understand that that container is using my hosts port 22 so I changed the ssh port in the container that I am trying to ssh into to port 2200 by adding "Port 2200" to my /etc/ssh/sshd\_config file

But the container still has no IP address because of the --network flag. So how do I ssh into a container that has no IP address from another container?

I am starting one container with the command

```
docker run --rm -it --privileged \
--network host \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /run/netns:/run/netns \
--pid="host" \
-w $CLAB_WORKDIR \
-v $CLAB_WORKDIR:$CLAB_WORKDIR \
--name sshtest \
ghcr.io/srl-labs/clab bash

```

The other command is just a Ubuntu container that is just started with `docker run -it --name ubuntu ubuntu`

# Answer
> 0 votes
Start both containers in the same network

First, create a network with driver type, I prefer bridge:

```
docker network create networkie --driver bridge

```

You can also IP-range and subnets using `--subnet` and `--ip-range`.

Then run both containers in them:

```
docker start --<your flags> container-1 --network networkie`

docker start --<your flags> container-1 --network networkie`

```

---
Tags: docker
---


# Question
Title: What’s the docker compose equivalent of my docker command?
I’ve been using the command line for docker for 2 weeks and just learned how to use `docker compose` yesterday using the excellent documentation I found at Jellyfin. I got a Jellyfin container up and running in a jiffy on my new Ubuntu Server (22.04.4 LTS) and couldn’t be happier.

I thought I’d go back to Emby and learn how to create a compose file as an exercise.

I currently use the following command line to successfully bring up Emby:

```
docker run -d \
        --name embyserver \
        --volume /home/udance4ever/emby:/config \
        --volume emby-cache:/cache \
        --volume /home/udance4ever/mnt/userdata/kodi:/userdata/kodi \
        --device /dev/dri:/dev/dri \
        --publish 8096:8096 \
        --publish 8920:8920 \
        --env UID=1001 \
        --env GID=1001 \
        --restart unless-stopped \
                emby/embyserver:latest

```

I can’t figure out the difference between the above command line and the following compose file I “translated”:

```
version: '3.5'

volumes:
  emby-cache:

services:
  embyserver:
    image: emby/embyserver:latest
    container_name: embyserver
    user: 1001:1001

    ports:
      - "8096:8096"
      - "8920:8920"

    volumes:
      - /home/udance4ever/emby:/config
      - emby-cache:/cache
      - /home/udance4ever/mnt/userdata/kodi:/userdata/kodi

    devices:
      - /dev/dri:/dev/dri

    restart: 'unless-stopped'

```

I use `docker compose up -d` to bring up the server and it just keeps restarting:

```
a0b30f16f381   emby/embyserver:latest   "/init"                About a minute ago   Restarting (1) 3 seconds ago

```

I’m real new to docker so I don’t know where to begin to debug this compose file (and reverted to recreating the container using the command line which, again, works without error)

I’m posting here because I think it’s less about Emby and more about my understanding of how Docker compose works and its grammar.

Why does my compose file produce different results?

# Answer
> 1 votes
Your compose file looks mostly correct, but there's a small difference between the `user` specification. In your original `docker run` command, you're using environment variables (--env UID=1001 and --env GID=1001) to set the `user`, while in the docker compose file, you are using the `user` directive directly.

To make your Docker Compose file equivalent to the original `docker run` command, you can add the environment section for setting the **UID** and **GID**. You can try out the below compose file:

```
version: '3.5'

volumes:
  emby-cache:

services:
  embyserver:
    image: emby/embyserver:latest
    container_name: embyserver

    ports:
      - "8096:8096"
      - "8920:8920"

    volumes:
      - /home/udance4ever/emby:/config
      - emby-cache:/cache
      - /home/udance4ever/mnt/userdata/kodi:/userdata/kodi

    devices:
      - /dev/dri:/dev/dri

    environment:
      - UID=1001
      - GID=1001

    restart: 'unless-stopped'

```

After making these changes, try running `docker-compose up -d` again, and it should bring up the Emby server without continuously restarting.

---
Tags: docker, docker-compose, ubuntu
---


# Question
Title: Aws EKS Prometheus helm failed pod CrashLoopBackOff with log "trigger reload: reload request failed: Post \"http://127.0.0.1:9090/-/reload\"
I followed the steps here https://docs.aws.amazon.com/prometheus/latest/userguide/AMP-onboard-ingest-metrics-new-Prometheus.html

In a nutshell, I installed the Prometheus community helm chart on my EKS cluster in AWS https://github.com/prometheus-community/helm-charts.

My Kubernetes version is 1.27

The cluster node group has two t3.large nodes (8gb memory, 2vCPU)

After creation, I have:

Pods:

```
kubectl get pods -n prometheus

NAME                                                        READY   STATUS             RESTARTS         AGE
prometheus-alertmanager-0                            1/1     Running            0                142m
prometheus-kube-state-metrics-69b5df7dc4-kfqc2       1/1     Running            0                142m
prometheus-prometheus-node-exporter-dkksq            1/1     Running            0                142m
prometheus-prometheus-node-exporter-fzr5z            1/1     Running            0                142m
prometheus-prometheus-pushgateway-847c6f4d57-dz5wx   1/1     Running            0                142m
prometheus-server-5fbb47548d-sxm72                   1/2     CrashLoopBackOff   32 (3m58s ago)   142m

```

Pod logs:

```
 kubectl logs prometheus-server-5fbb47548d-sxm72 -n prometheus

....
level=error ts=2023-08-04T17:45:27.636671778Z caller=runutil.go:100 msg="function failed. Retrying in next tick" err="trigger reload: reload request failed: Post \"http://127.0.0.1:9090/-/reload\": dial tcp 127.0.0.1:9090: connect: connection refused"

```

Deployments:

```
kubectl get deployments -n prometheus
NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE
prometheus-kube-state-metrics       1/1     1            1           144m
prometheus-prometheus-pushgateway   1/1     1            1           144m
prometheus-server                   0/1     1            0           144m

```

Describe failed deployment:

```
kubectl describe deployments prometheus-server -n prometheus

...
 Type           Status  Reason
  ----           ------  ------
  Available      False   MinimumReplicasUnavailable
  Progressing    False   ProgressDeadlineExceeded

```

The EBS volumes have successfully bound.

It seems that the prometheus-server pod is stuck in CrashLoopBackOff because it can't connect to itself on 127.0.0.1:9090

I have been searching for an answer all day and trying different things but can't solve the issue.

My helm values looks like this:

```
## The following is a set of default values for prometheus server helm chart which enable remoteWrite to AMP
## For the rest of prometheus helm chart values see: https://github.com/prometheus-community/helm-charts/blob/main/charts/prometheus/values.yaml
##
serviceAccounts:
  server:
    name: amp-iamproxy-ingest-service-account
    annotations:
      eks.amazonaws.com/role-arn: arn:aws:iam:###########:role/service-role/Amazon_EventBridge_Invoke_Api_Destination_###########
server:
  remoteWrite:
    - url: https://aps-workspaces.eu-west-1.amazonaws.com/workspaces/#############/api/v1/remote_write
      sigv4:
        region: eu-west-1
      queue_config:
        max_samples_per_send: 1000
        max_shards: 200
        capacity: 2500

```

I can't figure this out, please help! I'm sure someone else has struggled with the same issue because I can't find much info on this. Why is it failing? What Can I do to fix it? Thanks in advance

# Answer
I've had this issue in the past as well and I ended up noticing that my nodes were not joining the master.

Ensure that your nodes have joined the master and are in a ready state.
> 0 votes

# Answer
kubectl logs \[pod-name\] -c prometheus-server -n prometheus

level=error msg="Failed to apply configuration" err="could not get SigV4 credentials: WebIdentityErr: failed to retrieve credentials\ncaused by: InvalidIdentityToken: No OpenIDConnect provider found in your account for

check whether you missed the openID
> 0 votes

---
Tags: kubernetes, amazon-web-services, prometheus, kubernetes-helm
---


# Question
Title: Why is detect-secrets-hook ignoring my audit selections?
detect-secrets-hook in my pre-commit is alerting on secrets that are already marked as safe to be committed in the baseline.

I'm running detect-secrets version 1.4.0 on Python 3.11.0 on Windows 11.

To reproduce:

1. Have a file in a git repo containing something that will get caught as a secret. Update something in the file so that it needs to be committed.
2. Create/recreate a secrets baseline with `detect-secrets scan > .secrets.baseline`
3. Resave the baseline file as UTF-8, because I'm on Windows.
4. Run `detect-secrets audit .secrets.baseline` and accept that the secret is okay to be in the file. The segment of the baseline file looks like:

```
"results": {
    "xyz\\dev_local_settings.py": [
      {
        "type": "Secret Keyword",
        "filename": "xyz\\dev_local_settings.py",
        "hashed_secret": "62f4d1d18d106c8e6f3c79df832893f4b6e95c8e",
        "is_verified": false,
        "line_number": 26,
        "is_secret": false
      }
    ]
  },

```
5. Have a pre-commit config that looks like this:

```
repos:
-   repo: https://github.com/Yelp/detect-secrets
    rev: v1.4.0
    hooks:
    -   id: detect-secrets
        args: ['--baseline', '.secrets.baseline']
        exclude: package.lock.json

```
6. Attempt to `git commit -a`.

The pre-commit check will fail on the one known secret:

```
Detect secrets...........................................................Failed
- hook id: detect-secrets
- exit code: 1

ERROR: Potential secrets about to be committed to git repo!

Secret Type: Secret Keyword
Location:    xyz/dev_local_settings.py:26

```
7. Another run of `detect-secrets audit .secrets.baseline` yields "Nothing to audit!"

Am I misunderstanding how detect-secrets works? I would expect it to not alert on a secret that was marked as safe in the baseline.

# Answer
I resolved the issue by adding a comment containing `pragma: allowlist secret` in the offending line.

I didn't try it at first, because the issue was in an `appsettings.json` file of an asp.net project and according to Google json can't have comments. However, seems like Microsoft does allow `//` comments in `appsettings.json`.
> 0 votes

---
Tags: security, devsecops
---


# Question
Title: GitLab CI/CD get CI_MERGE_REQUEST_LABELS after merge successfully
I know that the docs say, that the variable `CI_MERGE_REQUEST_LABELS` is available if pipeline is a merge request pipeline and the MR is open.

My dream was it to label my MR's with `patch`, `minor` or `major` labels. When merge the MR, my pipline, for e.g. `master` branch, gets triggered and in the jobs of that pipeline I want to use the label.

In the job I want to do some Version-Number stuff with the label information. We use this for other connected services.

Does someone has an idea how to get access to the labels or has another approach how to get information about the kind of the release (`major`, `minor`, `patch`)?

# Answer
My team had the exact same use-case. Our merge strategy is the following:

1. We use merge trains
2. We squash our commits

And as pre-requisites, you need:

1. A project access token (made available in CI) with at least read api access
2. Scoped labels for the kind of release (obviously)

Based on these conditions, you can solve this problem by inspecting the commit message in your commit stage, extract out the branch name from the commit, and use that branch name to hit the Gitlab API to get the merge request's label.

We have two stages in our gitlab-ci:

```
verify_tags:
  stage: lint
  script:
    - sh build-scripts/verify-merge-request-labels.sh
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'

```

this stage verifies that merge requests have labels. If they don't, they can't be merged. The `verify-merge-request-labels.sh` script is pretty simple:

```
#!/bin/sh
BUMP_VERSION_TYPE=$(echo "$CI_MERGE_REQUEST_LABELS" | sed -E 's/.*bump version::(major|minor|patch).*/\1/')
case $BUMP_VERSION_TYPE in
    major|minor|patch)
        echo "Detected semver part $BUMP_VERSION_TYPE"
        ;;
    *)
        echo "Could not parse semver part out of $CI_MERGE_REQUEST_LABELS. Make sure to add a \`bump version\` label to your merge request."
        exit 1
        ;;
esac

```

The second stage actually gets ran when the branch is getting merged in.

```
increment_version:
  stage: release
  image: python:3.9
  before_script:
    - git remote set-url origin "https://gitlab-ci-token:${RUNNER_TOKEN}@${YOUR_GIT_HOST}/${CI_PROJECT_NAMESPACE}/${CI_PROJECT_NAME}.git"
    - git config --global user.email "${GITLAB_USER_EMAIL}"
    - git config --global user.name "${GITLAB_USER_NAME}"
    - poetry install --only dev
  script:
    - poetry run bumpver update $(python build-scripts/determine-bumpver-semver-flag.py)
    - git push -o ci.skip origin $(git tag) HEAD:${CI_COMMIT_REF_NAME}
  rules:
    - if: '$CI_COMMIT_BRANCH == "main"'

```

The second script is a bit more complex (and long) so I won't share its entirety, but its snippets (in python using `requests`) are as follows:

```
branch_pattern = r"Merge branch '(?P<branch>[^']*)'"
source_branch = re.search(branch_pattern, os.environ["CI_COMMIT_TITLE"]).group("branch")
# Get original merge request corresponding to source branch
query_params = {"source_branch": source_branch}
url = f"{os.environ['CI_API_V4_URL']}/projects/{os.environ['CI_PROJECT_ID']}/merge_requests"

r = requests.get(
    url=url,
    params=query_params,
    headers={"Authorization": f"Bearer {os.environ['RUNNER_TOKEN']}"},
)
labels = r.json()[0]["labels"]

```

The API reference is here
> 1 votes

---
Tags: gitlab, cicd, gitlab-ci-runner, release-management
---


# Question
Title: Best practice for building releases with Jenkins multibranch pipeline
We have started moving from Subversion to git (using Bitbucket, to be exact), and I'm unsure how to manage release builds with the multibranch concept.

This is how we do it in Subversion:

We have one Jenkins job based on a declarative pipeline (Jenkinsfile). It has a parameter `isRelease` that is false by default. When we decide to build a release, a developer starts the job manually setting `isRelease` to true. In the pipeline there is a *Release* step that is run when `isRelease` is true, performing a Maven release.

I've been reading a lot about how to work with git branches, but I wasn't able to make a decision which concept to use for building releases. Jenkins doesn't seem to offer a manual build in multibranch jobs, so it looks like we have to change our approach. As I understand, there are many options, for example:

### (1) Use a second Jenkins job

* create another Jenkins job specifically to manually start building a release

This is close to how we are doing it now, but I feel it's actually not the way it should be done with git.

### (2) Use a release branch

* create a *release* branch
* when we decide to build a release, merge it into the *release* branch
* have the Jenkins pipeline automatically create a release from the *release* branch

The manual step would change from starting a Jenkins job to merging into the *release* branch, which appears to be more 'gittish' to me.

### (3) Use a develop branch

* create a *develop* branch as the common branch during development
* when we decide to build a release, merge it into the *master* branch
* have the Jenkins pipeline automatically create a release from the *master* branch

Similar to (2), just using the master branch differently.

Which option is the best and why? Are there other (better) options?

---

Note that What's the practice to perform a releasing with Git? sounds like a similar question, but it's not about a general approach, but how to solve version conflicts in poms.

# Answer
> 0 votes
I have the exact same question. So sad nobody stopped to think the best option and respond.

For me the more correct one would be to have a separate job. Having a release branch that will end up having the code at the point of the last release (and never ever nobody will read code at that commits seems weird). Maybe have a separate job that will create a specific branch release/1.2.3 adn then create the release from there. After that the tag 1.2.3 is created and the relase/1.2.3 branch is deleted.

---
Tags: jenkins-pipeline, git, release
---


# Question
Title: How can I set an environmental variable with glab (glab-cli)?
I would like to set an environmental variable with `glab`. I don't see the option on `glab ci`. How can I set an environmental variable?

# Answer
This is currently under `glab variable`,

Manage GitLab Project and Group Variables

```
USAGE
  glab variable [flags]

ALIASES
  var

CORE COMMANDS
  delete:     Delete a project or group variable
  export:     Export project or group variables.
  get:        get a project or group variable
  list:       List project or group variables
  set:        Create a new project or group variable
  update:     Update an existing project or group variable

FLAGS
  -R, --repo OWNER/REPO   Select another repository using the OWNER/REPO or `GROUP/NAMESPACE/REPO` format or full URL or git URL

INHERITED FLAGS
  --help   Show help for command

LEARN MORE
  Use 'glab <command> <subcommand> --help' for more information about a command.

```
> 0 votes

---
Tags: gitlab, glab
---


# Question
Title: Does glab (glab-cli) support creating project access tokens?
I would like to make a chore commit in my CI. Currently, `CI_JOB_TOKEN` doesn't allow pushing to the repo. The workaround is to create a project access token with write perms. Is it possible to do this with `glab`?

# Answer
> 0 votes
Not current, there are two issues opened on this

And, this issue will provide a better workaround through prepopulating UI variables

---
Tags: gitlab, glab
---


# Question
Title: What can a token with a role of Guest or Reporter do when given scopes to read and write the repository?
If I create a Project Access Token, or Personal Access Token (PAT) token with a role of Guest, and a scope such that it can read\_repository and write\_repository, what can that token actually do? I'm getting a 403 when I try to use it.

```
$ git clone https://foo:myToken@gitlab.awe.eco.cpanel.net/evancarroll/foo.git
Cloning into 'foo'...
remote: You are not allowed to download code from this project.
fatal: unable to access 'https://acme.net/evancarroll/foo.git/': The requested URL returned error: 403

```

# Answer
**Presumably nothing.**

But there is already an issue which addresses this.

https://gitlab.com/gitlab-org/gitlab/-/issues/407725
> 0 votes

---
Tags: gitlab
---


title: What is a "chore commit"?

text:
# Question
Title: What is a "chore commit"?
I've seen the term "chore commit" used in multiple places, what is a chore commit?

And it's in the commit message here.

> chore(gitattributes): ensured consistent line endings across OSs

What does this mean?

# Answer
A "chore commit" is typically an automatically generated commit made by CI that either lands in a production branch, or in code on a merge-release. A chore can do things like automatically generate and update a ChangeLog, or bump a version based on the content of other commits in the merge request.

You can see it described here in semantic-release's commit analyzer,

> Commits with type 'chore' will not be associated with a release type.

That means a commit with a header of `chore:` will not trigger a release of any type. Which in the case of `semantic-release` is useful because the system itself generates the commit.

Chore commits pushed by themselves should not trigger CI. This typically happens in one of two ways,

* Using something like the `-o ci.skip` push option on GitLab.. That is to say, to tell the CI executor not to run on that commit.
* Using a convention in the the commit message like `chore:` which the CI is contextually aware of and short circuits (skips the run).

## History

This comes from Conventional Commits,

> types other than fix: and feat: are allowed, for example @commitlint/config-conventional (based on the Angular convention) recommends build:, **chore**:, ci:, docs:, style:, refactor:, perf:, test:, and others.

But as an aside, the reference Angular convention did not adopt this term, or the chore commit type, though Karma.js did,

> chore (updating grunt tasks etc; no production code change)
> -1 votes

# Answer
Commit types come from Conventional Commits. The idea is to annotate changes with metadata that describes the type of commit, such as fixing bugs or introducing new features. These commit types can also be tied to Semantic Versioning version numbers and allow for automatically calculating an appropriate version number.

Although the Conventional Commits specification only defines `fix`, `feat`, and methods for specifying breaking changes in the body of the commit message, other types are derived from other projects. For example, the Angular commit message guidelines add `build`, `ci`, `docs`, `perf`, `refactor`, `style`, and `test`. The commitlint tool specifies `chore` as an option.

Since the definition of `chore` is not part of the Conventional Commits specification but takes advantage of its allowances for the specification of other types, it would be up to a project to specify exactly what a `chore` commit is and when it is used.
> -1 votes

---
Tags: gitlab, git
---


# Question
Title: When using a Project Access Token, I'm getting "The requested URL returned error: 403"
When I try to use a project access token, I'm getting

```
$ git clone https://asdf:token@acme.com/evancarroll/foo.git
Cloning into 'foo'...
remote: You are not allowed to download code from this project.
fatal: unable to access 'https://acme.com/evancarroll/foo.git/': The requested URL returned error: 403

```

# Answer
In my case this was because the token I was creating, though it had the `read_repository` and `write_repository` permissions, it was not set to the role of "Developer" or higher.

**Recreating the token with the role of Developer instead of Guest solved the problem for me.**
> 1 votes

---
Tags: gitlab, gitlab-ci
---


# Question
Title: What topics should at least be included in a Java for DevOps course?
I am preparing a course about "Java for DevOps". From a DevOps perspective, what topics should be covered during the meeting?

I was thinking about the following topics:

1. What is DevOps?
2. Why should I learn Java or a programming language that the team is using from a DevOps perspective?
3. Java itself
    * keywords
    * keyword categories
    * OOP
    * FP
    * Hello World

The aim of this question is to get feedback. On one hand I think that these topics will cover "Java for DevOps", but on the other hand I would like to know whether other people agree with this or what topics they would cover if they will organize the course?

The target audience consists of Senior/Medior Quality Assurance, Senior/Medior Operations and some Junior Developers.

**Rationale**

According to Wikipedia, DevOps is the intersection of Development, Operations and Quality Assurance.

Development: creates features, i.e. add business value Operations: ensures that the software runs stable Quality Assurance: tests the software by clicking in the UI for example

DevOps: intersection, i.e. automated, stable, tested software sooner to the customer

As development creates new feature, it would be beneficial for the company and also the developers that Testers and Operations automate more and start to fix bugs, then Development could create more features.

If everything goes well, perhaps some testers and operations could create features in a couple of months/years as well.

# Answer
Even with the rationale you edited in, I don't really see a point.

DevOps is an intersection of Dev, Ops (and more, like Security and maybe Testing) in a cultural sense. But on a technological sense, you will always have people in a mixed team who are good at one of those but not the others, and that is fine.

For example: you might deploy something like Kubernetes or OpenShift; and then you will for sure have ops-sided people who are expert at running the low-level stuff, tuning your NAS, adapting your orchestration to a cloud provider of your choice, or to on-premise bare-metal servers and so on. You will *not*, usually, have all of your developers do the same; they may *use* those things provided by the others. The ops guys might never see a piece of application code at all; and you would still call it "DevOps".

What your devs will be able to do is manage the full lifecycle of their applications, from source code to production, using the tools provided by the ops guys. When one says that devs do jobs that ops used to do in the past, one means that the automation makes it so that the environments (dev, test, int, prod...) are mostly the same; that deploying to them is as automated as humanly possible (without requiring manual ops, and most importantly without requiring a handover of binaries etc. between devs and ops); that there is a complete, unbroken technological chain from source to prod; including things like virtual networks automatically springing into existence and so on. One does *not* mean that devs troubleshoot hardware or networking on a low layer.

The same goes for your ops guys. They will most certainly not try to change the application code proper after a few seminars on it, unless you are talking about really small companies with 3-person teams who need to do so out of sheer necessity.

Note that I am not saying that it is not possible to be one person who does both development and operations work. Is is an absolute gold mine to have such people available; and some companies foster that by, for example, rotating devs into low-level ops groups occasionally. But those guys are not your target audience. By the topics you listed in your question, you are targeting ops guys who have never in their life coded anything ("hello world").

If you actually *want* your ops guys to grow into coding instead of just clicking around in GUIs, modifying or creating config files and such, then here are some suggestions:

* If they are writing the occasional `bash` script already, point out advances constructs to them (loops, functions, error handling, etc.).
* Try to get them interested in some dynamic language like `perl` or `ruby` for a low-hanging fruit, one step above shell scripts.
* Introduce them to `go`, which is really nice for standalone command line applications and especially automation tools. It should satisfy the need to learn something new which should go really well on anyone's resume, and will satisfy the naysayers regarding non-natively compiled languages.

... and so on. But don't try to sell them "Java" as a route into "DevOps".
> 3 votes

# Answer
If you want to do "DevOps Engineering" then you should examine the various tools and APIs that you are going to use with the projects that are being planned.

For example if you are going to use Jenkins, that's a Java based system you could look at that. If you are going to implement microservices you could look at the framework you are going to use and figure out how to do full delivery starting with a product request

On the other hand, if you want to do "Devops Culture" then this is a different approach. The best short explanation of it is Gene Kims 3 ways

To make this kind of cultural transformation you need to see how to map the systems ideas that you have about your organisation (maybe there is some way to map this onto Java Patterns) how to optimise this by identifying waste or feedback loops and lastly how to schedule many different solutions to the problems and opportunities identified. Maybe a Java based approach might help with figuring out what metrics are available

Sorry this isn't a detailed answer but I don't really do Java myself
> 1 votes

# Answer
Here is the list of topics that should be covered:

* Introduction to DevOps
* DevOps tools
* System Integration
* Ansible
* Docker
* Jenkins
* Cloud computing
* Version control with Git
* DevOps implementations
* Maven build lifecycle
* Variables Management
* Creation of Docker file
* Custom images
* Essentials of Cloud Computing

**Installation**

* Installing Git
* Introduction to DevSecOps
* Jenkins plugins
* Linux Administration
* Maven plugins
* Networking
* Source code management with Git

By covering these topics, participants will gain a comprehensive understanding of how to integrate Java applications into DevOps practices effectively.
> 1 votes

# Answer
Preparing a "Java for DevOps" course where students could learn the DevOps culture and how to program with the Java programming language would be a very good idea. The given subjects provide a strong base. These are some of the ways you would structure these topics for your meeting, and further— in some of their expansion—to draw relevance to the DevOps perspective and ensure they are exhaustive.

**Introduction to DevOps:**

* Definition and key principles of DevOps.
* Importance of the mentioned processes, such as collaboration, automation, continuous integration, continuous deployment, and monitoring in the flow of work items in a DevOps approach.

**The Role of Programming in DevOps:**

* Why learning Java or another programming language is critical for DevOps professionals.
* These describe how the programming skills will support automation and script writing for the CI/CD pipelines and Infrastructure as Code, in addition to more tool customization.

**Java Programming Language Overview:**

* Key features of Java and its ecosystem.
* The significance of Java in enterprise environments and its compatibility with various DevOps tools.

**Keywords in Java:**

* Explanation of Java keywords and their roles in controlling program flow.
* Examples of keywords like class, public, static, void, and their usage.

**Keyword Categories:**

* Categorizing Java keywords (e.g., access modifiers, control structures, data types, exception handling keywords).
* Discussing how these categories relate to writing efficient and secure Java applications.

**Object-Oriented Programming (OOP) Concepts:**

* Core OOP principles: encapsulation, inheritance, polymorphism, and abstraction.
* Discussing how OOP principles support the modular, scalable software development that DevOps practices encourage.

**Functional Programming (FP) Concepts:**

* The basics of functional programming in Java.
* How FP concepts—such as lambda expressions and streams—help us make our coding easier and even more productive, especially in collections processing?

**Hello World Program:**

* Writing and explaining a simple "Hello World" program in Java.
* Demonstrates how to compile and run a Java application with enough context to appreciate the background for automated builds and deployments.

If the following are included, then the participants of your "Java for DevOps" course will have a pretty good idea of how the knowledge and skills acquired on Java programming will enable them to increase their impact in a DevOps environment. This course should be complete, covering all the theoretically required stuff; moreover, it should have practical exercises and examples on how to apply Java during automation and optimization of DevOps flows.
> 0 votes

---
Tags: java, training
---


# Question
Title: Is it possible to add a toleration in kube-fledged?
kube-fledged provides `nodeSelector` to select the node you want the image to be cached on. But there are taints on the nodes in my cluster. Would it be possible to add a toleration in the `imagecache` manifest?

# Answer
> 0 votes
It sounds like you need to add tolerations to the `imagepuller` Job created by the kube-flegdged controller. Looking through the code, I don't see any facility for setting that via the Helm chart (there is a `tolerations` setting in `values.yaml`, but that is used to set tolerations on the kube-fledged controller and doesn't get passed through to generated resources).

You could modify the source to add custom tolerations; alternatively, you could look into creating a mutating web hook that would perform the modification when the resources are created.

---
Tags: kubernetes
---


# Question
Title: how to check running containers with containerd
I've switched to containerd in a k8s cluster.

There's 2 pods running on one node, 2 github runners.

I used to check on a host what containers were running with "docker container ls" but how to check this when containerd is used?

```
sudo ctr container ls

```

comes back with nothing

# Answer
Try to namespace to list command

```
ctr -n k8s.io containers list

```

To list available namespaces run

```
ctr ns ls

```
> 14 votes

# Answer
I feel relevant and important to note that:

```
ctr c ls

```

lists **all** containers, i.e. either stopped or running.

In accordance with the question title: to precisely list all **running** containers (in the sense of docker terminology) you need to use this command:

```
ctr task list

```

or the shorthand

```
ctr t ls

```
> 1 votes

---
Tags: docker, kubernetes, containerd
---


# Question
Title: On Azure, how do I run a short-lived Docker container on a schedule?
I have a fairly simple Unix shell script packaged up in an Alpine Linux Docker container hosted on an Azure container registry. A VM runs this script with cron:

```
docker login <snip>
docker pull example.com/bar:latest
docker run  example.com/bar:latest

```

Can I do without the VM and use Azure services instead, perhaps with some sort of scheduler running this on an Azure Container Instance?

My motivation is not wanting to maintain and pay for the VM.

# Answer
> 15 votes
Azure Container Instances (ACI) may be a good option as you suggest. These let you run a container directly on Azure, without having to manage a VM, with per-second billing for the time the container is used.

Although one of the demos on that blog mentions Kubernetes, the idea of ACI is that you can create a container through the Azure CLI with `az container create`, just like on your local workstation with `docker create`.

To create the container, you can use Azure CLI (`az` command, see quick start docs) or Azure Cloud Shell.

You would need to create/run the container on a schedule from somewhere else - Azure Functions might be a good place to run the "container create" command from a scheduled function. This supports bash, PowerShell, and other languages \- all running on Windows.

If you want to keep using Docker containers without running VMs or learning Kubernetes, this might be a good option.

Alternatively, you could move all your code into Azure Functions, but that's a bigger decision.

**Update:** Jan 2019 - Azure Logic Apps can be used to run scheduled tasks as well. This replaced Azure Scheduler in Jan 2022.

# Answer
> 5 votes
For an alternative approach, I would investigate Azure functions:

No VM continually running.

# Answer
> 2 votes
A scheduled devops pipeline is an easy and free way to run azure cli tasks on a recurring basis.

https://docs.microsoft.com/en-us/azure/devops/pipelines/process/scheduled-triggers?view=azure-devops&tabs=yaml

# Answer
> 0 votes
I haven't done this myself, but I wonder if an Azure Container Registry (ACR) task could be used to trigger a container on a schedule?

This ACR tutorial seems to suggest that you can schedule a container run as follows:

```
az acr task create --name timertask --registry $ACR_NAME --cmd mcr.microsoft.com/hello-world --schedule "0 21 * * *" --context /dev/null

```

---
Tags: docker, containers, azure
---


# Question
Title: How does $CI_PIPELINE_SOURCE == 'merge_request_event' work?
I have in my gitlab.yml file,

```
rules:
  - if: $CI_PIPELINE_SOURCE == 'merge_request_event'

```

Before adding that my pipeline was working fine. I just wanted to make it such that the job above was *only* run on `merge_request_event`. Now that I added that, the *behavior* seems to change. How is this rule implemented?

# Answer
> 0 votes
This is because despite looking like a conditional with an `if:` statement,

```
rules:
  - if: $CI_PIPELINE_SOURCE == 'merge_request_event'

```

This **actually does change the behavior**, it enables "Merge Request Pipelines" by satisfying the criteria,.

> Your project’s .gitlab-ci.yml file must be configured with jobs that run in merge request pipelines. To do this, you can use:
>
> * rules.
> * only/except.

Assume we have a repo `foo` and we fork it into `bar`. If we open a merge request (adding a new file) from `bar` to `foo`, you'll see the same high level overview,

But adding the rule `$CI_PIPELINE_SOURCE == 'merge_request_event'` causes a few different things to happen,

* The pipeline is no longer a "Push Pipeline" it's a p"Merge Request Pipeline"\](https://docs.gitlab.com/ee/ci/pipelines/merge\_request\_pipelines.html) and it has a different tag.
* The pipeline is no longer run on the fork, but from the "base" project
* The commit the pipeline runs on is no longer run on the fork, but from `refs/merge_requests` in the "base" project.

I believe this gives a better idea of how this process works, but you should read the full official documentation page on "Merge Request Pipelines" to compliment any understanding this gives.

# Without *"Merge Request Pipelines"*

However, everything is different from this point on. When you're **NOT** using "Merged Request Pipelines", the merge request page will link to the pipeline like this,

Note *"Pipeline"* here is more correctly "Push Pipeline". If you click on that number, the pipeline itself will show under `bar` **the fork** for example under this path `bar/-/pipelines/455884`

And it will show the job,

If you dump the environment from that job you'll see

```
CI_PIPELINE_SOURCE=push

```

And this is true. The pipeline **is running on the branch** (in this case `foobar`), and that can be seen on `bar/-/tree/foobar`,

# With *"Merge Request Pipelines"*

Now let's add the configuration and rerun,

```
rules:
  - if: $CI_PIPELINE_SOURCE == 'merge_request_event'

```

Now you'll see this on the merge request sitting on `foo`,

Immediately take note to it saying *"Merge request pipeline"* rather than *"Pipeline"*. When you click on that pipeline number #455999 you'll be taken **NOT** to bar's pipelines, as you were above, but instead to foo's pipelines, `foo/-/pipelines/455999` and here you'll see this summary which has the tag `merge_request`.

Clicking on the job's tab, you'll be taken to foo's jobs, `foo/-/pipelines/455999/builds` and you'll see this,

And that's showing you that the merge request no longer comes from the branch `foobar` in reality, it's pulled foobar down to the git merge request directory `foo/-/commits/refs/merge-requests/1/head` and the merge request now points to that, which resembles the commit **AFTER** it has merged.

---

I posted this comment on the GitLab Rule Epic to let them know my thoughts on this

---
Tags: gitlab, gitlab-ci
---


# Question
Title: How can I use `glab api` commands to configure settings that don't have a developed UI?
According to the documentation on Merge Request Pipelines,

> To prevent users from running new pipelines for fork projects in the parent project use the projects API to disable the `ci_allow_fork_pipelines_to_run_in_parent_project` setting.
>
> > Pipelines created before the setting was disabled are not affected and continue to run. If you rerun a job in an older pipeline, the job uses the same context as when the pipeline was originally created.

How can I do that with glab? There is no example of setting REST parameters `glab api --help`.

# Answer
> 0 votes
The right syntax to set this attribute for a project `evancarroll/foo` is,

```
glab api "projects/evancarroll%2ffoo" \
  -X PUT \
  -f "ci_allow_fork_pipelines_to_run_in_parent_project=false"

```

Note the project here is `evancarroll/foo` but you must url encode the `/` with `%2f`.

---
Tags: gitlab, glab
---


title: Jenkins Health Check URL

text:
# Question
Title: Jenkins Health Check URL
I'm planning to supervise services by periodically checking if they are up and running, Jenkins is our central orchestration tool and must be running with near zero downtime.

How can I check health status of a Jenkins server - **By health status I mean service is up and web app application is operational** ?

Does Jenkins provide a native health check URL ? Or does I need to make additional scripts? if so what factors should be included in beside checking `jenkins.service` and curl web app home page.

# Answer
The complication reside in the definition of the health status (and its usefulness).

You mentioned the definition being `service is up and web app application is operational` and for that indeed checking the homepage *should* suffice.

However that doesn't mean the service is properly configured for and correctly performing all the jobs that it's supposed to be doing. Without a consistent, universally acceptable definition of a health status there is no point of providing such information as a single, one-stop location to check.

But Jenkins offers via its Remote access API various checking points that users can combine to derive their interpretation of the overall service health status. For example maybe checking the status of all the configured jobs or at least that of the critical ones would be more useful, which could be done using the `.../job_id/lastSuccessfulBuild/api/` or `.../job_id/api/` checkpoints.

The JenkinsAPI could also be of interest (better docs at least), found via Where can I find jenkins restful api reference?
> 9 votes

# Answer
Check the login page, e.g. http://localhost:8080/login

Most pages are password protected but not this one (or you couldn't log in).
> 16 votes

# Answer
Check the logo of jenkins is accessible from outside.

For ex. `http://localhost:8080/static/<anystring>/images/svgs/logo.svg`

This will less load to the server than login page.
> 0 votes

# Answer
`/api/json` will return 2.7 kB. This is smaller than the login page, the `logo.svg`, and `favicon.svg`. If Jenkins is responding but not ready for operation, then it returns HTML.
> 0 votes

---
Tags: jenkins
---


# Question
Title: Run Ansible win_shell without double quotes
Below is my script

`win_shell: 'C:\\TEMP\\myscript.ps1 -password {{ password }}'`

But when it executes, it get executed as below and as there is a dollar ($) in password its not working properly.

`"cmd": "C:\\\\TEMP\\\\myscript.ps1 -password abc$efg"`

I want the script to be executed either as

`"cmd": C:\\\\TEMP\\\\myscript.ps1 -password abc$efg,`

or as

`"cmd": 'C:\\\\TEMP\\\\myscript.ps1 -password abc$efg',`

Any hits?

# Answer
Use regex\_escape filter from Ansible. It will help you escape the special character `$` in your string.

```
win_shell: 'C:\\TEMP\\myscript.ps1 -password {{ password | regex_escape }}'

```

Ref: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/regex\_escape\_filter.html
> 1 votes

# Answer
If you have to do this, I would suggest using win\_template and copying the (file with the password in it) to the operating system and running it directly.

Do it in a `block` and shred the file in an `always` section so you can be pretty sure you've trashed that credential when you're done.

Don't copy it to c:\temp either, but copy it to your ansible users's local temp dir so at least it's not necessarily world readable.
> 0 votes

---
Tags: ansible
---


# Question
Title: Ignore 'host not found in upstream' in docker swarm environment
I have a problem with nginx configuration in docker swarm environment. I would like to set the **nginx** service used as a **reverse proxy** to make the dockerized services communicate, everything works correctly but I have a problem that occurs in case a service fails to start.

If one of the services is unable to start, the nginx service consequently fails to start as it is unable to detect the upstream associated with the service. nginx logs:

> Starting nginx ...
>
> reverse-proxy-stack\_nginx-reverse-proxy.1.lae48wy4me1d@ip-172-31-23-236 | nginx: \[emerg\] host not found in upstream "uptime-kuma" in /data/nginx/proxy\_host/5.conf:54

Since many services are hosted inside the swarm, and nginx allows them to communicate with the outside, I would like to avoid that if a service fails to start nginx also remains blocked, and I would therefore like it to still be able to work correctly for the other services.

I'm serving nginx via the image **jc21/nginx-proxy-manager:2.10.4** which also offers a GUI to set options for different hosts. I would like to know if there is an option that allows me to still run nginx even if one of these upstreams is not found

# Answer
When nginx start up it scans its config one time and resolves any static dns entries for servers using the system configured dns. If entries fail, nginx abandons them.

However, you can configure dynamic dns entries by making them variables. This causes nginx to resolve them at query time, using its configured resolver settings.

This means you need to configure a resolver (`127.0.0.11` for docker, or `kube-dns.kube-system.svc.cluster.local` for K8s), and configure any upstream servers by assigning the hostname to a variable and using the variable rather than hostname directly.
> 1 votes

---
Tags: docker, networking, nginx, docker-swarm
---


# Question
Title: Why i cant include a .yml in CI/CD, using rules with mine variables?
Why my pipeline doesn't work? I just want include this .yml if "NATIVE\_COMPILE" is "true".

```
variables:
  NATIVE_COMPILE : "true"
include:
  - project: 'pipelines/cicd'
    ref: improve/simplificando-pipes
    file: 'libraries/bundle/java/maven/.gitlab-ci-native.template.yml'
    rules:
      - if: $NATIVE_COMPILE == "true"

```

If i use something like:

```
include:
  - project: 'pipelines/cicd'
    ref: improve/simplificando-pipes
    file: 'libraries/bundle/java/maven/.gitlab-ci-native.template.yml'
    rules:
      - if $CI_COMMIT_MESSAGE =~ /.*--native-compile*/

```

And write "--native-compile" on commit, its works perfectly, but with my variables doesnt work.

Someone can help me, pls ?

# Answer
> 0 votes
Add the variable as a global project variable so that you can get its value in the linked yml file.

Settings -\> CI/CD -\> Variables

---
Tags: gitlab, cicd, yaml
---


# Question
Title: Concat multiple meta labels on prometheus for relabling
How can I change the value of the label `__param_target` to resemble how I would interact with a Kubernetes service in-cluster, following the format: `http://serviceName.namespace.svc:servicePort/path` ?

I am aware of the necessary meta labels for this task, but I am unsure how to relabel it using serviceMonitor. The meta labels I intend to use are:

```
__scheme__
__meta_kubernetes_service_name
__meta_kubernetes_namespace
__meta_kubernetes_pod_container_port_number

```

the path can be hardcoded since it is not something that can be on the meta labels :) but how can it be concatenated

# Answer
So i figured it out

```
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: blackbox-exporter-test
  namespace: monitoring
  labels:
    prometheus: apps
spec:
  endpoints:
  - interval: 15s
    targetPort: 3000
    path: /probe
    scrapeTimeout: 10s
    params:
      module: [http_2xx]
    relabelings:
    - sourceLabels: [__scheme__,__meta_kubernetes_service_name]
      targetLabel: __param_target
      separator: ://
    - sourceLabels: [__param_target, __meta_kubernetes_namespace]
      targetLabel: __param_target
      separator: .
    - sourceLabels: [__address__]
      targetLabel: svclabel
      replacement: "svc.cluster.local"
    - sourceLabels: [__param_target, svclabel]
      targetLabel: __param_target
      separator: .
    - sourceLabels: [__param_target, __meta_kubernetes_pod_container_port_number]
      targetLabel: __param_target
      separator: ':'
    - sourceLabels: [__address__]
      targetLabel: path
      replacement: "zorkiness"
    - sourceLabels: [__param_target, path]
      targetLabel: __param_target
      separator: /
    - sourceLabels: [__param_target]
      targetLabel: instance
    - targetLabel: __address__
      replacement:  prometheus-blackbox-exporter:9115
  selector:
    matchLabels:
      probe: "blackbox" # monitor the services only with this label
  namespaceSelector:
    matchNames:
      - zorki

```

so basically it rewrite the \__probe\_target to

```
http://serviceName.namespace.svc:servicePort/path

```
> 0 votes

---
Tags: kubernetes, monitoring, prometheus
---


title: get rid of PodSecurity warnings

text:
# Question
Title: get rid of PodSecurity warnings
I often use curl image to test out connectivity inside clusters:

`kubectl --kubeconfig ./kubeconfig run curly --image=curlimages/curl --rm -it --restart=Never -- /bin/sh`

However, with k8s v1.27+ a warning is emitted:

```
Warning: would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false
(container "curly" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "curly" must set securityContext.capabilities.drop=["ALL"]),
runAsNonRoot != true (pod or container "curly" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "curly" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")

```

I cant create a pod.yaml to set all these values as the pod needs to be interactive so how can I get rid of this warning?

# Answer
> 1 votes
maybe overrides can be helpful here

```
kubectl --kubeconfig ./kubeconfig run curly --image=curlimages/curl --rm -it --restart=Never --overrides='
{
  "apiVersion": "v1",
  "spec": {
    "securityContext": {
      "allowPrivilegeEscalation": false,
      "runAsNonRoot": true
    }
  }
}' -- /bin/sh

```

---
Tags: kubernetes
---


# Question
Title: kube fleged imagePullSecrets not working(ImagePullBackOff)
```

apiVersion: kubefledged.io/v1alpha2
kind: ImageCache
metadata:
  name: imagecache1
  namespace: kube-fledged
  labels:
    app: kubefledged
    component: imagecache
spec:
  imagePullSecrets:
    - name: regcred
  cacheSpec:
  - images:
    - exampleregdev.azurecr.io/vm-images/sample-img-1:0.0.1
    - exampleregdev.azurecr.io/vm-images/sample-img-2:0.0.3
    - exampleregdev.azurecr.io/vm-images/sample-img-3:0.0.1

```

I am using aregred secret to pull images from a private registry. But I get an image ImagePullBackOff error. I have my regred in the kube-fledged namespace.

# Answer
There are several debugging tasks you should perform check if you saved the pull secret correctly

```
kubectl get secrets -n kube-fledged regcred

```

if the type is `kubernetes.io/dockerconfigjson` it's okay after that, try to decode the secret and use it on your own machine to see if it pulls the image.

let us know if it helped you or you need further assistant :)
> 0 votes

---
Tags: kubernetes, azure-devops, azure
---


# Question
Title: Measuring image pull latency in k8s cluster
I maintain a Kubernetes clusters that pulls a number of images (mostly from Docker Hub, but also some from GCR and GHCR). I am interested in building a dashboard to measure the amount of time it takes for these images to be pulled (to determine how much this is contributing to pod start latencies). Is anyone aware of means to measure this latency using Prometheus / specific exporters?

# Answer
> 2 votes
**For Kubernetes \>=1.30**

Kubelet now exports a histogram, `image_pull_duration_seconds`, for exactly this case. See kubernetes/kubernetes#121719 for details.

---
Tags: docker, kubernetes, prometheus
---


title: No targets in Prometheus at Kubernetes

text:
# Question
Title: No targets in Prometheus at Kubernetes
Why are there no targets at my Prometheus? I did everything like in the Prometheus documentation. Also, I have pods with the label: "name=cadvisor".

I have the following configuration file:

```
# prometheus-config.yml

apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus
data:
  prometheus.yml: |
    global:
      scrape_interval: 5s
    scrape_configs:
      - job_name: "kubernetes-pods"
        kubernetes_sd_configs:
          - role: "pod"
            selectors:
              - role: "pod"
                label: "name=cadvisor"

```

# Answer
Please elaborate on how you deploy Prometheus to the cluster Kube-prometheus-stack ? default chart ?

with kube-prometheus-stack it is very easy to deploy external configuration.

also using prometheus operator you can use podMonitor/serviceMonitor.
> 0 votes

---
Tags: kubernetes, prometheus, grafana
---


# Question
Title: How can I push to a repo when CI_JOB_TOKEN doesn't permit it?
Currently, `CI_JOB_TOKEN` doesn't allow you to push to a repo. When I try I get,

> remote: You are not allowed to upload code.

How can I work around this if I need to push to the repo in the CI?

The `CI_REPOSITORY_URL` which has the `CI_JOB_TOKEN` can not be used.

# Answer
One method to do this is to generate a Project Access Token and store it as a protected environmental variable. You can do this like this,

* From the project page, add `/-/settings/access_tokens` to the end of the URL
* You need to create a token with

Now you need to set this this token up in the project,

* With the `glab` tool use a command like this

```
glab variable set --protected REPO_WRITE_TOKEN myToken

```
* With the UI

+ From the project page, add `/-/settings/ci_cd` to the end of the URL
    + Under Variables click Expand
    + Click "Add Variable"

Now all of your jobs have access to `REPO_WRITE_TOKEN` and can push up using,

```
git remote add merge_request "https://does_not_mattter:${REPO_WRITE_TOKEN}@${CI_SERVER_HOST}/${CI_PROJECT_PATH}.git"

```

---

Unfortunately, the Project Access Token UI page can't be pre-populated yet with query parameters and you can't currently make a Project Access Token from `glab` (the GitLab CLI tool)
> 1 votes

# Answer
Sorry for long read... You could try that solution: https://docs.gitlab.com/ee/ci/docker/authenticate\_registry.html#option-3-use-docker\_auth\_config.

That authentication approach can be defined in several ways:

> * In pre\_build\_script in the runner configuration file.
> * In before\_script.
> * In script.

1. Set DOCKER\_AUTH\_CONFIG CI\CD variable on project\group\instance level (don't forget to check Expand option). Once you have DOCKER\_AUTH\_CONFIG set up correctly, Docker commands like docker push or docker pull in your GitLab CI/CD pipeline will be able to authenticate with the registry automatically.
2. Set variables REGISTRY\_PULL\_AUTH\_BASE64 and REGISTRY\_URL. REGISTRY\_PULL\_AUTH\_BASE64 should contain base64 encoded login:password. Or use direct values registry.example.com and encoded login:password.

```
echo -n 'login:password' | base64
bG9naW46cGFzc3dvcmQ=

```

DOCKER\_AUTH\_CONFIG with nested variables.

```
{
    "auths": {
        "$REGISTRY_URL": {
            "auth": "$REGISTRY_PULL_AUTH_BASE64"
        },
        "$REGISTRY_URL": {
            "auth": "$REGISTRY_PUSH_AUTH_BASE64"
        }
    }
}

```

That's it. During pipeline execution variable will authenticate in docker automatically.

.gitlab-ci.yml example before\_script and script blocks:

```
before_script:
  - docker info

build:
  script:
    - docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG .
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG

```
> 0 votes

---
Tags: gitlab, gitlab-ci
---


# Question
Title: How to implement docker image update on edge device not connected to the Internet
We need to deploy docker container on edge devices which wont be having Internet. These device occasionally connect to network and one of the device (lets call it H) on the network will have internet access. So, I want to know how we can update docker containers in such scenario. I imagine following two approaches:

1. Create tar of image. Copy it to edge device (say over USB) and then update the image on the edge device.
2. Create local registry on device H. Pull the updated image on this device. Make edge device pull only updated layers from this local registry on H.

**Q1.** Machine H has x86 architecture and edge device is runs arm architecture. Running docker pull command on H wont pull arm image from public docker registry. How can we pull arm image on x86 machine? So that we can follow either of two approaches?

**Q2.** Our docker image tar is of 300 MB and it may grow. So we will require to copy whole 300 MB image tar from device H to edge device. I believe we can create tar only of whole image. Is it, by any chance possible to create tar of only updated layers or reduce the size of tar by any means? (Our image is minimal built by copying only bindaries to scratch docker image)

**Q3.** Is there any other better approach? What is usually followed in the industry?

# Answer
> 0 votes
Q1: you cannot pull images on the x86 machine which aren't x86 compatible, however you can host your local Docker registry to which you can push different arch compatible images. Also you don't have to use Docker client to do it, you can use skopeo (example).

Q2: doing `docker pull --platform=arm64 nginx:stable-alpine3.17-slim` gave me an image for a different architecture i run on (amd64): `https://paste.debian.net/1312387/` so you can do that

Q3: You can use automation like Ansible to make the deployment faster, so you'd convert each step you do manually to a play which you have to run each time you run an update. You can also use some CI/CD like GitLab CI or Jenkins (there are many options available, I mention them because I know them well) where you'd write a pipeline which does these steps for you. Out of these two the first option sounds more suitable.

---
Tags: docker, azure-devops, dockerfile, docker-registry, dockerhub
---


# Question
Title: How to manage lxd server configuration by ansible?
I am trying to set up the lxd global, server settings with ansible. Note, these are *not* the instance or container settings, these are the global settings of the lxd daemon. For example, `core.https_address`:

```
# lxc config show
config:
  core.https_address: 0.0.0.0:8443

```

How can I modify them with the super idempotent way by ansible modules? My impression about the docs of the relevant modules (`community.general.lxd_container`, `community.general.lxd_profile`, `community.general.lxc_container`, `community.general.lxd_project`) that they are doing everything, except this.

# Answer
> 0 votes
I started to develop a python module for the task. It had not been a big job, but honestly I only want to change some values yet, so I finally shortcutted it with this workaround:

```
- name: lxc config set core.https_address "0.0.0.0:8443"
  ansible.builtin.shell: if [ "$(lxc config get core.https_address)" = "0.0.0.0:8443" ]; then exit 0; else lxc config set core.https_address "0.0.0.0:8443" && exit 255; fi
  register: script_res
  changed_when: script_res.rc == 255
  failed_when: script_res.rc not in [0, 255]

```

Idea is that I substitute it with a singleline shellscript. Obviously that is quite a suboptimal solution.

---
Tags: ansible, containers, infrastructure-as-code
---