# Question
Title: How to get the IDs of Docker Containers that belong to a Service, given the Service name or id
**Background**

I'm trying to maintain some websites running in a Docker Swarm. I want to do security updates on them.

In a non-Docker environment, like a VPS, my simple process is that I would connect in and git pull updates and run any database updates associated with these.

I want to do the same with websites that are running in a Docker Swarm as Docker Containers that belong to a respective Docker Service.

But it seems to be a disproportionate of effort to find the Docker commands to help me with this.

**Specifically my question is:**

**If I know the Docker Service name or ID, what Docker command can I run to find out the IDs of all the containers to be able to connect into them.**

The reason why I start with the Docker Service when I list them on the Docker master node, using `docker service ls` I get a "user friendly" list of services, which helps me pick out the particular service for the site to maintain, e.g.:

```
    $ docker service ls
    ID                  NAME                       MODE                REPLICAS            IMAGE                      PORTS
    sr2w12kl8bg1        site1_db                    replicated          1/1                 animage2/mariadb:latest        *:13306->3306/tcp
    e083ij8abje1        site1_drupal                replicated          1/1                 animage2/drupal:d7             *:7150->80/tcp
    sejb0f2mfo22        site1_drush                 replicated          0/1                 drush/drush:latest
    kbae02doa8rs        site1_haproxy               replicated          2/2                 haproxy:alpine
    khgmnu1566rb        site2_db                    replicated          1/1                 mariadb:latest
    notoc1fam1gk        site3_drupal                replicated          1/1                 animage1/drupal:d8             *:7160->80/tcp
    0xtxx4jmmwa3        site4_db                    replicated          1/1                 mariadb:latest

etc...

```

I can look at this list above, and pick out a line and say "that's the site".

**So here's the blocker:**

Having picked out a line, I've got an ID, say `e083ij8abje1` for a service running Drupal.

Now I want to ssh into a Docker container inside that Docker Service to do the upgrade.

There's this command for ssh-ing into a Docker container:

```
docker exec -it <container_id> /bin/bash

```

Where container is a Hexadecimal number e.g. `a1b2c3d4ee8` so it would e.g. look like `docker exec -it a1b2c3d4ee8 /bin/bash`

But I don't have the Docker container ID, I have the Docker Service ID which is an alphanumeric number, as in the above example `e083ij8abje1` \- this won't work for the above docker exec command.

So I need a docker command that lists the Docker Container IDs that belong to a Docker Service.

To my surprise, there doesn't seem to be one!

I know I can also list the Docker Containers

```
$ docker ps
CONTAINER ID        IMAGE                  COMMAND                  CREATED             STATUS              PORTS                    NAMES
d5d45a94d45e        haproxy:alpine         "/docker-entrypoint.…"   3 days ago          Up 3 days                                    db_proxy_haproxy.2.91aw0apofd3iw537d249pyi62
6a4676df2b3a        22dbb7890585           "docker-entrypoint.s…"   3 days ago          Up 3 days           3306/tcp                 site2_db.1.xxizdccg9a2hgf83tcgg1tkaej
538172a86a57        site1/drupal:asite      "docker-php-entrypoi…"  3 days ago          Up 3 days           80/tcp                   site3.1.uofsfssl72aj867r0hfs86i7m24linae8
2d487ea3788e        22dbb7890585           "docker-entrypoint.s…"   3 days ago          Up 3 days           3306/tcp                 site4_db.1.3lnd0f9sl027f0skgza3kq373c0
d671069584d7        mariadb:latest         "docker-entrypoint.s…"   3 days ago          Up 3 days           3306/tcp                 site5.1.xoy0r6720362hlkhksu2qd
beb6bcb66f43        mariadb:latest         "docker-entrypoint.s…"   3 days ago          Up 3 days           3306/tcp                 site2.1.kobk423624kljfs1uhpflfs3

```

But this doesn't tell me what Service a Docker Container is running in. Neither command gives me the full information and that's frustrating.

**Research done already**

* How to get container id of a docker service Various answers that don't seem to work for me and they are convoluted not simple commands, I'm using Docker 19.03
* Swarm mode: getting Container ID from Task ID Nothing definitive here stands out. And they've introduced the idea of "Task" - what is that in relation to Docker, Service, Containe?
* I also found several similar question / answers that were just wordy and pointing to the general documentation. Not very helpful as they don't show by example.

I'm trying my best to understand these concepts and have drawn a diagram below. ( Please confirm that Docker Containers belong to a Service. )

Yes, I understand that Docker Swarm is declarative and can allocate Containers where it decides are best, in terms of running them on a particular node. That seems to be the approach encountered whenever I've been trying to find this answer, answers seem to want to give too much information for the reader, when what the reader needs is specific practical examples, not loads of theory and concepts.

So to summarise what I am looking for is a simple command or 2 that enables me to:

* list the Docker Containers by ID that belong to a Service, given the Service ID / name to start with
* for a given Docker Container and ID, find out what Docker Service it belongs to.

I am aware about Docker Swarm end-of-life so don't want answers telling me to use something else. please.

# Answer
As a little addition to such extensive answer, with `docker ps` you can `--filter` containers by label. One of the labels in swarm is a service name, so:

```
docker ps --filter "label=com.docker.swarm.service.name=<service_name>"

```

Should give you service containers on current node. To find more labels for possible filters hit

```
docker  ps --format "table {{.Names}}\t{{ .Labels }}"

```
> 1 votes

# Answer
Answering my own question & accepting it because it works - this is the solution:

**Step 1:** So I start by finding the Docker Service I recognise as my Drupal site "that's the site"

```
node1 $ docker service ls

```

As before, shown in the question, the output of that command is:

```
ID                  NAME                       MODE                REPLICAS            IMAGE                      PORTS
sr2w12kl8bg1        site1_db                    replicated          1/1                 animage2/mariadb:latest        *:13306->3306/tcp
e083ij8abje1        site1_drupal                replicated          1/1                 animage2/drupal:d7             *:7150->80/tcp
sejb0f2mfo22        site1_drush                 replicated          0/1                 drush/drush:latest
kbae02doa8rs        site1_haproxy               replicated          2/2                 haproxy:alpine
khgmnu1566rb        site2_db                    replicated          1/1                 mariadb:latest
notoc1fam1gk        site3_drupal                replicated          1/1                 animage1/drupal:d8             *:7160->80/tcp
0xtxx4jmmwa3        site4_db                    replicated          1/1                 mariadb:latest

```

I pick out the site `animage1/drupal:d8`, second from bottom in the list above and find the corresponding Service ID, `notoc1fam1gk`

**Step 2:**

I use this, Service ID, `notoc1fam1gk`, to list all the tasks for that Service, by doing this command

```
node1 $ docker service ps notoc1fam1gk

```

The output of that command is:

```
ID                  NAME                 IMAGE                      NODE                DESIRED STATE       CURRENT STATE          ERROR               PORTS
yiofs723gc89        site3_drupal.1       animage1/drupal:d8         node2               Running             Running 6 days ago
pgdgf783ff9v         \_ site3_drupal.1   animage1/drupal:d8         node2               Shutdown            Shutdown 2 weeks ago
1gfdsgcb0p2q         \_ site3_drupal.1   animage1/drupal:d8         node2               Shutdown            Shutdown 2 weeks ago
175lfds2jbjy         \_ site3_drupal.1   animage1/drupal:d8         node2               Shutdown            Shutdown 2 weeks ago
qo312e143p1q         \_ site3_drupal.1   animage1/drupal:d8         node2               Shutdown            Shutdown 3 weeks ago

```

**Step 3:**

Now for the crucial bit - I still don't have a Container ID to be able to ssh into.

So This command will do it for me:

```
docker inspect -f "{{.Status.ContainerStatus.ContainerID}}" <task_id>

```

( Credit: https://stackoverflow.com/a/42465558/227926 )

In my example, this looks like:

```
node1 $ docker inspect -f "{{.Status.ContainerStatus.ContainerID}}" yiofs723gc89

```

The output of the above command is just:

```
72d6a20c3d15dd9f0db4b1a60d93a91c0514946881f2dc00ca62d6a7a9ef1290

```

This is the container ID in the format of the 64 character SHA-256 ID.

But I'm familiar with the 12 character length Container IDs that docker ps shows.

The explaination is that the 12 character Container ID is just a truncated version of the full length 64 character ID - it takes the first 12 characters.

So in my case, `72d6a20c3d15dd9f0db4b1a60d93a91c0514946881f2dc00ca62d6a7a9ef1290` will be `72d6a20c3d15` in the 12 character format.

**Step 4:**

So now, when I docker ps on node1 to go looking for that container ID I can use the 12 character format for the ID, `72d6a20c3d15`

I'm on my Docker Swarm Node node1 and I do a docker ps:

```
node1 $ docker ps
CONTAINER ID        IMAGE                  COMMAND                  CREATED             STATUS              PORTS                    NAMES
d5d45a94d45e        haproxy:alpine         "/docker-entrypoint.…"   3 days ago          Up 3 days                                    db_proxy_haproxy.2.91aw0apofd3iw537d249pyi62
6a4676df2b3a        22dbb7890585           "docker-entrypoint.s…"   3 days ago          Up 3 days           3306/tcp                 site2_db.1.xxizdccg9a2hgf83tcgg1tkaej
538172a86a57        site1/drupal:asite     "docker-php-entrypoi…"   3 days ago          Up 3 days           80/tcp                   site3.1.uofsfssl72aj867r0hfs86i7m24linae8
2d487ea3788e        22dbb7890585           "docker-entrypoint.s…"   3 days ago          Up 3 days           3306/tcp                 site4_db.1.3lnd0f9sl027f0skgza3kq373c0
d671069584d7        mariadb:latest         "docker-entrypoint.s…"   3 days ago          Up 3 days           3306/tcp                 site5.1.xoy0r6720362hlkhksu2qd
beb6bcb66f43        mariadb:latest         "docker-entrypoint.s…"   3 days ago          Up 3 days           3306/tcp                 site2.1.kobk423624kljfs1uhpflfs3

```

But `72d6a20c3d15` isn't in the list. Why is that?

Well, in my case it is on the node2 of my Docker Swarm. How do I know this?

I know because the earlier command `docker service ps` `notoc1fam1gk` listed the task as node2, so I know that this task's corresponding container will be on node2.

**Step 5:**

So I connect to node2 where my container is located

```
node1$ ssh node2

```

Then list the Docker Containers, by ID. And I can see `72d6a20c3d15` in this list, bottom entry:

```
node2$ docker ps

CONTAINER ID        IMAGE                      COMMAND                  CREATED             STATUS              PORTS               NAMES
2e3bc7158d61        89up/drupal:unity          "docker-php-entrypoi…"   7 days ago          Up 7 days           80/tcp              someothersite_drupal.1.tg33od888aj6krqqhdx0tp8ta
53122123d035        haproxy:alpine             "/docker-entrypoint.…"   7 days ago          Up 7 days                               someothersite_haproxy.1.t1x7exq98huu8q3aaok57d22
9348345f84cb        haproxy:alpine             "/docker-entrypoint.…"   10 days ago         Up 10 days                              something-m_haproxy.2.02haarnwe1ssfo0x9rypx2yud
72d6a20c3d15        animage1/drupal:d8         "docker-entrypoint.s…"   2 weeks ago         Up 2 weeks          80/tcp              site3_drupal.1.5fqfizaxt0plh1tzt6caq0zon

```

**Step 6:**

I can now ssh into that container:

```
node2$ docker exec -it 72d6a20c3d15 /bin/bash

```

Triumph!

(In your case, your Container ID may be on your node1, or whatever. I've only added this extra step for node2 to illustrate the point that one should be aware of the node that the task (and corresponding container is running on))
> 1 votes

# Answer
```
# make a bash script called get-id-from-service.sh

CONTAINERID=$(docker ps -f name=$1 --quiet);

echo $CONTAINERID;

```

Make it executalbe:

```
chmod +x get-id-from-service.sh

```

Usage:

```
./get-id-from-service.sh <SERVICE_NAME_HERE>

```
> 0 votes

---
Tags: docker, containers, docker-swarm, ssh
---


# Question
Title: Gitea/GitHub Actions maven deploy causing 401 unauthorized
I'm new to CI/CD, Gitea/GitHub, Docker and Maven. I've setup a self-hosted Gitea server and Runner. Before migrating all my projects, I'm testing CI/CD. For this purpose I've made test projects and now facing a maven deploy issue.

Here is the test project `pom.xml` (note : I've replaced real domain with `"example"`) :

```
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.example.test</groupId>
    <artifactId>test-library-java</artifactId>
    <version>1.0.0</version>
    <packaging>jar</packaging>
    <name>TestLibraryJava</name>
    <description>
        Test library for gitea actions deploy
    </description>
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>1.7</maven.compiler.source>
        <maven.compiler.target>1.7</maven.compiler.target>
    </properties>

    <repositories>
        <repository>
        <id>example</id>
            <url>https://git.example.com/api/packages/example/maven</url>
        </repository>
    </repositories>
    <distributionManagement>
        <repository>
            <id>example</id>
            <url>https://git.example.com/api/packages/example/maven</url>
        </repository>
        <snapshotRepository>
            <id>example</id>
            <url>https://git.example.com/api/packages/example/maven</url>
        </snapshotRepository>
    </distributionManagement>
</project>

```

Here is the test project `build.yml` :

```
name: Build Java software package

on:
  push

jobs:
    publish:
        runs-on: ubuntu-latest
        permissions:
            contents: read
            packages: write
        steps:
            - name: Checkout
              uses: https://github.com/actions/checkout@v3

            - name: Install Maven
              uses: https://github.com/stCarolas/setup-maven@v4

            - name: Install Java
              uses: https://github.com/actions/setup-java@v3
              with:
                distribution: temurin
                java-version: 18
                cache: maven
                overwrite-settings: true
                server-id: example

            - run: cat ~/.m2/settings.xml

            - name: Deploy
              run: mvn -B deploy

```

The result from cat :

```
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 https://maven.apache.org/xsd/settings-1.0.0.xsd">
  <servers>
    <server>
      <id>example</id>
      <username>${env.GITHUB_ACTOR}</username>
      <password>${env.GITHUB_TOKEN}</password>
    </server>
  </servers>

```

The result from deploy (note : I've replaced real domain with `"example"`) :

```
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.7:deploy (default-deploy) on project test-library-java: Failed to deploy artifacts: Could not transfer artifact com.example.test:test-library-java:jar:1.0.0 from/to example (https://git.example.com/api/packages/example/maven): Failed to transfer file: https://git.example.com/api/packages/example/maven/com/example/test/test-library-java/1.0.0/test-library-java-1.0.0.jar. Return code is: 401, ReasonPhrase: Unauthorized. -> [Help 1]

```

I've done some tests :

Deploy from development workstation using `mvn deploy` and token in `settings.xml`, this works.

Overriding container `settings.xml` with hardcoded password works

```
#            - name: Configure Maven
#              env:
#                  GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
#              run: echo "<settings><servers><server><id>example</id><username>$GITHUB_ACTOR</username><password>TOKENFROMGITEA</password></server></servers></settings>" > ~/.m2/settings.xml

```

So now I'm wondering if the `settings.xml` really have the environment variables replaced or if the actions provided GITHUB\_TOKEN have the rights for packages/write. Also note that my user is the owner of the repository and admin of Gitea.

Versions used :

* Gitea 1.20.0-rc2
* Docker 20.10.18
* Containerd 1.6.8 9cd3357

Documentation used :

# Answer
> 0 votes
Finally after many trial and errors, I understood that Gitea doesn't support permission changes for the `GITHUB_TOKEN` so I've been able to create a personnal access token with packages read/write access.

Here is the final workflow :

name: Build Java software package

```
on:
  push
#    tags:
#      - "v*.*.*"

# Not tested but would probably work with GitHub but not Gitea yet
#permissions:
#  contents: read
#  packages: write

jobs:
    publish:
        runs-on: ubuntu-latest
        steps:
            - name: Checkout
              uses: https://github.com/actions/checkout@v3

            - name: Install Maven
              uses: https://github.com/stCarolas/setup-maven@v4

            - name: Install Java
              uses: https://github.com/actions/setup-java@v3
              with:
                distribution: temurin
                java-version: 18
                cache: maven
                overwrite-settings: true
                server-id: example

            - name: Deploy
              run: mvn -B deploy
              env:
                GITHUB_TOKEN: ${{ secrets.PACKAGES_TOKEN }}

```

---
Tags: github, maven
---


# Question
Title: In Istio, what controls throughput to a service?
I have a load testing application (jMeter) that is calling my service that is hosted in Kubernetes.

## Summary

I have a service hosted in Kubernetes that seems to have limits placed on how many calls it can take. This service uses Istio (both with a Side Car and using the Istio Ingress Gateway). But I cannot seem to figure out what is limiting the calls to it.

## Details

The primary action for my service (that is hosted in Kubernetes and uses Istio) is to enqueue a message to a RabbitMQ server. This takes a bit of time, but is not CPU nor memory intensive.

When I load test my service (using jMeter), neither the CPU nor memory get high enough to trigger my Horizontal Pod Autoscaling.

But, the throughput is way too low for what this service should be able to do.

My guess is that istio is throttling the calls to my service somehow, but I cannot seem to find what is causing it.

## What I have tried

---

**Destination Rule**

I made a destination rule like this:

```
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: my-testing-rule
  namespace: my-testing-namespace
spec:
  host: my-service-testing-host.svc.cluster.local
  trafficPolicy:
    connectionPool:
      http:
        http2MaxRequests: 5000

```

I did this in hopes of upping the max connections, but it seemed to have no effect when I re-run the load test. (Note: the name, namespace and host in the example above have been changed from the actual values.)

---

**No Istio**

I tried removing Istio from the picture, and it overloaded my cluster when I fired up the load test. I could not longer send kubectl commands and the cluster management tools would not respond. (Apparently some throttling is good.)

But this is what leads me to think that Istio is throttling things.

## Question

**What configurations in the default setup of Istio will throttle traffic?**

# Answer
> 1 votes
You can use Envoy Filter to apply limit rate. In the below example, you apply a limit rate to a workload based on workloadSelector labels. The main example values are that you have a bucket of 1000 tokens. Each connection takes "one token" and in the bucket we "return" 10 tokens per 5 second. The limit rate is enabled and applied to all connection towards selected workload through `local_rate_limit_enabled` and  `local_rate_limit_enforced`.

References that helped me a lot to applied the limit rate to our project was:

Limit Rate Example and Documentation

```

    apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: my-limit-rater
  namespace: mynamespace
spec:
  workloadSelector:
    labels:
      app: "myapp"
      component: "mycomponent"
      instance: "test"
      version: "1.0.0"
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: SIDECAR_INBOUND
        listener:
          filterChain:
            filter:
              name: 'envoy.filters.network.http_connection_manager'
      patch:
        operation: INSERT_BEFORE
        value:
          name: envoy.filters.http.local_ratelimit
          typed_config:
            '@type': type.googleapis.com/udpa.type.v1.TypedStruct
            type_url: type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
            value:
              stat_prefix: http_local_rate_limiter
              enable_x_ratelimit_headers: DRAFT_VERSION_03
              token_bucket:
                max_tokens: 1000
                tokens_per_fill: 10
                fill_interval: 5
              filter_enabled:
                runtime_key: local_rate_limit_enabled
                default_value:
                  numerator: 100
                  denominator: HUNDRED
              filter_enforced:
                runtime_key: local_rate_limit_enforced
                default_value:
                  numerator: 100
                  denominator: HUNDRED

```

---
Tags: kubernetes, istio
---


# Question
Title: Why doesn't ansible.builtin.lineinfile behave as expected in this particular task?
I'm attempting to update `/boot/firmware/cmdline.txt` on a few Raspberry Pis to enable `cgroup` support. Seemed a good fit for `ansible.builtin.lineinfile`.

The existing contents of `/boot/firmware/cmdline.txt` are a single line of key-value pairs

```
console=serial0,115200 dwc_otg.lpm_enable=0 console=tty1 root=LABEL=writable rootfstype=ext4 rootwait fixrtc quiet splash

```

I want the following key-value pairs to be appended to any existing contents of that file that might exist.

* `cgroup_enable=cpuset`
* `cgroup_enable=memory`
* `cgroup_memory=1`
* `swapaccount=1`

In the end, I want the file to look like

```
console=serial0,115200 dwc_otg.lpm_enable=0 console=tty1 root=LABEL=writable rootfstype=ext4 rootwait fixrtc quiet splash cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1 swapaccount=1

```

Here's the task I created:

```
- name: "Modify /boot/firmware/cmdline.txt to enable cGroup support if not already enabled"
    ansible.builtin.lineinfile:
      path: /boot/firmware/cmdline.txt
      regexp: '(.*) ((cgroup_enable=cpuset|cgroup_enable=memory|cgroup_memory=1|swapaccount=1)( {0,}|\n)){4}(.*)'
      line: '$1 cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1 swapaccount=1 $5'
      state: present
      backrefs: true
      backup: true
      owner: root
      group: root
      mode: '0755'

```

Unfortunately, it doesn't seem to update the file and I'm struggling to understand why. To clarify, the task executes fine but indicates that the file already matches the regexp with the following output:

```
TASK [common : Modify /boot/firmware/cmdline.txt to enable cGroup support if not already enabled] **************************
ok: [kube02.local]
ok: [kube03.local]
ok: [kube01.local]

```

As far as I can tell, it does not and I've tested in a few ways. Increasing verbosity of this doesn't help me figure this out, unfortunately.

I did find this post's answer which has a more robust solution, but I would love to understand why my solution *doesn't* work.

# Answer
> 1 votes
Put the new items into a list. For example,

```
  cmdline_add:
    - cgroup_enable=cpuset
    - cgroup_enable=memory
    - cgroup_memory=1
    - swapaccount=1

```

Add the items in the loop

```
  - lineinfile:
      path: /boot/firmware/cmdline.txt
      regexp: '^(?!.*?{{ item }})(.*)$'
      line: '\1 {{ item }}'
      backrefs: true
    loop: "{{ cmdline_add }}"

```

Because of the negative lookahead assertion *(?!...)* the *regexp* matches if the *item* is not in the line. The task is idempotent.

---

<sup>

**Example**. Consolidate the configuration into comma-separated lists

Given the file

shell\> cat /tmp/cmdline.txt

> console=serial0,115200 dwc\_otg.lpm\_enable=0 console=tty1 root=LABEL=writable rootfstype=ext4 rootwait fixrtc quiet splash

The play

```
- hosts: localhost

  vars:

    cmdline_path: /tmp/cmdline.txt
    cmdline_add:
      - cgroup_enable=cpuset
      - cgroup_enable=memory
      - cgroup_memory=1
      - swapaccount=1

    cmdline_add_items: "{{ cmdline_add|map('split', '=', 1) }}"
    cmdline_add_list: |
      {% filter from_yaml %}
      {% for i in cmdline_add_items %}
      - {key: {{ i.0 }}, value: {{ i.1|d('')|split(',') }} }
      {% endfor %}
      {% endfilter %}

    cmdline_items: "{{ cmdline_out.stdout|trim|split(' ')|map('split', '=', 1) }}"
    cmdline_list: |
      {% filter from_yaml %}
      {% for i in cmdline_items %}
      - {key: {{ i.0 }}, value: {{ i.1|d('')|split(',') }} }
      {% endfor %}
      {% endfilter %}

    cmdline_dict: "{{ [cmdline_list, cmdline_add_list]|
                      community.general.lists_mergeby('key', list_merge='append_rp')|
                      items2dict }}"
    cmdline: >-
      {% for k,v in cmdline_dict.items() -%}
      {% if v.0|length > 0 -%}
      {{ k }}={{ v|join(',') }}
      {% else -%}
      {{ k }}
      {% endif -%}
      {% endfor -%}

  tasks:

    - command: "cat {{ cmdline_path }}"
      register: cmdline_out

    - block:
        - debug:
            var: cmdline_add_items|to_yaml
        - debug:
            var: cmdline_add_list|to_yaml
        - debug:
            var: cmdline_items|to_yaml
        - debug:
            var: cmdline_list|to_yaml
        - debug:
            var: cmdline_dict|to_yaml
        - debug:
            var: cmdline
      when: debug|d(false)|bool

    - copy:
        dest: "{{ cmdline_path }}"
        content: "{{ cmdline }}"

```

gives

shell\> cat /tmp/cmdline.txt

> cgroup\_enable=cpuset,memory cgroup\_memory=1 console=serial0,115200,tty1 dwc\_otg.lpm\_enable=0 fixrtc quiet root=LABEL=writable rootfstype=ext4 rootwait splash swapaccount=1

</sup>

---
Tags: ansible
---


# Question
Title: how to run powershell script as windows administrator user using ansible
i want to run a power shell script on remote windows machine. The PowerShell script will install software on C drive. The user which used to login is already in administrator group.

But when the command triggered manually in server it works but through ansible it is not working. The ansible task I have configured:

```
tasks:
    - name: Execute PowerShell script
      win_shell: |
        powershell.exe -NoProfile -ExecutionPolicy Bypass -File "C:\Path\to\your\script.ps1"

```

Any help?

# Answer
You can use

```
become: yes
become_method: runas

```

https://stackoverflow.com/a/48675395/15962190

https://docs.ansible.com/ansible/2.7/user\_guide/become.html#become-and-windows
> 1 votes

---
Tags: ansible, windows, powershell
---


# Question
Title: Is there a distinction between Flux and GitOps Toolkit?
What is the difference between Flux and the GitOps Toolkit?

Is Flux v2 simply a distribution of the GitOps Toolkit components (such as the source controller for polling/scraping git repos, the kustomisation controller for syncing cluster resources with manifests, and the helm controller for declarative releases from manifest templates)? Does Flux include anything else in addition to the toolkit components, or in other words, is it possible to use all of the toolkit components and still not be "using flux"? (For example, is the flux CLI just a bootstrap for deploying the toolkit components into a k8s cluster?) Conversely, does the toolkit include anything that is not used in flux?

# Answer
The short answer is, GitOps Toolkit is Flux. If you are using Flux then you are using the GitOps Toolkit. Flux is an un-opinionated distribution of the GitOps Toolkit, and I know of no other distributions that do call themselves GitOps Toolkit.

The philosophical answer is a bit deeper, since a GitOps Toolkit is defined here, as a set of components, and somewhat innate in the definition of sets is the idea that they can be updated, or changed:

https://fluxcd.io/flux/components/

Indeed in Flux itself, in the 6 components that are all definitely part of Flux, 4 are defaulted, and only 3 of those are now considered "generally available" in the sense that they receive the "GA" label on our roadmap, and have left the Beta period of the API. Those GA controllers are the Source Controller, Kustomize Controller, and the Notification Controller. There's also Helm Controller, which will be GA soon, and the Image Automation and Reflector controllers that may go GA some time later on, depending on the roadmap priorities.

The GitOps Toolkit officially contains those 6 APIs. The APIs are defined as Kubernetes CRDs. Any product that depends on bits of Flux's functionality can integrate GitOps Toolkit APIs, and the promise of GA is that those APIs are stable and you can build on them, and expect them to go on working as long as Flux is being developed. Those are Flux's guarantees, but they're also GitOps Toolkit's guarantees as an API, you don't need to integrate or even consider the Flux CLI in order to depend on those guarantees.

Other products can *integrate*, but can also **implement** "GitOps Toolkit." One such example is the TF-Controller project, that leverages the Source API to provide Flux's Git capabilities to a Terraform K8s-controller experience. While this isn't strictly speaking a part of Flux's "trademark" GitOps Toolkit, Flux itself is made under the CNCF, and with contributions from many other members of the community, also mainly by engineers at Weaveworks.

TF-controller does not simply consume the Source API and use it to do some work, that's integrating GOTK, and while anyone can do that, it's not the same as implementing GOTK. TF-controller is also a (hopefully) faithful implementation of the principles and patterns of Flux's GitOps Toolkit.

A faithful implementation uses the Controller Runtime, it implements Kubernetes Conditions and Status following the guidance and patterns laid out in the GOTK developer guides. TF-controller depends on Flux libraries and its authors are working in collaboration with the Flux core team to produce a cohesive experience.

The project roadmap tracks Flux development and it includes a roadmap that shows how its development can be expected to progress in line with future plans. While there's no official conformance suite or test to say if your project implements the "GitOps Toolkit," there's no set of requirements that you must adhere to be considered for membership in the group and so you get invited when we have parties, etc. you know, maybe there should be a conformance guide for GitOps Toolkit as well!

Hopefully this disambiguated the term somewhat so you can understand the difference better.
> 0 votes

---
Tags: kubernetes, continuous-deployment, terminology
---


title: Pod to pod communication GKE

text:
# Question
Title: Pod to pod communication GKE
I have this terraform-kubernetes script terraform1

so prometheus scraping list of pods, and hitting `/metrics` based on custom resource `Prometheus` and `PodMonitor` rules

it's working fine locally using minikube,

but in GKE, it doesn't work,

podmonitor works normally:

but scraping failed (this part is green when on local minikube):

normally when I open the website on using port forwarding (32009 to random localhost port) it works normally 200, no error:

does GKE disallow pod to pod communication? seems not since the error is 403 instead of something else, or I need to do something so Prometheus pod that lives on 10.63.1.4 can scrape other pods in the target list (10.63.1.252, 10.63.1.5, 10.63.3.5, 10.63.4.210, 10.63.1.133, etc)

# Answer
> 0 votes
Apparently it's software issue, the software being deployed have blocking on specific headers, it only allows specific `Host:` header, that's why it's 403 instead of timeout or something else.

---
Tags: kubernetes, google-kubernetes-engine
---


# Question
Title: How to pass terraform locals variables into AWS resource template body
I am trying to apply aws conformance pack to my aws account using terraform. I am using the sample aws config rules in cloudformation syntax provided in the awslabs repository.

Now from an example conformance pack provided on AWS documentation we have this:

```
resource "aws_config_conformance_pack" "s3conformancepack" {
  name = "s3conformancepack"

  template_body = <<EOT

Resources:
  S3BucketPublicReadProhibited:
    Type: AWS::Config::ConfigRule
    Properties:
      ConfigRuleName: S3BucketPublicReadProhibited
      Description: >-
        Checks that your Amazon S3 buckets do not allow public read access.
        The rule checks the Block Public Access settings, the bucket policy, and the
        bucket access control list (ACL).
      Scope:
        ComplianceResourceTypes:
        - "AWS::S3::Bucket"
      Source:
        Owner: AWS
        SourceIdentifier: S3_BUCKET_PUBLIC_READ_PROHIBITED
      MaximumExecutionFrequency: Six_Hours
  S3BucketPublicWriteProhibited:
    Type: "AWS::Config::ConfigRule"
    Properties:
      ConfigRuleName: S3BucketPublicWriteProhibited
      Description: "Checks that your Amazon S3 buckets do not allow public write access. The rule checks the Block Public Access settings, the bucket policy, and the bucket access control list (ACL)."
      Scope:
        ComplianceResourceTypes:
        - "AWS::S3::Bucket"
      Source:
        Owner: AWS
        SourceIdentifier: S3_BUCKET_PUBLIC_WRITE_PROHIBITED
      MaximumExecutionFrequency: Six_Hours
  S3BucketReplicationEnabled:
    Type: "AWS::Config::ConfigRule"
    Properties:
      ConfigRuleName: S3BucketReplicationEnabled
      Description: "Checks whether the Amazon S3 buckets have cross-region replication enabled."
      Scope:
        ComplianceResourceTypes:
        - "AWS::S3::Bucket"
      Source:
        Owner: AWS
        SourceIdentifier: S3_BUCKET_REPLICATION_ENABLED
  S3BucketSSLRequestsOnly:
    Type: "AWS::Config::ConfigRule"
    Properties:
      ConfigRuleName: S3BucketSSLRequestsOnly
      Description: "Checks whether S3 buckets have policies that require requests to use Secure Socket Layer (SSL)."
      Scope:
        ComplianceResourceTypes:
        - "AWS::S3::Bucket"
      Source:
        Owner: AWS
        SourceIdentifier: S3_BUCKET_SSL_REQUESTS_ONLY
  ServerSideEncryptionEnabled:
    Type: "AWS::Config::ConfigRule"
    Properties:
      ConfigRuleName: ServerSideEncryptionEnabled
      Description: "Checks that your Amazon S3 bucket either has S3 default encryption enabled or that the S3 bucket policy explicitly denies put-object requests without server side encryption."
      Scope:
        ComplianceResourceTypes:
        - "AWS::S3::Bucket"
      Source:
        Owner: AWS
        SourceIdentifier: S3_BUCKET_SERVER_SIDE_ENCRYPTION_ENABLED
  S3BucketLoggingEnabled:
    Type: "AWS::Config::ConfigRule"
    Properties:
      ConfigRuleName: S3BucketLoggingEnabled
      Description: "Checks whether logging is enabled for your S3 buckets."
      Scope:
        ComplianceResourceTypes:
        - "AWS::S3::Bucket"
      Source:
        Owner: AWS
        SourceIdentifier: S3_BUCKET_LOGGING_ENABLED
EOT
}

```

Now in my terraform code I will pick one of the config rules in the example provided in the awslabs repo, which is this

```
AcmCertificateExpirationCheck:
    Properties:
      ConfigRuleName: acm-certificate-expiration-check
      InputParameters:
        daysToExpiration:
          Fn::If:
          - acmCertificateExpirationCheckParamDaysToExpiration
          - Ref: AcmCertificateExpirationCheckParamDaysToExpiration
          - Ref: AWS::NoValue
      Scope:
        ComplianceResourceTypes:
        - AWS::ACM::Certificate
      Source:
        Owner: AWS
        SourceIdentifier: ACM_CERTIFICATE_EXPIRATION_CHECK
    Type: AWS::Config::ConfigRule

```

Now I have a section in the above snippet where we have an **if statement** that checks for the **variable acmCertificateExpirationCheckParamDaysToExpiration**, then is passed the value **Ref: AcmCertificateExpirationCheckParamDaysToExpiration** that references a **parameter**:

```
AcmCertificateExpirationCheckParamDaysToExpiration:
    Default: '90'
    Type: String

```

Now in my terraform code this **parameter** would be set like this:

```
locals {
 AcmCertificateExpirationCheckParamDaysToExpiration = var.acm_certificate_expiration_check_param_days_to_expiration != ""

}

variable acm_certificate_expiration_check_param_days_to_expiration {
  type = string
  default = "90"
}

```

Now I want to use this in the **conformance pack** resource in terraform. So how do I use the if section in a **terraform** way:

```
resource "aws_config_conformance_pack" "s3conformancepack" {
  name = "s3conformancepack"

  template_body = <<EOT

Resources:
  AcmCertificateExpirationCheck:
    Properties:
      ConfigRuleName: acm-certificate-expiration-check
      InputParameters:
        daysToExpiration:
          Fn::If:
          - acmCertificateExpirationCheckParamDaysToExpiration
          - Ref: AcmCertificateExpirationCheckParamDaysToExpiration  <======[ How to pass the locals variable into this as done on cloudformation or is this ${local.AcmCertificateExpirationCheckParamDaysToExpiration} sufficient ]
          - Ref: AWS::NoValue
      Scope:
        ComplianceResourceTypes:
        - AWS::ACM::Certificate
      Source:
        Owner: AWS
        SourceIdentifier: ACM_CERTIFICATE_EXPIRATION_CHECK
    Type: AWS::Config::ConfigRule

```

# Answer
> 0 votes
In my question I have this code snipet:

```
AcmCertificateExpirationCheck:
    Properties:
      ConfigRuleName: acm-certificate-expiration-check
      InputParameters:
        daysToExpiration:
          Fn::If:
          - acmCertificateExpirationCheckParamDaysToExpiration
          - Ref: AcmCertificateExpirationCheckParamDaysToExpiration
          - Ref: AWS::NoValue
      Scope:
        ComplianceResourceTypes:
        - AWS::ACM::Certificate
      Source:
        Owner: AWS
        SourceIdentifier: ACM_CERTIFICATE_EXPIRATION_CHECK
    Type: AWS::Config::ConfigRule

```

This section:

```
InputParameters:
        daysToExpiration:
          Fn::If:
          - acmCertificateExpirationCheckParamDaysToExpiration
          - Ref: AcmCertificateExpirationCheckParamDaysToExpiration
          - Ref: AWS::NoValue

```

can be written as:

```
InputParameters:
        daysToExpiration: ${local.AcmCertificateExpirationCheckParamDaysToExpiration ? var.acm_certificate_expiration_check_param_days_to_expiration : null}

```

I tested this on AWS and it deployed successfully

---
Tags: amazon-web-services, terraform, aws-cloudformation
---


# Question
Title: Jenkins checkout GitSCM and git step how does it work internally and why
When I execute my pipeline and I go and see on the actual Jenkins slave, the checked out commit doesn't have latest changes. But the logs and actual files in the workspace on the Jenkins slave running this build are confusing me.

In the logs Jenkins shows it is resolving commit id for the previous commit on the branch (notice the carrot at the end):

```
/usr/bin/git rev-parse refs/remotes/origin/BranchName^

```

but in a log line after this it shows a fetch command with correct latest commit id and latest commit message.

```
/usr/bin/git checkout -f 25d8dd262b61a8f703912db297kuhg76768 # timeout=10
Commit message: "Latest commit message"

```

**I HAVE**

```
always {
            cleanWs()
        }

```

in the post stage **SO THIS IS NOT WORKSPACE CLEAN UP ISSUE.** Workspace is getting cleaned after every build I checked it in logs and also on the server.

Other thing that I did is, when build was triggered I did SSH into the Jenkins slave machine where build was running and checked the files in actual workspace but the files were old and the head was is detached state pointing to the correct latest commit. As mentioned above the log was showing Jenkins executed git -f checkout . (Full log is provided below)

Why the git and checkout steps in the Jenkins pipeline take the code from the repository this way?

I want to understand why this plugin does all these steps in the log below, why can't it just clone new repo, pull a latest commit using reference and that's it.

I have following step in my declarative pipeline:

```
dir("${branch_name}/somedir"){
                    checkout(
                        [
                            $class: 'GitSCM',
                            branches: [[name: 'refs/heads/${branch_name}']],
                            doGenerateSubmoduleConfigurations: false,
                            extensions: [], submoduleCfg: [],
                            userRemoteConfigs: [
                                [
                                credentialsId: 'XXXXXX',
                                name: 'origin',
                                refspec: '+refs/heads/${branch_name}:refs/remotes/origin/${branch_name}',
                                url: 'https://github.com/xyz/repo.git'
                                ]
                            ]
                        ])
                }

```

And in the logs I see following:

```
    Cloning the remote Git repository
Cloning repository https://github.com/xyz/repo.git
 > /usr/bin/git init /pathToWorkspace/somedir # timeout=10
Fetching upstream changes from https://github.com/xyz/repo.git
 > /usr/bin/git --version # timeout=10
 > git --version # 'git version 2.23.4'
using GIT_ASKPASS to set credentials Used to pull the code from GitHub as JenkinsUser user
 > /usr/bin/git fetch --tags --force --progress -- https://github.com/xyz/repo.git +refs/heads/*:refs/remotes/origin/* # timeout=10
Checking out Revision 25d8dd262b61a8f703912db297kuhg76768 (refs/remotes/origin/BranchName)
 > /usr/bin/git config remote.origin.url https://github.com/xyz/repo.git # timeout=10
 > /usr/bin/git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > /usr/bin/git config remote.origin.url https://github.com/xyz/repo.git # timeout=10
Fetching upstream changes from https://github.com/xyz/repo.git
using GIT_ASKPASS to set credentials Used to pull the code from GitHub as JenkinsUser user
 > /usr/bin/git fetch --tags --force --progress -- https://github.com/xyz/repo.git +refs/heads/BranchName:refs/remotes/origin/BranchName # timeout=10
 > /usr/bin/git rev-parse refs/remotes/origin/BranchName^{commit} # timeout=10
 > /usr/bin/git rev-parse refs/remotes/origin/refs/heads/BranchName^{commit} # timeout=10
 > /usr/bin/git config core.sparsecheckout # timeout=10
 > /usr/bin/git checkout -f 25d8dd262b61a8f703912db297kuhg76768 # timeout=10
Commit message: "Latest commit message"
 > /usr/bin/git rev-list --no-walk 25d8dd262b61a8f703912db297kuhg76768 # timeout=10

```

# Answer
> 0 votes
`git.exe rev-parse --resolve-git-dir C:\workspace\.git # timeout=10:` This command checks if the current directory is a Git repository. This is an initial sanity check.

`git.exe config remote.origin.url https://github/repo.git # timeout=10:` This command sets the URL for the remote repository where your code is located.

`git.exe rev-parse --verify HEAD # timeout=10:` This command is verifying the currently checked-out commit.

`git.exe reset --hard # timeout=10:` This command resets the workspace to match the HEAD commit.

`git.exe clean -ffdx # timeout=10:` This command removes untracked files from the workspace.

`git.exe --version # timeout=10` and `git --version #` 'git version 2.36.0.windows.1': These commands display the git version.

`git.exe fetch --tags --force --progress --depth=1 -- https://github/repot.git +refs/heads/*:refs/remotes/origin/* # timeout=10:` This command fetches the changes from the remote repository, limiting the depth of the history to 1 commit which is helpful for saving time and space.

`git.exe rev-parse "refs/remotes/origin/branch^{commit}" # timeout=10:` This command is checking out the exact commit in the branch "branch".

`git.exe config core.sparsecheckout # timeout=10:` This command checks if sparse checkout is enabled, which would mean only certain specified directories are checked out instead of the whole repository.

`git.exe checkout -f 1fb75f0e6daf5********0db829b97b9bee3` \# timeout=2000: This command checks out the specified commit.

`git.exe config --get remote.origin.url # timeout=10:` This command gets the remote URL of the repository.

---
Tags: jenkins, jenkins-pipeline, git, jenkins-plugins, github
---


# Question
Title: What should I learn next as a devops beginner?
I'm working as a devops engineer in a big tech company (local). The technologies used here are nginx, linux, sql, redis, docker, glassfish and payara, kubernetes, tomcat, gitlab, jenkins etc. I've learnt linux and sql till date. What should I learn next? I mean the order might be important. But the important thing to note is that for example for jenkins, mostly just building jenkins is done, there's not much of other tasks. So, will it even be worth it to learn all stuffs in detail?

They're planning to use kafka after few months as well.

# Answer
What *you* should learn next is highly contextual, so I fear this question will be closed as opinion-based. However, there are a few patterns which we tend to see, which may help you understand where you are on *your* journey *with respect to the average* DevOps figure's career roadmap.

I propose as an answer to this question:

1. Take a look at https://roadmap.sh/devops
2. Select the items you feel you've mastered
3. **Decide whether you want to specialise or generalise**

If you want to specialise, spend time in the area where you have the most experience (maybe Design Patterns), else if you want to generalise, pick off topics that are outside of your current comfort zone.

As an example I attach a screenshot of what a a roadmap of a typical engineer might look like:

This person might decide to specialise in monitoring, or generalise by learning more in the CI/CD space.
> 1 votes

---
Tags: docker, kubernetes, jenkins, nginx, tomcat
---


# Question
Title: How can I log every command for which an Ansible playbook uses elevated permissions?
I have an Ansible playbook that runs against a Linux target host and uses `become` to gain elevated permissions for those actions that require it.

For auditing purposes, I need a list of all the *Unix commands* that are executed using those elevated rights.

I.e., suppose the playbook uses the `yum` module to install a package:

```
- name: Example
  become: yes
  tasks:
    - name: Provide current Apache
      yum:
        name: httpd
        state: latest

```

I need to see that the `/usr/bin/dnf` binary is called with root permissions.

How can I configure the various logging systems involved to provide that information?

# Answer
> 0 votes
Maybe setting verbosity level inside Ansible could provide that information. You could either pass it as an argument '**-vv or -v**' or setting it in the ansible.cfg =\> **verbosity = 2**

---
Tags: ansible, logging, audit
---


# Question
Title: No Kubernetes Option in Docker Desktop for Windows
Just getting started on Docker and after installing Docker Desktop on a Windows 10 Pro machine, the option to enable Kubernetes is missing from the context menu when using Windows containers. See the attached image. When I switch to Linux containers the option appears. Various reading I've done seems to indicate that it should be available to Windows containers too. What am I missing?

Windows 10 Pro (10.0.17134.1006)

Docker Desktop 2.1.0.5

After update:

Settings after update:

# Answer
> 4 votes
You need to first switch to Edge version by going to the

Docker -\> Settings -\> General tab.

Then, you have to enable the kubernetes.

> Open Docker for Windows settings by accessing your system tray and navigate to the Kubernetes tab. Select the checkbox for Enable Kubernetes to start a single-node cluster when starting Docker.

Reference: https://medium.com/@damiannolan/kubernetes-on-docker-for-windows-5ca0c6395668

As per this old link and a latest one, Kubernetes needs a Linux master node and thus the kubernetes cluster can be initialized only from `linux containers` in Docker desktop for Windows. After enabling and starting the docker containers, you can add Windows containers the the running kubernetes cluster.

This will give you much clear context.

# Answer
> 1 votes
As Samit commented, the options for Kubernetes are only available if Linux containers are currently set as the environment.

# Answer
> 0 votes
My problem was that I had installed some version or configuration of docker that I got from a Udemy course. There was no Kubernetes anywhere. Once I removed that version of docker and installed a new one downloaded from https://docs.docker.com/get-docker/ , the k8s option was there.

# Answer
> 0 votes
I faced the similar issues with Docker Windows. I realized we need to switch to linux by clicking on **windows tray right click docker** as below:

ur.com/C0e4r.png

Please note, if your previous images are on windows you need to run the same process after switching to linux and then continue with your Kubernetes kubectl

---
Tags: docker, kubernetes, windows, configuration
---


# Question
Title: Ansible Connecting to remote Docker container over SSH
I have the following situation: I have a remote server that runs docker with a Postgres container. The Postgres instance is only reachable through a docker network, not directly from the remote server.

I am trying to create an ansible role that makes sure all the required databases and users exist in the Postgres instance. For this I am trying to make Ansible first connect to the remote server, then connect to the Postgres docker container, and from there use the `community.postgresql.postgresql_*` modules to create the databases and users. However, I am stuck trying to get Ansible to do the nested connection. I currently have the following ansible-playbook:

```
# playbook.yml
- name: Postgres
  hosts: dockerservers
  roles:
    - name: postgres
      tags: [postgres, never]

# roles/postgres/tasks/main.yml
- name: Connect to database docker container
  add_host:
    name: "postgres"
    ansible_connection: docker
  changed_when: false

- name: Inside postgres docker container
  delegate_to: "postgres"
  block:
    - name: Debug
      community.postgresql.postgresql_info:

```

Docker tries to use the local docker cli (which is not installed) to connect to the remote docker daemon. Is there a way to somehow hop from the remote server to the docker container, instead of trying to directly connect to it?

# Answer
> 1 votes
I have found a hacky workaround for my problem. I replicated the behavior of the `community.postgresql.*` modules that I want to use in a bash script and execute it using the `community.docker.docker_container` module.

```
- name: Ensure databases and users exist
  register: "pg_ensure_db"
  changed_when: "not 'created' in pg_ensure_db.container.Output"
  community.docker.docker_container:
    name: postgresql-ensure-db
    image: "{{postgres_image}}"
    cleanup: true
    detach: false
    networks:
      - name: "db"
    command_handling: correct
    command: ["bash", "-c", "{{lookup('template', 'postgres-ensure-db.j2.sh')}}" ]

```

While this is not directly an answer to my question, and certainly less maintainable than if it was done natively using Ansible, I thought I would share it anyway until I find a better solution.

---
Tags: docker, ansible, ssh, postgresql
---


# Question
Title: What is IPC:BufferIO in managed postgresql metrics on AWS
I have a managed postgresql instance on AWS.

Sometimes the database session load seems excessive

in `RDS > Databases > [instance]` page

Then in the metrics I see the following graph:

My question is what is `IPC:BufferIO` I couldn't find anything in docs or google about it.

# Answer
IPC:BufferIO wait event is actually LWLock:BufferIO , starting from PostgreSQL 14 BufferIO wait event moved from LWLock wait event type to IPC wait event type , please check this commit and this email thread .

also above information already got mentioned in AWS RDS for PostgreSQL Docs

"For RDS for PostgreSQL versions earlier than PostgreSQL 14, this wait event type is lwlock. For PostgreSQL 14 and later releases, the wait event type is IPC. "
> 1 votes

---
Tags: amazon-web-services, postgresql
---


# Question
Title: What is the equivalent of eksctl option "--alb-ingress-access" in a config file?
I read How to run Serverless Kubernetes: AWS EKS on Fargate, and am wondering what the equivalent option for `--alb-ingress-access` is in a config schema shown in Config file schema? At least it's not obvious to me. I even created a sample cluster and printed out the info and didn't see it there.

```
$ eksctl create cluster \
    --name ingress-sample \
    --version 1.19 \
    --region us-east-1 \
    --fargate \
    --alb-ingress-access
$ eksctl get cluster --name ingress-sample -o yaml
- Arn: arn:aws:eks:us-east-1:0123456789:cluster/ingress-sample
  CertificateAuthority:
    Data: <redacted>
  ClientRequestToken: null
  CreatedAt: "2021-02-19T00:44:27.976Z"
  EncryptionConfig: null
  Endpoint: https://<redacted>.gr7.us-east-1.eks.amazonaws.com
  Identity:
    Oidc:
      Issuer: https://oidc.eks.us-east-1.amazonaws.com/id/redacted
  KubernetesNetworkConfig:
    ServiceIpv4Cidr: 10.100.0.0/16
  Logging:
    ClusterLogging:
    - Enabled: false
      Types:
      - api
      - audit
      - authenticator
      - controllerManager
      - scheduler
  Name: ingress-sample
  PlatformVersion: eks.1
  ResourcesVpcConfig:
    ClusterSecurityGroupId: sg-xxxxxx
    EndpointPrivateAccess: false
    EndpointPublicAccess: true
    PublicAccessCidrs:
    - 0.0.0.0/0
    SecurityGroupIds:
    - sg-xxxxxx
    SubnetIds:
    - subnet-111111
    - subnet-222222
    - subnet-333333
    - subnet-444444
    VpcId: vpc-xxxxxx
  RoleArn: arn:aws:iam::0123456789:role/eksctl-ingress-sample-cluster-ServiceRole-D0X7QCIRBSDN
  Status: ACTIVE
  Tags: {}
  Version: "1.19"

```

**UPDATE**: Hmmm, maybe

```
$ eksctl create cluster --alb-ingress-access -f config.yaml

```

# Answer
Yep this worked, or at least I didn't get an error

```
    $ eksctl create cluster --alb-ingress-access -f config.yaml

```
> 2 votes

# Answer
You can use the `iam` key in your node groups.

```
    iam:
      withAddonPolicies:
        albIngress: true

```

See an example here: 23-kubeflow-spot-instance.yaml
> 0 votes

---
Tags: kubernetes
---


# Question
Title: How to defragment etcd database in Kubernetes cluster
I am trying to resolve a recurring Prometheus alert I'm getting that states that my Kubernetes cluster `etcd` database is fragmented and should be defragmented. I have never used the `etcdctl` command before, which is apparently installed automatically in the container in which `etcd` is running.

I have found a post that gives a very brief description of what to do, but it does not work for me. I have also been looking at documentation on the etcd.io site, but everything there seems to assume the user is in the pod, running the `etcdctl` command directly.

The command I have tried to use is:

```
kubectl exec $(kubectl get pods --selector=component=etcd -A -o name | head -n 1) -n kube-system -- etcdctl defrag --cluster --cacert /etc/kubernetes/pki/etcd/ca.crt --key /etc/kubernetes/pki/etcd/apiserver.key --cert /etc/kubernetes/pki/etcd/apiserver.crt

```

I am not 100% sure if I'm specifying the correct file paths for the `--key` and `--cert` options. The first link just uses `server.key` and `server.crt`, which it complains don't exist for my set-up. I did find the files `apiserver.key` and `apiserver.crt`, but maybe those are not the files to use. If they are not, I don't know what **should** be used.

I'm running this from Git Bash. It errors out with a message that seems to make no sense. It's almost as though it's trying to use the local filesystem (which is a PC running Windows 11).

The error message:

```
Error: open C:/Program Files/Git/etc/kubernetes/pki/etcd/apiserver.crt: no such file or directory
command terminated with exit code 128

```

Are these directions correct? Does anyone know why I might be getting the strange error?

**UPADTE**:

As suggested by @Botje, I ran the following (note the extra slash at the start of each file path):

```
$ kubectl exec $(kubectl get pods --selector=component=etcd -A -o name | head -n 1) -n kube-system -- etcdctl defrag --cluster --cacert //etc/kubernetes/pki/etcd/ca.crt --key //etc/kubernetes/pki/etcd/apiserver.key --cert //etc/kubernetes/pki/etcd/apiserver.crt

```

Unfortunately this did not work either:

```
Error: open //etc/kubernetes/pki/etcd/apiserver.crt: no such file or directory
command terminated with exit code 128

```

# Answer
> 1 votes
You have to login to that host where etcd pods is running (usually Mater node) and execute :

```
etcdctl defrag

```

OR

```
ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379  --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key defrag

```

prerequisite it to install etcdctl (by installing etcd to that host)

# Answer
> 1 votes
You need to run the command on one of the Kubernetes nodes where the etcd pods are running.

> I'm running this from Git Bash. It errors out with a message that seems to make no sense. It's almost as though it's trying to use the local filesystem (which is a PC running Windows 11).

That's because this is exactly what you are doing, and your Windows 11 install doesn't have the certificate files.

---
Tags: kubernetes, prometheus, etcd
---


# Question
Title: Using Jenkins gitlab token to make an API call
From inside a Jenkins pipeline, I want to create a merge request on gitlab

```
def toJson = {
    input ->
    groovy.json.JsonOutput.toJson(input)
}
def body = [
    private_token: "...",
    source_branch: "...",
    target_branch: "...",
    title: "..."
]
httpRequest httpMode: 'POST', requestBody: toJson(body), contentType: 'APPLICATION_JSON', url: 'https://my.gitlab.domain/api/v4/projects/1234/merge_requests'

```

My problem is with the private\_token, I don't want to put it in clear in the Jenkinsfile.

I didn't found a way to get it from Jenkins credentials, is there one ?

# Answer
Try something similar to the following:

```
environment {
  GITLAB_API_TOKEN = credentials('gitlab_api_token').
}

```

or

```
withCredentials([[
  $class: 'com.dabsquared.gitlabjenkins.connection.GitLabApiTokenImpl',
  credentialsId: 'gitlab_api_token',
  variable: 'GITLAB_API_TOKEN'
]])

```

See https://jenkins.io/doc/pipeline/steps/credentials-binding/ and https://github.com/jenkinsci/gitlab-plugin/issues/536 for more information.
> 5 votes

# Answer
You can do this using the Credential Binding Plugin. Open an existing pipeline job, scroll down to the text area where you enter the pipeline code, and click the Pipeline Syntax link to open the Snippet Generator.

From the Sample Step drop-down, select `withCredentials: Bind credentials to variables`. Under the Bindings section, select the "Add" button next to Credentials. Add a new credential of kind "Secret text". Enter the API token, give it an easy to remember ID (this is how we reference this secret) and a description.

When the token is added, select it from the **Credentials** drop-down selection. Enter the variable you want to use to reference the API key. Using your example, I named it `private_token`. When finished, click the **Generate Pipeline Script** button to generate a sample withCredentials() function. Using this function, any reference to the API key will be hidden from the user when called as `private_token`

```
withCredentials([string(credentialsId: 'Gitlab-API-Key', variable: 'private_token')])
{
    echo("My private token is: ${private_token}")
}

```
> 2 votes

# Answer
What worked for me was

* Create access token in gitlab
* Create credential in Jenkins as username, password. Use a dummy username & token as the password

Use the below code block

```
withCredentials([gitUsernamePassword(credentialsId: 'your-jenkins-token-name',
gitToolName: 'Default')]) {  ... }

```
> 0 votes

---
Tags: jenkins, gitlab
---


# Question
Title: What is the best way to install ArgoCD as code?
Working on getting a full deployment pipeline set up using GitOps and Everything as Code. I, therefore, need to be able to spin everything up and down with a single click (or pretty close to it!). My target is to Terraform a Kubernetes cluster in EKS, then have Argo CD pull containers from docker hub.

In the article "How To Set Up Argo CD With Terraform To Implement Pure GitOps", the author advocates using Terraform to install Argo CD.

I have always tried to *avoid* using Terraform for configuration management. In the Terraform documentation, Hashicorp note that

> Terraform is not a configuration management tool

Is Terraform the best-practice way to perform this action, or is there a better method?

# Answer
The question seems posed subjectively, and will hence be difficult to answer well, but here goes.

Perhaps what is referred to here by using Terraform via the k8s and/or helm providers to provision the state

> "Terraform is not a configuration management tool"

This means that it is a way to achieve a desired state via a given provider. The article you mention does exactly this, using the k8s provider to provision the state of the argocd cluster.

When would this be a "best" way? When you want to have the *entire* state of the system declared in a single place.

* If you are responsible for *only* ArgoCD, there would be no "best" way, since the documentation itself provides a guide for using either Kustomize or Helm.
* If, however, you are responsible for other components, including the infrastructure that ArgoCD requires (network, cluster, security groups, etc), then you would benefit from using Terraform to provision *everything*

Consider the alternative: each part of the system would use its own configuration and provisioning tool meaning lots of context-switching and the need to learn more than one configuration language.

The answer therefore is: The best way to install ArgoCD, or **anything at all** is to first determine the boundaries of the system, then attempt to minimise the tooling necessary to achieve the desired state of that bounded system.
> 2 votes

# Answer
It seems that Kustomize is the way to go.

That is what they use for most of the official Argo CD repositories (e.g., argoproj/argo-cd, argoproj/argoproj-deployments and argoproj-labs/argocd-extensions).

Note that the they do also have a Helm chart, which is a perfectly fine way to install it too.

For the purposes of configuring Argo CD as code, please also have a look at Declarative Setup.
> 1 votes

---
Tags: kubernetes, amazon-web-services, terraform, continuous-deployment, cicd
---


title: Estimate storage space for a git server

text:
# Question
Title: Estimate storage space for a git server
How to estimate the space a git repository will take on a source version control server? For example, given k commits every day, with x kB of code per commit, my repository will take n GB of server storage after 1 month.

**Context:**
I am trying to investigate increased storage usage on a server I use solely for hosting an instance of Gitlab. No CI/CD (yet), just source version control. I recently realized that one year of using the server compounded to about 80GB of storage, while our repositories barely come up to 1GB, as seen in the GitLab Project menu.
GitLab is installed via Docker.

# Answer
> 0 votes
Assuming you are already familiar with how to clean up GitLab, the best way to track the disk use and predict growth is to monitor it. Based on the mounting data you can see the trend.

The problem with calculating the growth in storage is that the git heuristics make it unpredictable. Git is optimizing storage as you commit, a bit like a real time compressed archive.

By monitoring the storage, you can see the rate of growth and add an alarm for when it falls below a safe threshold.

I use DataDog for this. Other tools are available.

# Answer
> 0 votes
As per the documentation:

> GitLab package requires about 2.5 GB of storage space for installation

and

> The necessary hard drive space largely depends on the size of the repositories you want to store in GitLab but as a guideline you should have at least as much free space as all your repositories combined take up.

What you are describing seems to relate more to logs not being properly rotated or docker images not regularly purged.

# Answer
> 0 votes
Number of commits is not a good heuristic for determining storage size on a server. Commits will vary by size, and 1 commit with a large blob can consume more than the 100s of commits before it. This further escalates if you don't have any measures in place to block commits with large files from being pushed to your instance.

Commits aside, repos themselves will almost certainly not be the largest source of storage space on your instance. Metadata (MRs, comments, attachments, etc.) will likely consume far more even than even the most poorly managed of repositories.

Ultimately, having proper monitoring and alerting on your instance for consumed storage, and being able to scale up rapidly is far more important than being able to forecast the approximate future storage. Forecasting the storage is impractical and infeasible.

---
Tags: docker, gitlab, git, storage
---


# Question
Title: why does my Ansible playbook throw "Found existing installation: requests 2.25.1\n\n:stderr:ERROR: Cannot uninstall requests 2.25.1"?
I'm trying to run an ansible playbook that installs docker and docker-compose and runs a compose file on a ec2 t2.micro.

To use the docker modules in ansible, I need to have docker and docker-compose python modules installed in the ec2.

Here is my playbook:

```
- name: Update yum repo, install docker, pip, docker-compose & pip and python modules
  hosts: all
  become: true
  tasks:
    - name: update yum repo and cache
      yum:
        name: '*'
        state: latest
        update_cache: true
    - name: install docker and pip
      yum:
        name:
          - docker
          - pip
    - name: download and install docker-compose
      get_url:
        url: https://github.com/docker/compose/releases/latest/download/docker-compose-Linux-{{lookup('pipe','uname -m')}}
        dest: /usr/local/bin/docker-compose
        mode: +x
    - name: start docker daemon
      systemd:
        name: docker
        state: started
  ########################################################### this is the trouble part
    - name: Install docker and docker-compose python module
      pip:
        name:
          - docker
          - docker-compose
  ############################################################

```

the rest of the playbook adds ec2-user to the docker group, resets the ec2 connection, copies docker-compose file to the server and executes it, which all works perfectly fine.

Take a look if required

```
#play2
- name: Reset connection
  hosts: all
  become: true
  tasks:
    - name:  add ec2-user to docker group
      user:
        name: ec2-user
        groups: docker
        append: yes
    - name: reconnect to server session
      meta: reset_connection
#play3
- name: start docker containers
  hosts: all
  vars_files:
    - docker-vars.yml
  tasks:
    - name: copy docker-compose file
      copy: # copy files from controller to remote node
        src: /home/ansible-project/docker-compose.yaml
        dest: /home/ec2-user/docker-compose.yaml
    - name: login to docker private repo
      docker_login:
        registry_url: https://index.docker.io/v1
        username: "{{user_name}}"
        password: "{{docker_passwd}}"
    - name: execute docker-compose and start container
      docker_compose:
        project_src: /home/ec2-user
        state: present # equivalent of docker-compose up

```

My host file is simple with a single group and 1 server-ip configured with ansible\_user and ssh private key for the login. The docker-vars.yaml has only my docker username and passwd.

I have tried to run the playbook by installing python3 specifically and select the "interpreter\_python= /usr/bin/python3" in the ansible.cfg file and also using the default python installed i,e usr/bin/python3.9, but the playbook runs into an error saying:

> > > "Found existing installation: requests 2.25.1\n\n:stderr:ERROR: Cannot uninstall requests 2.25.1 Hint: The package was installed by rpm."

But if I manually log in to the ec2 and do

```
$ pip install docker
$ pip install docker-compose

```

the modules are successfully installed. And if I comment out the highlighted code and run the playbook it executes successfully.

I need to run this completely in single click, as I have to integrate it into the terraform file later, and I cannot perform that single step this manually.

# Answer
> 0 votes
I found your post while fighting with ERROR: Cannot uninstall requests 2.25.1, RECORD file not found. Hint: The package was installed by rpm. in Amazon Linux 2023 AMI 2023.0.20230517.1 x86\_64 HVM kernel-6.1

You gave me an idea to try locally (ec2-user account, not root) and it seems that commenting out become: true line fixes the problem for me. But I'm using docker\_container not docker compose.

# Answer
> 0 votes
I have also got the same issue while practicing Ansible lab from "Techworld with Nana" boot camp, Below are steps that resolved my issue:

My machine Platform: Amazon Linux (Inferred)

**Did docker python module installation as ec2-user**, instead of mixing it with other rpm installations as the root user

```
- name: Install docker python module
  hosts: aws
  become: yes
  become_user: ec2-user
  tasks:
    - name: Install docker python module #This is dependency with ansible docker_image module
      pip:
        name: docker

```

Full Ansible playbook in case if you are interested

```
---
- name: Install python3,docker and docker-compose
  hosts: aws
  become: yes
  tasks:
    - name: Make sure python3 and docker are installed
      vars:
        ansible_python_interpreter: /usr/bin/python
      dnf:  #yum module as ofnow supports only python2 interpreter
        name:
          - python3
          - docker
          - python3-pip
        update_cache: yes
        state: present
    - name: Install docker-compose
      get_url:
        url: https://github.com/docker/compose/releases/download/1.27.4/docker-compose-Linux-{{lookup('pipe', 'uname -m')}} #using "lookup" from jinja template to interpret commands inside url
        dest: /usr/local/bin/docker-compose
        mode: +x
    - name: Ensure docker is running
      systemd:
        name: docker
        state: started

- name: Add user to docker group
  hosts: aws
  become: yes
  tasks:
    - name: Add ec2-user to docker group
      user:
        name: ec2-user
        groups: docker
        append: yes
    - name: Reconnect to server session
      meta: reset_connection  #To reset connection immediate after above task

- name: Install docker python module
  hosts: aws
  become: yes
  become_user: ec2-user
  tasks:
    - name: Install docker python module #This is dependency with ansible docker_image module
      pip:
        name: docker

- name: Test docker pull
  hosts: aws
  tasks:
    - name: pull redis
      # command: docker pull redis
      docker_image:
        name: redis
        source: pull

```

---
Tags: docker, ansible, amazon-ec2, automation, configuration-management
---


# Question
Title: Report on job status by agent in Azure Devops
I have a bunch of on-prem agents and I want to create a report that lists the jobs per agent and their status.

There doesn't seem to be anything in the Pipeline Analytics Metadata that references the agent that the job was run on.

Is this data available anywhere else?

# Answer
You should be able to get the current job & status for an agent via the rest api:

https://learn.microsoft.com/en-us/rest/api/azure/devops/distributedtask/agents/list?view=azure-devops-rest-7.0

You can find the pool id via the the URL from the UI or make a separate call to: https://dev.azure.com/{organization}/\_apis/distributedtask/pools?api-version=7.0
> 0 votes

---
Tags: azure-devops, azure-pipelines
---


# Question
Title: Persistant storage claim stuck on pending with local storage in minikube, even with a storage class provided
Minimum reproducible example:

```
# mongo-storage.yaml
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: slow
provisioner: kubernetes.io/gce-pd
parameters:
  type: pd-standard
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mongo-volume-claim
spec:
  storageClassName: "slow"
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 512Mi

```

Given the above yaml, when created with `kubectl apply -f mongo-storage.yaml`, the PVC remains pending given minutes to create.

When `kubectl get -f mongo-storage.yaml` is run, the below is returned.

```
NAME                               PROVISIONER            RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
storageclass.storage.k8s.io/slow   kubernetes.io/gce-pd   Delete          Immediate           false                  58s

NAME                                       STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/mongo-volume-claim   Pending                                      slow           58s

```

I didn't have a storageclass at all, but I found this thread troubleshooting. The storageclass is copy pasted from the docs for dynamic storage provisioning.

I'm just trying to make my DB persist past restarts. I've pasted the whole yaml at https://pastebin.com/BSFEB5Dp, let me know if there's any other ways to have this persistent storage. This is mostly scrapped together from various google searches, I can't pretend to fully understand every config option here.

I'm extremely new to Kubernetes, please let me know if there's anything I'm missing, or any way I could improve my question.

# Answer
> 0 votes
You need to create in your mongo-storage.yaml the resource "PersistentVolume" and set too the storageClassName, and persistentVolumeReclaimPolicy to RETAIN like following.

```
apiVersion: v1
kind: PersistentVolume
metadata:
  name: mongo-volume-pv
spec:
  storageClassName: slow
  capacity:
    storage: 3Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  hostPath:
    path: "/mnt/data"
    type: DirectoryOrCreate

```

The PersistentVolumeClaim does a call to StorageClass that looks for a PersistentVolume idle. Don't forget to include in your storageclass the line ***volumeBindingMode: WaitForFirstConsumer*** Now, if you restart your pod, the kubernetes will persist your data.

# Answer
> 0 votes
Try running the command kubectl get pvc mongo-volume-claim. This should give you a log message telling you why it is pending.

Looking at your provisioned it looks like you are using GKE to spin this cluster up. Did you provision the storage already? Im not too familiar with GKE but i use AWS EKS for my clusters..When using AWS you also have to ensure that the storage add-on has been enabled for the cluster..

So I would ensure that you provisioned the storage for this cluster. Check the logs. Make sure you

---
Tags: kubernetes, kubernetes-volume
---


# Question
Title: Metrics Server and Prometheus have overwriting resources
I am trying to install Metrics Server, but it needs to create a resource that already exists from an installation of Prometheus. They both want to make `v1beta1.metrics.k8s.io`.

I went to see if the resource was the same, so I could just leave it. But they are not. They each have a `service:` section that points to their respective services (`prometheus-adapter` and `metrics-server`).

If I install the Metrics Server version, then I lose the settings for Prometheus. I am hoping that these are not mutually exclusive tools. (They do not fill the same function. Prometheus is for monitoring and the Metrics Server is for Horizontal Pod Scaling.)

**How can I install Metrics Server without messing up Prometheus?**

---

In case it is useful, here is the full yaml for the conflicting definitions of v1beta1.metrics.k8s.io:

**Prometheus:**

```
apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
  labels:
    app.kubernetes.io/component: metrics-adapter
    app.kubernetes.io/name: prometheus-adapter
    app.kubernetes.io/part-of: kube-prometheus
    app.kubernetes.io/version: v0.10.0
  name: v1beta1.metrics.k8s.io
spec:
  group: metrics.k8s.io
  groupPriorityMinimum: 100
  insecureSkipTLSVerify: true
  service:
    name: prometheus-adapter
    namespace: ntnx-system
    port: 443
  version: v1beta1
  versionPriority: 100

```

**Metrics Server:**

```
apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
  labels:
    k8s-app: metrics-server
  name: v1beta1.metrics.k8s.io
spec:
  group: metrics.k8s.io
  groupPriorityMinimum: 100
  insecureSkipTLSVerify: true
  service:
    name: metrics-server
    namespace: kube-system
  version: v1beta1
  versionPriority: 100

```

# Answer
A way around this issue is to just change the name of each APISerice..Or better yet just change the metric servers name for example.

```
apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
  labels:
    app.kubernetes.io/component: metrics-adapter
    app.kubernetes.io/name: prometheus-adapter
    app.kubernetes.io/part-of: kube-prometheus
    app.kubernetes.io/version: v0.10.0
  name: v1beta1.metrics-server.k8s.io
spec:
  group: metrics-server.k8s.io # Change the group to something unique
  groupPriorityMinimum: 100
  insecureSkipTLSVerify: true
  service:
    name: prometheus-adapter
    namespace: ntnx-system
    port: 443
  version: v1beta1
  versionPriority: 100

```

Just keep in mind that the metadata.name must be \`spec.version+"."+spec.group. I hope this helps, good luck!
> 1 votes

---
Tags: kubernetes, prometheus
---


title: AWS lambda failed after enabled VPC

text:
# Question
Title: AWS lambda failed after enabled VPC
I am getting connection timeout error with VPC enabled but working fine without enabled VPC.

```
{
  "errorMessage": "HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/tokenizer_config.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7fd7f0a07040>, 'Connection to huggingface.co timed out. (connect timeout=30)'))",
  "errorType": "ConnectTimeout",
  "requestId": "",
  "stackTrace": [
    "  File \"/var/lang/lib/python3.10/importlib/__init__.py\", line 126, in import_module\n    return _bootstrap._gcd_import(name[level:], package, level)\n",
    "  File \"<frozen importlib._bootstrap>\", line 1050, in _gcd_import\n",
    "  File \"<frozen importlib._bootstrap>\", line 1027, in _find_and_load\n",
    "  File \"<frozen importlib._bootstrap>\", line 1006, in _find_and_load_unlocked\n",
    "  File \"<frozen importlib._bootstrap>\", line 688, in _load_unlocked\n",
    "  File \"<frozen importlib._bootstrap_external>\", line 883, in exec_module\n",
    "  File \"<frozen importlib._bootstrap>\", line 241, in _call_with_frames_removed\n",
    "  File \"/var/task/spellCorrection_rest_app.py\", line 4, in <module>\n    response = requests.head(url, timeout=30)  # Increase the timeout value as needed\n",
    "  File \"/var/task/requests/api.py\", line 100, in head\n    return request(\"head\", url, **kwargs)\n",
    "  File \"/var/task/requests/api.py\", line 59, in request\n    return session.request(method=method, url=url, **kwargs)\n",
    "  File \"/var/task/requests/sessions.py\", line 589, in request\n    resp = self.send(prep, **send_kwargs)\n",
    "  File \"/var/task/requests/sessions.py\", line 703, in send\n    r = adapter.send(request, **kwargs)\n",
    "  File \"/var/task/requests/adapters.py\", line 507, in send\n    raise ConnectTimeout(e, request=request)\n"
  ]
}

```

# Answer
```
You need to create a NAT Gateway in either one of your Public subnets 'subnet-0a41b43c04095c248' and 'subnet-00303000' of the VPC. Please follow the documentation[1] to get the detailed steps of creating a NAT Gateway.

Edit the route tables of the private subnets which are associated with your Lambda function, to allow the traffic to the destination 0.0.0.0/0 with the target set to the NAT Gateway, that you created in the public subnet.

```

Ref: https://docs.aws.amazon.com/vpc/latest/userguide/vpc-nat-gateway.html#nat-gateway-creating
> 0 votes

---
Tags: aws-lambda, aws-vpc
---


# Question
Title: Config not found in /home/$USER/.kube/config
When I run `k3s kubectl version`

```
$ k3s kubectl version
W0730 19:46:32.066086   17273 loader.go:222] Config not found: /home/ecarroll/.kube/config
[...]
The connection to the server localhost:8080 was refused - did you specify the right host or port?

```

How can I fix this problem?

# Answer
> 0 votes
This is another error message relate to your kubectl config. In this case you likely have an environmental variable `KUBECONFIG` pointing to a location that doesn't exist.

First confirm this is set to the same location above that didn't exist

```
echo $KUBECONFIG

```

If it's already set, simply regenerate a kubeconfig for k3s with

```
mkdir ~/.kube 2> /dev/null
sudo k3s kubectl config view --raw > "$KUBECONFIG"
chmod 600 "$KUBECONFIG"

```

---
Tags: kubectl, k3s
---


# Question
Title: How do I view helm's install message again?
I installed something with `helm install`, and it output advice in the form of a message on the console for things I would want to do. How can I again get that information?

# Answer
# Getting the helm notes

You must first find the name of the release, for that you can use `helm list -A`

```
❯ helm list -A
NAME        NAMESPACE   REVISION    UPDATED                                 STATUS      CHART                       APP VERSION
kubeapps    kubeapps    1           2023-07-31 13:57:09.698002603 -0500 CDT deployed    kubeapps-12.4.8             2.8.0
my-release  default     1           2023-07-31 15:17:46.534953551 -0500 CDT deployed    nextcloud-3.5.20            27.0.1
traefik     kube-system 1           2023-07-30 17:06:33.680339822 +0000 UTC deployed    traefik-21.2.1+up21.2.0     v2.9.10
traefik-crd kube-system 1           2023-07-30 17:06:32.239719019 +0000 UTC deployed    traefik-crd-21.2.1+up21.2.0 v2.9.10

```

Then you can the get the notes by using the name and namespace (argument to `-n`),

```
helm get notes -n default my-release

```

Note, `-n default` is the default, and not strictly required.
> 0 votes

---
Tags: helm
---


# Question
Title: Specify container IP as extra_hosts in Docker Compose
I have a compose.yaml serving WordPress with a Traefik reverse proxy. The https port 443 loopback for the WordPress container fails by default because the WordPress container only listens on http port 80. I need to have the loopback go through the reverse proxy. I can do this by adding the site's domain to the `extra_hosts`, but I have to hardcode the traefik container IP address, which happens to be 172.18.0.2. What's the proper way to add a record to the WordPress container's `/etc/hosts` file with the Traefik container's IP address?

```
services:
  traefik:
    image: traefik
    ports:
      - 80:80
      - 443:443
    ...

  wordpress:
    image: wordpress
    extra_hosts:
      # how to avoid hardcoding traefik container IP here?
      - mysite.com:172.18.0.2
    ...

```

# Answer
> 1 votes
Solution from here:

> What’s the proper way to add a record to the WordPress container’s `/etc/hosts` file with the Traefik container’s IP address?

The proper way is not adding anything to the hosts file but setting a network alias:

https://docs.docker.com/engine/reference/commandline/network\_connect/ https://docs.docker.com/compose/compose-file/06-networks/

```
services:
  traefik:
    networks:
      default:
        aliases:
          - mysite.com

```

It registers the domain as an alias for the container in Docker’s built-in DNS server.

The reverse proxy is usually in a different compose project. If that’s the case, you have an external network and you need to set the alias on that network and not on the default.

---
Tags: docker, docker-compose, traefik, wordpress
---


# Question
Title: how to create three different environments in aws
I want to deploy a public facing web project on aws for which I have setup ec2 for front end and another ec2 for backend (leave discussion about hpa and other services etc for now). I have to integrate jenkin ci/cd pipeline with this. The setup have 3 environment (staging, production and development with 3 individual databases). My query is how to setup these 3 individual env? Do I need to create 3 different individual projects in aws including frontend ec2, backend ec2 and cicd pipeline? code will be committed in all the env by developer manually and respective cicd pipeline will take care of rest in the respective environment? OR there would be one infrastructure with separate databases per env and common cicd.

# Answer
> 0 votes
This question is very valid, as the setup you create today will stay with you for longer. And believe me, it become more tough to change every single day later :)

So, what options do you have?

In the simplest but "ok-ish" solution. Create VPCs per environment in your account. All resources must be properly tagged (for example key: environment, value: production"). In this way you will have logical separation withing one account, you can set boundaries, who can access what (using the tag I mentioned) and also you will have reasonably good way of understand your costs.

There is a better solution. Every environment goes to its own account. Now, you may think "omg, this will be overcomplicated". I know, it looks like this on the beginning. So I will go further, and create fourth account :D Let's call it master. This account is the one where you will create organization and SSO.

But let's forget now about this fourth one.

With accounts dedicated to environment you have full separation and you use full power of AWS.

Now CI/CD. Infrastructure is a beautiful but treacherous thing :) You will learn in time, that to work with IaC in CI/CD you have to have a slightly different approach than for your applications. Generic CI/CD tools, however powerful, don't ofer full control over the whole life cycle of infrastructure. Well, yes, you can do it, but you need to invest a lot into it.

I suppose you use IaC (terraform, cloudformation, etc) for the rest of my answer.

You need to stage your Jenkins pipelines. First, I recommend to do some SAST test of the quality of your IaC. Checkov, tfsec, etc are your friends here. After that you can create stages for all your environments. Example:

tfsec -\> deploy\_on\_dev -\> deploy\_on\_staging -\> deploy\_on\_prod

You can have some tests between stages:

tfsec -\> deploy\_on\_dev -\> tests\_on\_dev -\> deploy\_on\_staging -\> tests\_on\_staging -\> deploy\_on\_prod -\> tests\_on\_prod

You can force the manual acceptance, something like:

... -\> tests\_on\_staging -\> manual accept -\> deploy\_on\_prod ...

And so on. These are like the generic approaches.

The point is, you should separate your environments. On AWS and in your pipelines. Of course, this requires more work related to authentication, but hey, you want easy things or secure things? ;)

I hope this helps. If you have any more questions, ping me!

On the end, the best way of manage IaC is to use dedicated tools, like Spacelift. I mention it, because I am Developer Advocate there :) But orchestration tools plays very important role in proper management of huge environments. You will be there someday (I mean, with huge infrastructure :) )

Best, Pawel

---
Tags: amazon-web-services
---


# Question
Title: Where to get OS Hardening Scripts for CIS Hardening Standards?
I have 2 questions:

1. The pricing of these images says its $130/year (for software + AWS usage): Does this mean running costs of EC2 are also covered in this pricing? Also, is this pricing per instance? So if we need to use 100 instances, should we pay 13K per year?
2. Are there scripts available to "perform" these hardening tasks on the OS (to meet CIS hardening standards)?

# Answer
> The pricing of these images says its $130/year (for software + AWS usage): Does this mean running costs of EC2 are also covered in this pricing?

No.

If you read the details in the AMI Subscription page, it says in the last sentence: "Charges for using Amazon EC2 and other services from AWS are separate and **in addition** to what you pay to purchase AWS Marketplace software products."

> "Also, is this pricing per instance? So if we need to use 100 instances, should we pay 13K per year?"

Yes. In the same AMI subscription page from the first answer, it states: "You can apply one annual subscription to an AWS Marketplace software product **to one Amazon EC2 instance**"

To find an estimate on total pricing, you need to go to the AWS Marketplace and filter for CIS. You choose the AMI, and click the cost details link. Then choose region and the instance type and a total cost estimate for that instance is shown.

> "Are there scripts available to "perform" these hardening tasks on the OS (to meet CIS hardening standards)?"

Yes with a cost. They provide build kits if you are a member of the CIS SecureSuite. But not for every operating system. See the "Leveraging Build Kits" in this article.
> 1 votes

# Answer
https://www.open-scap.org/getting-started/

Use OpenSCAP to scan and remediate, it has CIS level 1 and 2 server profiles. I'm applying level 1 to my server, it's internal and level 1 leaves it quite usable. I tried applying the general server profile and it broke the server.
> 1 votes

---
Tags: amazon-web-services, amazon-ec2, security
---


# Question
Title: Should I use multiple Terraform projects or only one?
I am starting to build a Terraform GIT project to build resources for azure. Currently my idea is to have a project that is taking care of identity and access topics & to have a project that is taking care of infrastructure topics.

Is it a good idea to have multiple terraform projects for different topics, or would it be better to just have one terraform project that is creating all resources for my organisation?

# Answer
> 1 votes
First of all, I propose a change in terminology. What you're calling a "project" should be referred to as a "module".

An environment is built out of a composition of modules.

This is a design decision. A key part of architecting a cloud environment is to design its components in a such a way that you can maintain them independently. In terraform, these come in the form of modules:

> You can use modules to create lightweight abstractions, so that you can describe your infrastructure in terms of its architecture, rather than directly in terms of physical objects.

Instead of having a huge lump of resources, you have a set of building blocks which express higher levels of abstraction. An example might be "compute cluster". This will be comprised of the instances, the networks, block storage, object storage buckets, access permissions, etc, all packaged in a module. If this concept (compute cluster) is a useful one for you, then you should design a Terraform module which encapsulates them, so that you can reuse the concept.

There is a tradeoff to be made between tiny modules (ie single resource), and mega-modules (ie, all the resources).

Consider your architecture, try to identify which components are re-usable and build modules out of those. To make an analogy with construction: a window and a door are both re-usable components. A wall is a useful concept, not a brick.

# Answer
> 0 votes
All three ways will work (modules mentioned in answers as third answer), in some cases better, in some - well.

But the exercise you have to do is you have to think, how your infrastructure will look in the nearest future. This is, as it was mentioned, a design and architecture decision. For sure you have some goals to achieve and this should be a base for your design.

You have to find the answers to couple of questions:

* how your infrastructure will be multiplied?
* do you allow to re-use of specific elements of the infrastructure (for example VPC)?
* are you going to serve these templates through Platform Engineering teams - self-service solutions?
* how do you deal with multiple environments?
* how do you manage the state?
* how do you orchestrate different modules/templates/environments?
* how do you want to manage the whole SDLC of your infrastructure?

These questions are the starting point for your design. I know, it sounds like I complicate things, but I try to think about big picture. And believe me, it will be very important later.

For example: if my solution is simple and I will run it generally only once, I will go with the simplest solution. If I know I will re-use some components, I will go with modules. If I will re-use my modules in wider way (many projects, env, etc), I will go with modules stored in registry. If I go even further, it is time to think about dedicated tools, like Spacelift (disclosure: I am Developer Advocate there). But if you want to orchestrate very complicated setup, with multiple teams, projects, accounts (or subscriptions), modules, dependencies, you should, really, look at tools like this.

# Answer
> 0 votes
\[I agree with everything in the two other answers, but I'd like to add something else to the discussion.\]

# Terraform challenges

I've had a number of cases with terraform where the number of resources in a terraform module/directory become so huge that terraform takes an unreasonable amount of time to converge on a plan. This often seems to happen with `for_each` loops or other data-driven approaches where a few lines in a data file lead to lots of resources and work for terraform. With these huge directories it can takes hours for terraform to think about it. This is worse on Intel MacBook Pros than ARM/M2-based ones, but I've seen a plan that took hours on an Intel CPU take 15-30 minutes on an ARM processor.

Another issue is provider upgrades. These can lead to needing to do state transformations and often require debugging and iteration to get right. Do you want this debugging to be happening on all of your infrastructure? Probably not.

# Solutions

* As others have said, using modules to keep your code organized is a very good habit to get into with terraform.
* Use environments - you can have different values for variables per environment and adding environments doesn't entail writing more code.
* Keep logical things together in the same directory.
* Beyond common providers that you need everywhere (like vault), minimize the number of different providers in each directory. I wouldn't develop something for AWS in the same directory where I'm managing datadog or another cloud service.
* Use remote state references to share data internally. Terraform modules that are just `local`s that get turned into `output`s are good for centralizing information like domain names, IP lists, and other things so you're not copying and pasting this information all over the place.

---
Tags: terraform
---


# Question
Title: Is there a way to declare global(?) variables inside an ansible task?
I have an Ansible playbook, this playbook has a role on it, this role is configured to run only on 2 hosts:

```
yaml
- hosts: node1,orchestrator1
  gather_facts: True
  any_errors_fatal: True
  remote_user: root
  roles:
    - role: configure_vault_backups
      when: "add_vault_ds|bool == True"

```

Inside the role there are some tasks that saves on some variables the output of some `shell` commands, BUT these tasks run only to one host (node1) **OR** the other host (orchestrator1), as an example:

```
yaml
- name: Check if nfs_mount is online
  shell: mount | grep "/var/lib/hypercx/datastores/nfs_mount"
  register: nfs_mount_output
  ignore_errors: yes
  changed_when: false
  when: "'orchestrator' in ansible_hostname and one_ip in ansible_all_ipv4_addresses"

```

Now, I do have the variable with some values, BUT if I try to use this variable (in this case the variable es `nfs_mount_output`) in a task that will run on both hosts, when the role runs the role in the host that has NOT the variable defined in the previous tasks ansible complains telling me that the variable is not defined.

What I did was to create another task that set the a new variable with the previous variable, something like:

```
yaml
- set_fact:
    nfs_mount_output_globlar: "{{ nfs_mount_output }}"

```

But I guess there is a better way to achieve it...

So the question is: can I make those variables somehow global(?) inside the task?

# Answer
> 1 votes
It is a little bit tricky with Ansible. And also depends on the version you use. But I suppose you use relatively newest one.

I hope my answer helps, and if not, it gives you new ideas to explore :) And that I get your issue properly.

You can define global variable, however Ansible understoods it in specific way. If you declare it through configuration or environment vars, then it will be seen as "global".

For example, you can define it in your `inventory` file:

```
[myservers]
server1
server2
...

[myservers:vars]
foo="bar"

```

You can define variable in `host` section of your play:

```
- name: my play
  hosts: myservers
  vars:
    foo: bar

```

And also, you can work with `facts`, but as you already said, sometimes it doesn't look like the "right" option.

You can find some best practices for variables on Spacelift blog post (disclaimer: I am Developer Advocate there).

---
Tags: ansible
---