# Question
Title: How reliable is NFS for using in Kubernetes?
I'm setting up a self-managed Kubernetes cluster where I'm limited by the number of choices for the backend storage for Pods.

I should use a remote storage, so that multiple Pods can read/write at the same time, and Pods can get migrated easily to other nodes. Of course the reliability of storage is important.

Due to hardware limitations, Ceph is not an option. GlusterFS is also depreciated.

NFS seems to be my only option. So my question is, how reliable is NFS for using in Kubernetes? Would applications (mostly Java-based) and databases (mysql, postgres) behave abnormally?

# Answer
NFS is a commonly used storage option for Kubernetes, but it is not as reliable as some other options such as persistent volumes or distributed file systems. This is because NFS relies on a single server for storage, which can be a single point of failure. Additionally, NFS can experience performance degradation when handling high volumes of data or a large number of concurrent connections.
> 1 votes

---
Tags: docker, kubernetes, containers, storage, kubernetes-volume
---


# Question
Title: Why don't I see conflicts in git, but ADO shows conflicts, and how to fix this?
I have already read How to resolve merge conflicts in Azure DevOps current UI but nothing there helped. Also, no, I cannot use "Pull Request Merge Conflict Extension", I am not the admin of the box, therefore I cannot add extensions.

ADO shows there is a conflict. However if I get pull the branch I see no conflicts.

I also tried selecting the file in ADO under "Conflicts" and "Take source `<repo>`". After that ADO shows that the conflict is resolved. But when I go back to the PR, the conflict still shows.

Why am I seeing no conflicts in git, but ADO shows conflicts? How do I fix this?

# Answer
Think I figured it out …

Make sure you are in the branch you are trying to merge. For the example "new\_branch" is the new one I made a PR for, which is merging into "dev"

```
git branch

```

Change to the correct branch if you are not in the new\_branch branch

```
git checkout new_branch

```

Perform a merge

```
git merge dev

```

Find conflicts / fix

```
git conflicts

```

My problem was the dev branch had a single file called "create\_new" which was old code. The new branch I created had many changes and I created a new folder called "create\_new", the conflict was the folder name. This was why it was so hard to figure out what the problem was.

Check status that changes will be made on merge

```
git status

```

Commit the changes, this will create a new merge/commit, keep as is and save to push with the merge.

```
git commit -a

```

Now in ADO, the conflict is gone.
> 0 votes

---
Tags: azure-devops, git
---


# Question
Title: Is it possible to run a droplet on Digital Ocean without a public IP?
I am provisioning droplets on Digital Ocean with Terraform. I would like some of those droplets to be blocked from the internet, except for the Shared Private Network, and their appropriate gateway on that Network.

Is it possible to do this with Terraform?

# Answer
> I would like some of those droplets to be blocked from the internet The only way to do this on Digital Ocean is with the appropriate firewall rules

The only way to do this (Terraform or otherwise) is to create the firewall rules and associate them to the droplets. The droplets are always assigned a public IP address by Digital Ocean, and the network firewall can be used to manage access via that endpoint.

However, internal configuration of the machine can be done to configure the droplets to use a custom preconfigured gateway and not use the public interface as a gateway.

Digital Ocean has a writeup on how they suggest this, using IP Tables and cloud-init:

* set up gateway server as NAT router `sysctl -w net.ipv4.ip_forward=1`
* change default route to your NAT gateway: `ip route change default via <private IP of gateway host>`
> 1 votes

---
Tags: terraform, digital-ocean
---


# Question
Title: Where can I find GitHub's key id to import key for github_repository_deploy_key resource?
I've added GitHub Deploy key manually. Now I would likt to `import` it to my state. But when I look into github\_repository\_deploy\_key resource docs I can read

> *"Repository deploy keys can be imported using a colon-separated pair of repository name and GitHub's key id. The latter can be obtained by GitHub's SDKs and API."*
>
> *`$ terraform import github_repository_deploy_key.foo test-repo:23824728`*

I was looking for this GitHub's key id for a long time but without a luck. If I go to https://github.com/\/\<REPO\>/settings/keys I cannot see any ID (numerical) in my key information.

I've try to use `SHA256:` bit from deploy key information, but it end's up with the following error:

> Error: Unexpected ID format ("ddf62fc..."), expected numerical ID. strconv.ParseInt: parsing "ddf62fc...": invalid syntax

How can I find my GitHub's (deploy) key ID?

# Answer
> 1 votes
OK, I found it..

```
gh api https://api.github.com/repos/<OWNER>/<REPO>/keys | jq .[].id

```

---
Tags: terraform, github
---


# Question
Title: IAM Permissions issue in API Gateway Deployment
AWS threw a below error saying the user doesn't have access to `apigateway:TagResource` permission, but when I checklist of available permissions in the IAM console, I'm not able to find any such permission.

```
xxxx is not authorized to perform: apigateway:TagResource on resource xxx

```

Currently, I've given the below permissions for that API.

```
{
    "Effect": "Allow",
    "Action": [
        "apigateway:DELETE",
        "apigateway:PUT",
        "apigateway:PATCH",
        "apigateway:POST",
        "apigateway:GET"
    ],
    "Resource": [
        "arn:aws:apigateway:*::/restapis*",
        "arn:aws:apigateway:*::/apikeys*",
        "arn:aws:apigateway:*::/usageplans*",
        "arn:aws:apigateway:*::/domainnames*",
        "arn:aws:apigateway:*::/tags*",
        "arn:aws:apigateway:*::/apis*"
    ],
    "Condition": {
        "StringLikeIfExists": {
            "apigateway:Request/apiName": "my-api*"
        }
    }
}

```

How to resolve this, any other permission is required?

# Answer
> 1 votes
I was dealing with a similar issue. Ultimately, the following worked for me:

```
{
  "Effect": "Allow",
  "Action": [
    "apigateway:POST",
    "apigateway:TagResource"
  ],
  "Resource": [
    "arn:aws:apigateway:*::/apis",
    "arn:aws:apigateway:*::/apis/*/stages"
  ],
  "Condition": {
    "StringEquals": {
      "aws:RequestTag/WbyProjectName": [
        "webiny-js"
      ]
    }
  }
}

```

---
Tags: amazon-web-services, aws-iam, api-gateway
---


# Question
Title: How crunchydata Postgresql operator and Service works
We have deployed a PostgreSQL with crunchydata operator. We followed the link `https://access.crunchydata.com/documentation/postgres-operator/v5/quickstart/#connect-using-a-port-forward` and connected with port-forward successfully to pods. Further, our services works and connect to Databases through Service created.

My question is:
Service that created is with name my-test-db-primary. This service has no pod selector, however the pods are connecting while port-forward not. How this works and how it knows the destination?

```
kind: Service
apiVersion: v1
metadata:
  name: my-test-db-primary
  namespace: test-databases
  labels:
    postgres-operator.crunchydata.com/cluster: my-test-db
    postgres-operator.crunchydata.com/role: primary
  ownerReferences:
    - apiVersion: postgres-operator.crunchydata.com/v1beta1
      kind: PostgresCluster
      name: my-test-db
      controller: true
      blockOwnerDeletion: true
spec:
  ports:
    - name: postgres
      protocol: TCP
      port: 5432
      targetPort: postgres
  clusterIP: None
  clusterIPs:
    - None
  type: ClusterIP
  sessionAffinity: None
  ipFamilies:
    - IPv4
    - IPv6
  ipFamilyPolicy: RequireDualStack
  internalTrafficPolicy: Cluster
status:
  loadBalancer: {}

```

# Answer
Further reading the k8s documentation Documentation

Quoting from Documentation:

> When you create an EndpointSlice object for a Service, you can use any name for the EndpointSlice.
>
> Each EndpointSlice in a namespace must have a unique name. You link an EndpointSlice to a Service by setting the kubernetes.io/service-name label on that EndpointSlice.
> 0 votes

---
Tags: kubernetes, postgresql
---


# Question
Title: How can I map a domain to docker containers?
I have a Redhat Linux server. In that, I installed docker and run docker containers in it. Now I want to map a domain to docker containers by using Nginx.

What is the correct process to map domains by using Nginx, install Nginx as a container or Install Nginx in the Host server? Any references?

# Answer
> 0 votes
**Basics** Nginx is a webserver that can be used as a reverse proxy. I would suggest having a look at the nginx.conf file, where you can configure DNS redirects accordingly.

**Host** Nginx on host means you would need a port mapping( host -\> docker container) of all the docker containers in order to redirect traffic to the correct container. Wouldn't suggest this.

**Docker** Nginx on docker on same docker network would be my suggested approach: you can use docker DNS and only map the nginx server itself(ports 80/443).

Maybe a tool based on nginx would help here, which already has lets encrypt support: https://nginxproxymanager.com/

---
Tags: docker, docker-compose, nginx
---


# Question
Title: kubelet connecting to docker.sock instead of containerd.sock
Kubernetes version: latest Installation method: kubeadm Host OS: debian9 container runtime: containerd

Kubelet fails when starting and after inspecting the logs using journalctl -xeu kubelet i see that it tries to connect to the docker.sock socket instead of containerd.sock. I tried running kubeadm --config=config.yaml using this config file

```
apiVersion: kubeadm.k8s.io/v1beta1
kind: InitConfiguration
nodeRegistration:
        criSocket: "unix:///var/run/containerd/containerd.sock"

```

I have also tried running the kubeadm with the container runtime 85 argument, also doesn’t work. any other ideas are appreciated and thanks

# Answer
> 2 votes
In kubeadm docs

> --cri-socket stringPath to the CRI socket to connect. If empty kubeadm will try to auto-detect this value; use this option only if you have more than one CRI installed or if you have non-standard CRI socket.

# Answer
> 2 votes
I'm not sure it's correct, but if anyone encounters similar issue, you can try again by adding the following parameters to `/var/lib/kubelet/kubeadm-flags.env`

```
KUBELET_KUBEADM_ARGS="--container-runtime=remote \
--container-runtime-endpoint=unix:///run/containerd/containerd.sock \
"

```

---

I had a similar error while updating the cluster I set up with kubespray. In this case, the file I needed to edit was `/etc/kubernetes/kubelet.env`

```
KUBELET_ARGS="--container-runtime=remote \
--container-runtime-endpoint=unix:///run/containerd/containerd.sock \
"

```

Then I restarted the kubelet service and it started working with containerd instead of docker as I expected.

The answer I gave may not have been able to fully answer the question asked, but if there are those who come here by looking at the title of the question, one of the two methods I suggest will be useful.

---

Edit After Serve Laurijssen's answer;

According to the kubernetes docs, if you are using kubeadm, you should be aware that this tool stores cri sockets as annotation for each node, so the new cri socket must be annotated for each node.

You can do this using below command for each node you have;

```
kubectl annotate node $NODE_NAME --overwrite kubeadm.alpha.kubernetes.io/cri-socket=unix:///run/containerd/containerd.sock

```

Or according to above doc, you can also use `kubectl edit node $NODE_NAME` and change cri socket annotation manually.

# Answer
> 1 votes
Apart from adding to the KUBELET\_ADM\_ARGS variable you should also set every node config to containerd.

```
kubectl edit no nodeX.example.com

kubeadm.alpha.kubernetes.io/cri-socket: /var/run/containerd.sock

```

---
Tags: containers, kubeadm
---


# Question
Title: Running builds requiring docker daemon in Jenkins installed using helm and running as containerd container
Jenkins is installed using official helm chart on Kubernetes cluster that uses `containerd` as container runtime.

How to configure containerized agents which will run docker client commands e.g. for building docker images.

There's no docker daemon on Kubernetes nodes. Is there an option to use containerd client which will interact with containerd server on the nodes? The idea is similar to case where docker client inside the container interacts with docker daemon running on the node.

I'm also thinking about below options for Jenkins agents:

Is there a best practice?

# Answer
Eventually, I chose to use docker daemon in docker as a long running pod and Jenkins agents that have docker client inside triggered on demand. They interact with containerized Docker daemon.

I summarized my choice of Jenkins Docker in Docker Agent.
> 0 votes

# Answer
There are ways to do it but I've not been able to manage it. I would recommend sidestepping the problem by using kaniko. You can add a purpose built kaniko container into your 'build' agent k8s podTemplate and let it handle the container build (and push) part of the workload with no docker daemons or sockets necessary.

It's working well for me and I am also using jenkins via helm on k8s. Here is a snippet of my helm chart `values.yaml` agent.podTemplates section.

Note that the 'builder' agent contains 2 containers (actually 3 including the default 'jnlp' container which handles communicating with the jenkins controller) and they share a common workspace:

```
agent:
  namespace: jenkins
  podTemplates:
    builder: |
    - name: 'builder'
      label: 'builder'
      serviceAccount: jenkins
      nodeUsageMode: EXCLUSIVE
      yamlMergeStrategy: "merge"
      yaml: |-
        apiVersion: v1
        kind: Pod
        metadata:
          name: builder
          namespace: jenkins
        spec:
          serviceAccountName: jenkins
          containers:
          - name: builder
            image: builder:latest
            imagePullPolicy: IfNotPresent
            command: ["sleep"]
            args: ["99d"]
            tty: true
            resources:
              requests:
                cpu: 1
                memory: 1Gi
              limits:
                cpu: 2
                memory: 2Gi
            workingDir: /home/jenkins/agent
          - name: kaniko
            image: gcr.io/kaniko-project/executor:debug
            imagePullPolicy: IfNotPresent
            command: ["sleep"]
            args: ["99d"]
            tty: true
            volumeMounts:
            - name: docker-config
              mountPath: /kaniko/.docker
            workingDir: /home/jenkins/agent
            env:
            - name: DOCKER_REPOSITORY
              value: "private_docker_repository"
          volumes:
          - name: docker-config
            configMap:
              name: docker-config

```

Note that I used the `yaml: |-` field because it was the only way I could find to mount a volume containing the configMap needed by kaniko for docker authorization, configuration reference. So I also kubectl applied a simple docker-config configMap in the jenkins namespace first before applying this helm chart values.yaml.

```
apiVersion: v1
kind: ConfigMap
metadata:
  name: docker-config
  namespace: jenkins
data:
  config.json: |-
    {
       "credsStore": "ecr-login"
    }

```

Find more details on what that docker-config needs to contain for you here.

And then here is an example of using the different containers in a declarative Jenkinsfile script:

```
pipeline {

  agent {
    label 'builder' // Refers to the custom k8s pod template defined in the jenkins helm chart values.yaml
  }

  environment {
    RELEASE_TAG = "1.0.0"
  }

  stages {
    stage('Build') {
      steps {
        container('builder') { // refers to the 'builder' container within the 'builder' agent pod
          echo "do a maven build or whatever other steps you need to take here"
        }
        container('kaniko') { // refers to the 'kaniko' container within the 'builder' agent pod sharing the same workspace
          /kaniko/executor --dockerfile `pwd`/dockerfile --context `pwd` --destination=${DOCKER_REPOSITORY}:${RELEASE_TAG}
        }
      }
    }
  }
}

```
> 1 votes

---
Tags: docker, jenkins, kubernetes, docker-in-docker, containerd
---


# Question
Title: aws_availability_zones returns an availability zone ap-southeast-2c that is not available
I'm attempting to create an AWS RDS Serverless v2 cluster in region `ap-southeast-2` (Sydney) using the following Terraform configuration:

```
provider "aws" {
  region = "ap-southeast-2"
}

data "aws_availability_zones" "available" {
  state = "available"
}

resource "aws_rds_cluster" "backend_app_db_cluster" {
  availability_zones = data.aws_availability_zones.available.names
  # ...
}

```

When I try to `apply` this configuration, I get the following error:

> InvalidVPCNetworkStateFault: Availability zone '\[ap-southeast-2c\]' is unavailable in this region, please choose another zone set.

Full output:

```
module.database.aws_rds_cluster.backend_app_db_cluster: Creating...
╷
│ Error: creating RDS Cluster (<munged>20221204221217538100000001): InvalidVPCNetworkStateFault: Availability zone '[ap-southeast-2c]' is unavailable in this region, please choose another zone set.
│   status code: 400, request id: <munged>
│
│   with module.database.aws_rds_cluster.backend_app_db_cluster,
│   on modules/database/main.tf line 54, in resource "aws_rds_cluster" "backend_app_db_cluster":
│   54: resource "aws_rds_cluster" "backend_app_db_cluster" {

```

If I specify the availability zones manually and specifically exclude `ap-southeast-2c`, it works as expected:

```
data "aws_availability_zones" "available" {
  exclude_names = ["ap-southeast-2c"]
  state         = "available"
}

```

```
$ terraform apply
...
module.database.aws_rds_cluster_instance.backend_app_db_cluster_instance: Creation complete after 12m43s [id=tf-<munged>]

Apply complete! Resources: 3 added, 0 changed, 1 destroyed.

```

Why is `aws_availability_zones` returning an AZ that's unavailable?

# Answer
Do you have a subnet in ap-southwest-2c? In order to build the the Subnet group for RDS you have to have subnets in all AZs. If you don't have, Terraform throws this error received from API.
> 2 votes

---
Tags: amazon-web-services, terraform, amazon-rds
---


# Question
Title: ansible fatal: "sudo: a password is required", even after priveleges are escalated
while trying to setup a remote machine using ansible, I am running into the following issue:

```
FAILED! => {"changed": false, "module_stderr": "sudo: a password is required\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

```

I am already escalating privileges in the playbook using the following method:

```
  become: yes
  become_method: community.general.sudosu

```

two of the tasks are successfully being executed with escalated privileges as well. i am running into this error when I run the following task:

```
    - name: Pushing Nebula configs to node
      synchronize:
        src: <local dir>
        dest: "/etc/<target_dir>/"
        dest_port: "{{ ssh_port }}"
        mode: push
      delegate_to: localhost

```

the user I am using does not require a sudo password.

# Answer
> 0 votes
Try this, please

```
- name: Pushing Nebula configs to node
  synchronize:
    src: <local dir>
    dest: "/etc/<target_dir>/"
    dest_port: "{{ ssh_port }}"
    mode: push
  delegate_to: localhost
  vars:
    ansible_become: false

```

---
Tags: ansible
---


# Question
Title: Passing Gradle credentials into Jenkins `dockerfile`agent?
I am looking at getting a newly provisioned Jenkins to run an existing dockerfile which does a gradle build and produces a runtime image which is then to be pushed to our internal docker repository.

I have the following so far:

```
pipeline {
    agent {
        // https://www.jenkins.io/doc/book/pipeline/docker/#dockerfile
        dockerfile {
            // https://www.jenkins.io/doc/book/pipeline/syntax/#agent-parameters
            additionalBuildArgs  "--build-arg 'GIT_REPO=(git remote -v | head -1)' " +
                    "--build-arg 'JENKINS_BRANCH=$BRANCH_NAME' " +
                    "--build-arg 'JENKINS_BUILD_URL=$BUILD_URL' " +
                    "--build-arg 'GIT_SHA=(git rev-parse HEAD)' " +
                    "--build-arg 'ORG_GRADLE_PROJECT_artifactoryUsername=setnow' " +
                    "--build-arg 'ORG_GRADLE_PROJECT_artifactoryPassword=setnow'"
            registryCredentialsId "svc-artifactory"
            label 'set by us'

        }
    }

```

and I would like to provide the proper credential to the gradle build inside by setting the `ORG_GRADLE_PROJECT_*` variables. The credential to be used is "svc-artifactory" for this as well.

How am I to approach this?

(The GIT\_\* arguments are disabled shell commands I'm coming back to afterwards. Hints appreciated)

# Answer
I would suggest to use Docker secrets with mount command.

```
RUN --mount=type=secret,id=secret_name myVariable=$(cat /run/secrets/secret_name) && $command

```

After that use withCredentials function and parse secret ID to a variable.

```
withCredentials([string(credentialsId: 'jenkinsSecretName', variable: 'mySecretENV')]) {
  docker build -f Dockerfile --secret id=secret_name,env=mySecretENV .
}

```

Please be aware that Docker secret is available only on this specific layer where it is defined and you will not be able to access it on another layer later.
> 2 votes

---
Tags: docker, jenkins, jenkins-pipeline, gradle
---


# Question
Title: How to escape dollarsign in groovy shell command?
I'm making a declarative Jenkins pipeline which has a for loop to iterate over the output of a bash command. In that command it is necessary to use bash arguments e.g. "$1" for first argument. And it is a series of pipes with many types of quotes.

I tried using the triple double quotes for a raw string in groovy but it seems groovy is still trying to parse "$1" as me trying to reference the value of a variable named "1". So, this is not expected "raw string" behavior.

How does one go about doing this?

Here's an example of what the command is like.

```
kubectl get namespaces --no-headers -o=custom-columns=":.metadata.name,:.metadata.creationTimestamp" | grep -E "myproject_" | awk '$2 <= "'$(date -d'now-2 days' -Ins --utc | sed 's/+0000/Z/')'"{print $1}' | sed 's/myproject_//g'"""

```

Ignore the specifics of the command. My question is purely how does one get around the dollar sign syntax errors. The actual command is something different.

# Answer
Either use a single-quoted string, or escape the dollar sign with a backslash:

```
echo('This is a literal dollar sign: $')
echo("This is a literal dollar sign: \$")

```

Both will print `This is a literal dollar sign: $` to the Jenkins console output.

By the way, this question was already asked and answered over on StackOverflow.
> 1 votes

---
Tags: jenkins, jenkins-pipeline, groovy
---


title: Ansible event driven capacity

text:
# Question
Title: Ansible event driven capacity
I've been using SaltStack a lot in my previous job, and loved its event driven model. Now, in my new job, we are using Ansible, and definitely not in an event driven model.

I was wondering if there was this capacity hidden (or not hidden :)) anywhere in Ansible ?

Also if you think that it was a good idea to try and implement this event driven model based on Ansible (or if Ansible was not designed to be used like this and it would be an mistake).

Thanks

\[EDIT\] I got what Arcege is telling me, my question was if there was an event-driven sub-system in Ansible : The answer is NO.

So my second question is : What event system would you plug into Ansible in order to get a truly automated infrastructure ? (*e.g.* Kafka)

# Answer
> 2 votes
I can't say that I've used SaltStack, but I have a good deal of experience with Ansible. The two are different approaches. Event based use is not hidden, it is just not there by itself.

Ansible is agent-less (no minions), and as such it relies on a management system to be able to control. This adds flexibility on how to manage events, where they come from, how they are raised and what to do in an automated fashion. This means that it can be used for provisioning through CI/CD, fixing faults through a monitoring system, combined with a server agent like Telegraf.

The questions I think that should be asked is: what would trigger the tool, where it should run and what should be performed. The rest is just getting the right tools to link up. Generally, it will involve a CI/CD system, but there are plenty of those.

Some example uses cases:

* detecting a flake in a build on a bare-metal server in a pipeline, triggering to re-provision the server.
* a trigger coming from the PR system, to start a build to update test servers.
* Telegraf detects a disk issue, which triggers an alert which can then start a build to resize the disk or provision a larger server and transfer services there.

Except for the Telegraf use case, the events are likely coming from sources other than the agent running. And then with the Telegraf case, the option of provisioning a different server would be outside the scope of SaltStack.

To answer your last question of even to use a event-driven model, it's a matter of necessity and choice depending on the use cases.

# Answer
> 2 votes
There's a new Ansible Event-Driven automation. It constantly listens to triggers from variety of sources such as - Altertmanager, Kafka clients etc.

Official Site - https://www.ansible.com/use-cases/event-driven-automation

For detailed review check out this:- https://www.linkedin.com/pulse/testing-new-event-driven-ansible-prabhushakti-haridaswa

# Answer
> 0 votes
Most likely, the event driven system you refer to is SaltStack's Reactors and Beacons. As it turns out, Ansible allows the use of Connection plugins to support other transport methods, one of which is SaltStack. This means that there is nothing stopping you from having your cake and eating it too - you can use Ansible to run a playbook using a saltstack minion and use you minions to run Ansible playbooks.

In short why not use SaltStack to drive your Ansible events? You already know it and love it, so why not use what's already familiar?

---
Tags: ansible, automation, salt
---


# Question
Title: ArgoCD Autosync based off of Docker Image `latest` Tag
Using ArgoCD, I'd like to have the App AutoSync Update/Reploy when the docker image is updated on a specific tag such as `latest` or `stable`.

For example, if I have a deployment in my git repo like this:

```
apiVersion: apps/v1
kind: Deployment
metadata:
  name: k8s-nginx-example
spec:
  selector:
    matchLabels:
      app: k8s-nginx-example
  replicas: 2 # tells deployment to run 2 pods matching the template
  template:
    metadata:
      labels:
        app: k8s-nginx-example
    spec:
      containers:
      - name: nginx
        image: my-repo/k8s-nginx-example:latest
        ports:
        - containerPort: 80
        resources:
          requests:
            memory: "64Mi"
            cpu: "250m"
          limits:
            memory: "128Mi"
            cpu: "500m"

```

I'd like ArgoCD to update the deployment when a new docker image is pushed to `latest` tag or `stable`... etc. I know that ArgoCD polls the git repo of the app every 3 seconds so it is not aware of the changes to the Docker Image Tag. Another option could be (but I'm not sure how to do it) would be to have ArgoCD Update via a webhook that notifies ArgoCD that there is a new image to pull with the `latest` tag, although even after pushing a Docker Image to the repository with the `latest` and then manually `syncing` the app in ArgoCD, Argo does nothing. I suspect that Argo is only reaching out to the git repo to check for changes and is not concerned with the Docker image in the deployment.

The only way I can think of doing this currently with ArgoCD is to version the Docker images with version tags `1.2.3` and then script the build process to update the deployment in the git repository with the image tag version each time.

```
...
containers:
  - name: nginx
    image: my-repo/k8s-nginx-example:1.2.3
    ports:
...

```

Is there a way to do this in ArgoCD? Or would a better option be using Argo Workflows with Argo Events? I was hoping that there might be a plugin or something that can either trigger the event or monitor the docker image.

Thanks

Meta: Need tags Argo and ArgoCD

# Answer
This isn't really an ArgoCD thing. Keel is a good tool to help keep your images updated across the cluster and is super easy to implement. Basically, you'd just add some annotations to your manifest.

```
metadata:
  annotations:
    keel.sh/policy: force
    keel.sh/trigger: poll
    keel.sh/pollSchedule: "@every 10m"

```

It may be worth noting that if you are moving to run this in production, avoid the use of the latest tag. This is considered bad practice as it can burn you when breaking changes are pushed. Keel can handle that through an innate understanding of semver tags allowing you to update on minor version changes, but avoid major without testing, etc.
> 8 votes

# Answer
The canonical, ArgoCD-native way to do so is to use the ArgoCD image updater.

It is currently being mainlined:
> 4 votes

---
Tags: docker, kubernetes
---


title: Why is NodePort discouraged?

text:
# Question
Title: Why is NodePort discouraged?
In Kubernetes, is there any general reason to avoid NodePort services? In what particular circumstances are NodePorts appropriate?

# Answer
> 2 votes
First, let's review quickly what is `NodePort` service.

### What is NodePort? How does it work?

* **Internal Service** accessible on `Service IP address + Port`
* NodePort also creates ClusterIP internal Service
* NodePort, Opens Port on **each Worker Node**
* External traffic has access to a fixed port on each Worker Node! Range: `30000-32676`

NodePort Service Accessibility:

* NodePort accessible from outside the cluster
* On IP address of `Node IP + The NodePort`
* NodePort accessible from outside the cluster

---

Well, it is an excellent choice for testing purposes.

But in production is not a good choice.

Why?

Because you should type each worker node IP with the corresponding NodePort.

InProduction, we want only 1 Ip that automatically LoadBalance between each worker Node.

So, let's quickly review the `LoadBalancer` type.

### Why LoadBalancer?

At this point, our application is accessible from the outside, but it has a couple of downgrades:

* Not user-friendly (`Node IP:NodePort`)
* Insecure & messy!
* is it OK to test
* Better Alternative: LoadBalancer Service Type

```
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
  labels:
    app: nginx
    svc: nginx-test
spec:
  type: LoadBalancer # The configuration of Loadbalancer is exactly like NodePort except for this part...
  selector:
    app: nginx
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 80
      nodePort: 30000 # Notice we still have a nodePort!

```

#### How LoadBalancer works?

* LoadBalancer outside of K8s cluster
* Which accepts traffic as an entry point
* **LoadBalances to 1 of Worker Node on the NodePort**
* Then gets forward to ClusterIP
* Accessible at own IP address & port
* LoadBalancer is not created inside the cluster

---

Okay, so we learned that LoadBalancer is a better alternative in production.

But, still, it is not the Best practice and the best solution in **Production**.

The Best Practice in the Production environment is the `Ingress` type.

---

### Why Ingress?

* Self-Managed K8s cluster: Create LoadBalancer yourself
* Managed K8s cluster: Creates LoadBalancers automatically
* Loadbalancer Disadvantages:

+ LoadbBlancer that all become entry points
    + Configure the Domain Name
    + Each LoadBalancer exposes new NodePort
    + Each Loadbalancer increases the cloud bill
    + Configure everything outside the cluster
* Isn't it good:

+ Having this as part of the K8s cluster?
    + Configure secure connection
    + LoadBalancing to different services

#### Ingress

* A K8s component
* Configure `Routing`
* Configure `HTTPS`
* Ingress is deployed and available inside the cluster
* We need to expose it either as NodePort or LoadBalancer
* 1 NodePort or LoadBalancer, which is the single entry point

---

### Summary

* `ClusterIP`: `Internal` Access
* `NodePort`: `External` Access / Testing
* `LoadBalancer`: `External` Access / Production
* `Ingress`: `External` Access / Production / Best Practice

---

Be sure to check these resources as well:

# Answer
> 0 votes
NodePorts are discouraged by sources such as this article.

Some problems:

* Nonstandard port numbers get assigned to services. These are undesirable to use, due to lack of human readability and potential to be blocked by firewalls/gateways.
* Services may clobber each other. (For example, if a service gets randomly assigned a port number that another service was going to request.)
* Security may be more difficult to verify, with direct external traffic to many nodes (and ports).
* DNS may behave poorly (hostnames of different services may be synonymous, and address record sets may become too large to fit in UDP query responses).

Situations where NodePort is still justified:

* With a gateway (such as a load balancer) that translates automatically between external hostnames and internal port numbers.
* For temporary testing, although `kubectl port-forward` is preferred where possible.

---
Tags: kubernetes
---


# Question
Title: How is 'self-healing' to be reconciled with Infrastructure as Code?
As a relative newcomer to the developments happening in Operations - DevOps, SRE, etc. - I'm struggling with a big-picture problem. The following is a specific example, using a network load-balancer.

From an IaC perspective, I take it that we should hold the configuration (ie. the pools for the different virtual servers) outside of the load-balancer itself. Changes to the load-balancer should be made in the first place to this external configuration and applied to the actual application via a pipeline.

From a 'self-healing' perspective - taking automated remedial actions to problems - we might want code to change the network load-balancer configuration in response to some event. If some sensor indicates a problem with a system, for example, we might want to have it removed from a pool.

Now, the obvious way of reconciling these two requirements is for the automated remedial actions to take place via a change to the load-balancer's external configuration. However, some of the reasons for wanting the IaC approach, and some possible implementations of IaC, can make this problematic. For instance, suppose that the external configuration of the load-balancer is held in yaml files in a Git repository, and there are controls over the check-ins (someone senior has to review them). In this case, the remedial action could get stuck in someone's to-do list.

Obviously the problem gets worse the more frequently and more rapidly we want such automated responses.

I was wondering what people's responses to this might be. Is the main problem using something like Git to hold the configuration? Is it a mistake to apply the same controls to all check-ins? Is it a mistake to try to treat the load-balancer set-up as 'configuration' in this way (note that it has some run-time state which isn't captured anywhere, like internal responses to health-checks that it runs). Would appreciate any thoughts.

# Answer
You do not specify any technology, but let me get an inspiration from one particular approach to this problem and maybe it can be then extended to other use cases. In AWS you have a load balancer (ELB) and an autoscaling group (ASG).

The load balancer has its own health check and the ASG may have its own as well. If an instance stops responding, it is removed from the ELB. If an ASG detects an unhealthy instance, it will just destroy it and it will rebuild a new one, which is then automatically added to the ELB again. The ASG knows its minimal number of instances and its maximum and it maintains the number of actual instances based on some metric (CPU load would be a simple example).

So the only use case when you need to change the configuration is when the system behavior gets out of the expected boundaries. We have some ASGs where we may have the minimum 3 and maximum 30 instances. Usually we have around 4, so if we need more than 30 - the Internet is probably falling down anyway.

Regarding the config change via git - that can work ok together with a full CI/CD pipeline with maybe some blue/green deployment. But maybe it should be considered to have the parameters stored somewhere outside of git unless you want to avoid all reusability and you want to always make copies of the definitions of your infrastructure. You can have an s3 versioned yaml/json file or SSM parameters (speaking still about AWS) or in general a persistent versioned parameter storage (no-sql database may work as well).

As mentioned in the comments - there is no silver bullet here, but it may be useful to stop thinking in the framework of a static universe. If your aim would be to make at least 2 production deployments every day, your "senior reviewer" will need to be probably replaced by something better.
> 3 votes

---
Tags: automation, infrastructure-as-code, sre
---


title: How to test the CI/CD pipeline itself?

text:
# Question
Title: How to test the CI/CD pipeline itself?
I have gitlab CI/CD defined through `.gitlab-ci.yml`.

It runs different jobs if the pipeline is triggered manually, from a merge request or from a scheduled pipeline.

It accepts a bunch of environment variables used to modify some job behaviors or launch specific jobs (for example for developers to build debug images).

When I change `.gitlab-ci.yml` I currently have to run manual tests triggering the pipelines in all possible ways with all the different variables to ensure I have no regression.

Do I have a way to automate that? Is there other CI/CD tools that allow that?

# Answer
You should use CI lint. See this to simulate your CI/CD pipeline and validate when you make changes to the yml file - https://docs.gitlab.com/ee/ci/lint.html
> 1 votes

# Answer
This is where using Terraform or some other Infrastructure as Code tool can really help. You could on a merge, create a test environment and then run through the various deployments to that environment. If you have some sort of acceptance tests built-in, you can verify that the builds/deployments still work, and that the code still functions. After the tests are done, you can teardown the test environment and proceed to deploy normally.

Another option would be to build a pipeline that triggers these builds you need to test. So when you change this root file -\> trigger one "build" that causes all of the dependent builds to fire.
> 0 votes

---
Tags: gitlab, testing
---


# Question
Title: How to make a CI/CD of an ASP.net core app to Linux compute engine instance
I'm using Google cloud, I have a linux VM up and ready for use, I have an ASP.net core web app on Azure devops. I have configured the build pipeline for this webapp, now I want to automate its deployment on my Linux server, but I don't know how to do that.

I have 2 options I tried:

* Building a docker container with my app and deploy on google container registry, but I couldn't find a way to later trigger the docker pull of my container on my linux VM to run it there, or run it with a docker compose.
* Building a self contained package of the ASP.net core app, but I can't find a way of carrying the package and deploying it to my VM instance.

There are several tutorials about deploying to Google app engine, GKE and Cloud Run. But I can't find one that shows how to do what I want with my linux VM.

Any help to overcome this?

# Answer
You need to develop some way of deploying your build into the Google Cloud VM. Your solution will vary depending on how you are hosting the asp.net:

If you are running it as a container on the VM, then you'll need some way of stopping the container, pulling the new one from a registry that the VM has access to, and restarting it. One way to achieve this would be to have your Azure DevOps pipeline execute a script local to the GCP VM that does these steps.

If you are using nginx or another webserver, you will need to copy the build artifacts to the new server. You'll need to use a protocol such as SSH to establish a connection from your Azure DevOps instance to the GCP Linux VM.
> 0 votes

---
Tags: docker, linux, google-compute-engine
---


# Question
Title: run ansible command as another user using their ssh key
i am trying to figure out how to run an ansible ping command as another user, using their ssh key.

User setup i have essentially:

* master\_user (me)
* ansible (sudo user, pwdless sign in)

I logged in as ansible user and did:

* ssh-keygen (generate key)
* ssh-copy-id XXXX (copy the ssh key to corresponding server)

Tested it all and it works fine (each server has ansible user already fyi). I updated the hosts file to have all my hosts as well.

Now when i remote into master\_user, i want to run a ansible command but as ansible user something like this:

```
ansible all --become-user=ansible -m ping

```

This fails because it is trying to ssh into the servers using the master\_users ssh key.

Error message: `Failed to connect to the host via ssh`

When i run -vvvv flag i can see it trying to user master\_user ssh by looks of it

```
Offending key for IP in /home/master_user/.ssh/

```

How can configure it so that it uses ansible user always and use ansible users ssh key?

# Answer
`become_user` will tell which user to become to run a particular ansible module **on the remote server**.

If you want to run the `ansible` command on your control machine as an other user, you need to use `sudo` or `su` on this machine e.g.

`sudo -s -u ansible ansible all -m ping`

The `-s` option will load the user's shell an make sure homedir and all other envs are correctly set prior to running the command.

An other possible solution would be to use the ssh key you created for the ansible user from your account

`ansible all --private-key=/home/ansible/.ssh/id_rsa -m ping`

This will potentially work only if your current user has read access to the given key.
> 4 votes

# Answer
If you're able to connect to the *host*

```
$ ssh ansible@<host>

```

then try

```
ansible <hosts> -u ansible -m ping

```
> 1 votes

# Answer
Also, you need to point to the ssh user's certificates created. It should like this:

```
sudo /usr/local/bin/ansible-playbook \
  example.yml \
  -i example_hosts.ini\
  --private-key=/home/example/.ssh/id_rsa

```

This worked for me.
> 0 votes

---
Tags: ansible
---


# Question
Title: How to get ride of running "sudo swapoff -a" command everytime before running "kubectl" commands?
It seems I must run `sudo swapoff -a` command everytime I want to run `kubectl` commands because if I don't do that it gives me such a following error:

```
The connection to the server 192.168.1.2:6443 was refused - did you specify the right host or port?

```

Is there a way to do that permanently and getting ride of always running `sudo swapoff -a` command?

# Answer
In order to disable the swap to a Linux machine, you have to elevate with root and edit `/etc/fstab`.
From there, remove the swap entry by commenting it out.
Upon reboot, swap will not mount.
> 1 votes

# Answer
This error is not only related to the swapfile.

It comes also as soon you join an additional node and you dont have a network fabric installed.

Like flannel https://github.com/flannel-io/flannel
> 0 votes

---
Tags: kubernetes, kubectl
---


title: Container logs for helm install

text:
# Question
Title: Container logs for helm install
Is it possible to print to `helm install` (or `helm upgrade`) output the logs of the deploying containers (logs that are available by `kubectl logs` command)?

# Answer
Helm does not launch pods to deploy your workloads. Helm locally renders the charts into standard Kubernetes manifests, and applies them. There is no server side component to Helm (since Helm 3).

If you want to get logs from deployed pods, do so with standard `kubectl logs` commands against the deployed pods. If you want to view what Helm is doing, use the `--debug` parameter to Helm, which will output the manifests that Helm applied.
> 1 votes

# Answer
Short answer: no. You would most likely run helm commands from your own workstation.

For those logs to be available through "kubectl logs", you would have to start a Pod, executing your "helm install" command. Maybe using a Job. Although this also means having some ServiceAccount with enough privileges to install objects on behalf of Helm.
> 0 votes

# Answer
You can try a workaround by using the --wait flag with `helm install` or `helm upgrade`. This will cause the command to wait until the deployment is complete and all of the pods are ready, and then it will print the logs for the pods.

example: `helm install my-release my-chart --wait`

This will install the chart and then wait until the deployment is complete before printing the logs for the pods.

You can also specify a timeout for the --wait flag, like so:

```
helm install my-release my-chart --wait --timeout 600

```

This will wait up to 10 minutes for the deployment to complete before timing out and returning.

The catch is that the `--wait` flag only works if the chart being installed or upgraded has a Deployment or StatefulSet as the primary resource type. It will not work with other resource types such as DaemonSet or Job.
> 0 votes

---
Tags: kubernetes, helm
---


title: GCP log based availability metric

text:
# Question
Title: GCP log based availability metric
we have a GKE hosted API served by Go/Gin service. It logs every request with some structured metadata (including status\_code). We'd like to create an availability SLI (which is defined as a proportion of good events / all events). So, we'd have to go through the logs and categorize the good/bad event by status code. Technically, my first bet would be to define log-based metric for that, however I am not sure whether such logic is achieavable with log based metric syntax alone. For some services ie. App Engine app or services hosted with Istio availability metric comes out of the box, but I have not found any predefined metric that could be used in this case (we do not have any api gateway in front of it)

# Answer
> 2 votes
You can use a metric filter to extract the status codes from your logs and use those values to calculate the proportion of good events to all events.

Here's an example of how you can set up a log-based metric for this purpose:

1. Go to the Cloud Logging page in the Google Cloud Console.
2. Select the log group for your GKE-hosted API.
3. Click the "Create Metric" button.
4. In the "Create Metric" dialog, specify a name for your metric (e.g. "API Availability") and a description (optional).
5. In the "Filter" field, enter a filter expression to extract the status codes from your logs. For example, you can use the following expression to extract the status codes from the "status\_code" field in your logs:

resource.type="gke\_container" AND jsonPayload.status\_code
6. In the "Aggregation" field, specify how you want to aggregate the extracted values. You can use the "COUNT" aggregation to count the number of occurrences of each status code.
7. Click the "Create" button to create the metric.

Once the metric is created, you can use the Cloud Monitoring UI or API to view the metric and track the availability of your API over time. You can also set up alerts to be notified when the availability falls below a certain threshold.

---
Tags: kubernetes, gcp, sre
---


# Question
Title: What is the usage of the cluster external IP address?
I'm confused about the usage of the cluster external IP address.

Is this an address that can be used for ingress to access pods running on the cluster?

If so should this be the same as the control plane machine IP address (I only have a single control plane)? Or rather should it be an unused IP address on the subnet that the cluster sits? For example, if I have the below setup:

```
Master: 192.168.86.50 Worker 1: 192.168.86.101 Worker 2: 192.168.86.102 Worker 3: 192.168.86.103

```

Should the external IP address of the master be set to 192.168.86.50 or could I set it to 192.168.86.20 for example?

Also I notice the workers also can take an external IP address should these be set to the same external IP address as the master? If not say they were 192.168.86.21 192.168.86.22 and 192.168.86.23 would that mean I could reach any pod (with ingress setup) to access it on 192.168.86.20, 192.168.86.21, 192.168.86.22 and 192.168.86.23?

I've done some reading around it but I'm still struggling to grasp the concept of external IP address.

# Answer
> 2 votes
External IP address is used to access applications from outside, which is running inside the cluster.

# Answer
> 1 votes
I wonder if you might be conflating multiple concepts when you say “cluster external ip”.

Each node in the cluster has (within the status information recorded by the kubernetes API server) a list of addresses that are believed to be able to be used to reach that machine. This can include internal and external ips. (External means *not only* useable from within pods on the cluster, so potentially public ips.)

If you have a node which you have assigned multiple static public ip addresses, you can create service resources of type **ExternalIP** so that traffic addressed to each of those public ip addresses will be forwarded (by kubeproxy on that node) to a different selection of pods in the cluster. (This could be slightly more user friendly than NodePort services, but it may create a bottleneck and also cause transient client errors when that node has to be swapped out e.g. for patches/upgrades. Also, public ips are a limited resource, whereas a load balancer can generally host multiple services with different external DNS hostnames through the same public ip.)

If you create a service resource of type **ClusterIP**, it will be assigned an ip address (and dns hostname) that is *not* directly reachable (except by other pods in the same cluster), necessitating some ingress/gateway for external access.

If you create a service resource of type **LoadBalancer**, then kubernetes will try to have your cloud provider provision a load balancer (or gateway proxy) to forward traffic to your pods, and then the service resource will be updated to note the external ip address of that particular load balancer. (However, it is typically more cost efficient to instead use an ingress controller that can configure one load balancer to be shared for multiple services.)

You would create a kubernetes service resource of type **ExternalName** if you want a local hostname that pods can use as an alias to refer to an externally provided service. (This allows changing where the service is hosted, without needing to reconfigure the pods that are clients of that service. However, some providers may not permit access under alternative hostnames.) It is implemented using kubedns.

---
Tags: kubernetes, networking
---


# Question
Title: Is it possible to assign a jenkins pipeline to a variable and run in groovy script?
I have a groovy script that defines a pipeline. The pipeline has several stages. I would like to assign that pipeline definition into a variable and run it later in the same groovy script. Is it possible?

I could not find a documentation on how to do that. Thanks in advance.

# Answer
I think what you are trying to describe can be done with a feature of the groovy programming language called closures.

Assuming you have a groovy script similar to this:

```
node() {
    stage('A') {
        println("I'm stage A!")
    }

    stage('B') {
        println("I'm stage B!")
    }
}

```

You could rerun this pipeline multiple times doing something like this:

```
node() {
    // Create the closure here
    def reusablePipeline = {
        node() {
            stage('A') {
                println("I'm stage A!")
            }

            stage('B') {
                println("I'm stage B!")
            }
        }
    }

    // Run the pipeline once
    reusablePipeline()

    // Run it a second time
    reusablePipeline()
}

```

If all the pipelines will run on the same `node()` than we can remove the second definition:

```
def reusablePipeline = {
    stage('A') {
        println("I'm stage A!")
    }

    stage('B') {
        println("I'm stage B!")
    }
}

```

If you find the reusable part of the job really long, you might want to load it from a second file instead. You can do this with the pipeline load function. Here is an example:

Put your reusable pipeline in a new file. Here I call it `pipeline.groovy`.

```
//contents of pipeline.groovy
def doStuff() {
    stage('A') {
        println("I'm stage A!")
    }

    stage('B') {
        println("I'm stage B!")
    }
}

// This is really important
return this

```

Now your pipeline would look something like this:

```
node() {
    def reusablePipeline = load('pipeline.groovy')

    // Run the pipeline once
    reusablePipeline.doStuff()

    // Run it a second time
    reusablePipeline.doStuff()
}

```

The above methods work great if you have a pipeline with a series of complicated steps and you would like to call those steps multiple times based on some criteria. A common use case is recalling a test suite with different parameters or re-running the test suite if some flaky tests fail.

But if you want to re-use pipeline steps across multiple Jenkins pipelines. Then you are looking for Jenkins Shared Libraries.

I have a public shared library with lots of utilities to make writing pipelines easier called the jenkins-std-lib. I'm also the creator of the Jenkins Extension Pack which makes it easier to write Jenkins pipelines using the VSCode IDE.
> 1 votes

---
Tags: jenkins
---


# Question
Title: Limit the number of pods bought up at the same time in Kubernetes
I maintain an application that has occasional problems caused by races during start up but need multiple instances running. Is it possible to limit the number of pods kubernettes brings up at the same time?

e.g. if it's configured to have 3 replicas and 2 of them are down, can I make sure it doesn't start two pods at the same time and instead wait for liveliness check to pass on the first new pod before starting the second one?

# Answer
> 0 votes
If you use a hpa to increase the deployment from zero to e.g. 3 replicas, you can set the scaling behaviour to limit it to say one new pod per 60 seconds. However, this won’t affect how quickly pods or containers are restarted if they crash at the same time.

Would definitely consider implementing some form of semaphore inside your image, to wrap the problematic process.

---
Tags: kubernetes
---


# Question
Title: How to upload a file as user input in Github Actions workflow?
I would like to upload .env file to a build in Github Actions. Is there any way I can do it?

# Answer
> 0 votes
It would be easier to store secrets in either secrets or specific environments in the repo:

```
Secrets:
https://github.com/**account_name**/**repo_name**/settings/secrets/actions

```

```
Environments:
https://github.com/**account_name**/**repo_name**/settings/environments

```

You can then access the secrets directly or generate an .env file when the GitHub actions runs.

In the workflow file access secrets with `${{ secrets.TOKEN }}` or generate an .env file and save secrets in there:

```
- name: do some stuff
        run: |
          touch .env
          echo TOKEN='${{ secrets.TOKEN }}' >> .env
          ...etc

```

If using environments to store secrets, don't forget to declare which environment is being used by that job! (documentation)

---
Tags: github, builds, github-actions, npm
---


# Question
Title: Ansible: How to get hostname without domain name?
How to get value from `/etc/hostname` ignoring domain name. For example, `/etc/hostname` says `"client.test.dom"` and I need to get `"client"`.

I have tried `"{{ ansible_nodename | replace('.rail.dom','') }}"` but I get syntax problem and can't figure out how to solve it.

# Answer
It is assumed that you have a proper and valid inventory defined. Then, according Information about Ansible: magic variables

> *You can use the magic variable `inventory_hostname`, the name of the host as configured in your inventory, as an alternative to `ansible_hostname` when fact-gathering is disabled. If you have a long FQDN, you can use `inventory_hostname_short`, which contains the part up to the first period, without the rest of the domain.*

If facts are gathered you can use additionally `ansible_hostname` or `ansible_facts['nodename']`.

A minimal example playbook

```
---
- hosts: localhost
  become: false
  gather_facts: true

  tasks:

  - name: Show hostname
    debug:
      msg: "Host: {{ ansible_hostname }} FQDN: {{ ansible_nodename }}"

```

will result into an output of

```
TASK [Show hostname] *********************
ok: [localhost] =>
  msg: 'Host: test FQDN: test.example.com'

```

---

You may then have a look into the following examples

```
  - name: Split nodename
    debug:
      msg: "{{ ansible_nodename.split('.')[0] }}"

  - name: Split nodename
    debug:
      msg: "{{ ansible_nodename | split('.') | first }}"

```

resulting both into an output of

```
TASK [Split nodename] ******
ok: [localhost] =>
  msg: test

```

**Further Documentation**

**Similar Q&A**
> 4 votes

---
Tags: ansible
---


# Question
Title: Reserve cpu and memory resources for pods in kubernetes?
Let say there's a daemonset, each pod of this daemonset requests 1 CPU and 1 Gi memory. For some reason pod terminated, during this time kubernetes allocate cpu/memory resource to other pods which causes daemonset pod re-create failed(Insufficient cpu/memory resource).

Is there any way to reserve 1 CPU and 1 Gi memory for this pod during it terminated ?

# Answer
> 0 votes
You could raise the scheduling priority of your pod so that it will preempt (evict) the other pods if necessary.

You could also check that there isn’t a restart policy causing your pod to end (rather than restart its containers) when the processes terminate.

Normally though, those aren’t supposed to be issues for daemonsets anyway. You shouldn’t need to try to pre-reserve capacity (other than by ensuring the resource requests are ample for the contained processes). You may need to configure your daemonset to skip particular node types that have too little capacity for your collection of daemonsets.

(To reiterate, the most likely cause of the error described by the question is if one of your nodes only has 3GB total of memory, and there are two different daemonsets that request 2GB each of memory, which is unsatisfiable regardless of other pods.)

---

Daemonsets are controlled quite differently from other pods. For a bare pod (or even a pod from a deployment) that you intend to assign to a particular node (say by setting `nodeName` or `nodeSelector` in the PodSpec, or merely by lack of a cluster autoscaler) then definitely do raise the scheduling priority (so that once created it will most likely be able to evict other pods if necessary to make space, notwithstanding several exceptions..). Also keep the default configuration of restarting any containers if they terminate (rather than relinquishing the requested memory back to the scheduler, then recreating a replacement pod..).

It simply is **not** a kubernetes pattern to reserve resources for a potential future pod. Or rather, you may think of a "pod" API object as already a pre-allocation for the node resources that could be consumed in future by *processes* in the pod's containers (but allocated before those containers even get created, and usually long before the processes will ever consume that amount of memory). Your pod could use an init container to wait arbitrarily after reserving node resources and before starting the main container process (and more complex orchestration might be achievable with a sidecar container configured to share process namespace..).

An exception is that placeholder pods are sometimes used to over-provision an autoscaled cluster. A pod with very low priority can still trigger the slow process of adding another node to the cluster, so that the new node has a chance to become ready *before* the scheduler needs it for a normal pod.

---
Tags: kubernetes
---


# Question
Title: How to point Environmental variable SONAR_JAVA_PATH to Java Executable?
After I launched command `StartSonar`, I get this error message:

> ERROR: java.exe not found. Please make sure that the environmental variable SONAR\_JAVE\_PATH points to the Java executable.

What is Sonar Java Path, and how to point Environmental variable SONAR\_JAVA\_PATH to Java Executable?

# Answer
> 0 votes
Download jdk 11 using the following link: https://www.oracle.com/in/java/technologies/javase/jdk11-archive-downloads.html. Open another command prompt after installing jdk 11 and enter "StartSonar" command.

---
Tags: java, sonarqube
---


# Question
Title: Kubernetes apply to get to desired state
I feel like I have a terrible knowledge gap when it comes to managing the resource states within Kubernetes.

Suppose I have 2 deployments in my cluster, `foo1` and `foo2`. They are both defined in separate `yaml` files, `foo1.yaml` and `foo2.yaml` that are both inside a `my-dir` directory and have been applied with `kubectl apply -f my-dir/`

Now I want to make a third deployment, but also delete my second deployment. I know that I can do this in 2 steps:

* Make another `foo3.yaml` file inside the directory and then do `kubectl apply -f my-dir/foo3.yaml`
* Run `kubectl delete -f my-dir/foo2.yaml` to get rid of the second deployment.

My question is, can I do this in one shot by keeping the "desired state" in my directory. i.e. Is there any way that I can delete `foo2.yaml`, create a new `foo3.yaml` and then just do `kubectl apply -f my-dir/` to let kubernetes handle the deletion of the removed resource file as well? What am I missing here?

# Answer
> 1 votes
The professional solution is something like fluxcd. This will automatically "apply" any .yaml files that you commit to your git repo, and also automatically remove those resources if it tracks that they have been deleted from the repo.

The problem with what you are asking for, is that kubernetes needs a way of knowing *which* resources you want it to delete. (For example, you may be using a shared cluster, and not want to inadvertently delete other applications and system pods, even if they are not explicitly mentioned in your private local working directory.) That said, if you *have* diligently labelled the specific resources you want to manage, then the feature *does* exist: use the `--prune` argument to `kubectl apply` as described in the docs.

---
Tags: kubernetes, kubectl
---


title: Error: a number is required

text:
# Question
Title: Error: a number is required
I'm in the process of upgrading our version of the Terraform AWS Provider and after fixing/updating every resource and data source which has been deprecated/changed between versions (`3.75.2` -\> `4.0.0`) I'm left with one final error message (`Error: a number is required`) I can't get to the bottom of because it doesn't give any indication of module/file/resource/line number or anything really to go by.

I get the error on a `terraform plan` as follows

```
$ terraform plan
aws_s3_bucket.static_nlb: Refreshing state... [id=<redacted>]
#
#...more resources refreshing state...
#
aws_ssm_parameter.processor_sqs: Refreshing state... [id=/config/scripts/sqs/iam_access_policy]

Error: a number is required

ERRO[0052] Hit multiple errors:
Hit multiple errors:
exit status 1
$

```

Environment Details:

* Terraform version: `0.14.11`
* Terraform AWS Provider: `4.0.0` (upgraded from `3.75.2`)

Troubleshooting/Things I've Tried

* Commenting out two of the modules I thought might be responsible and there was no change. I've tried running
* Running with `TF_LOG=DEBUG` or `TF_LOG=TRACE`

I didn't see anything in the output though it was looking for a needle in a haystack. I did try grepping for obvious key words: `type`, `error`, `number` etc.

Its a very large terraform project and I am at a bit of a loss how to proceed with debugging this error

Has anyone seen this before and can suggest what it is/how I might be able to dig further?

# Answer
Thanks @Martin Atkins, a big upgrade of terraform and the aws provider appears to have solved the problem.

After upgrading to terraform to `1.3.6` I got a bit more information about the nature of the error. It appears it was related to the object expiry property of s3 bucket object lifecycle policy configuration.

This was also manifesting itself by the object lifecycle property resource "flapping" between, needs creating and needs deleting every terraform run which was the motivation for embarking on this upgrade in the first place.

Upgrading AWS terraform provider to `4.0.8` resolved warnings and state flapping issues.
> 2 votes

---
Tags: amazon-web-services, terraform
---


# Question
Title: Argo workflows health checks failing in GKE
I am trying to expose my argo workflows deployment to a domain but I keep getting **Server Error** when I access the domain: argo.example.com. I setup a GKE ingress and been trying to fix the health checks by redirecting http to https, which is why I added a custom FrontendConfig and BackendConfig. However, I see the health check for "/" passing. See configuration below.

argo-server command args:

```
...
- args:
    - server
    - --namespaced
    - --auth-mode
    - server
    - --secure
    - "true"
...

```

ingress:

```
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: argo-ingress
  annotations:
    networking.gke.io/managed-certificates: "argo-cert"
    networking.gke.io/v1beta1.FrontendConfig: argo-frontend-config
spec:
  rules:
    - host: argo.example.com
      http:
        paths:
          - backend:
              service:
                name: argo-server-expose
                port:
                  number: 2746
            path: /*
            pathType: ImplementationSpecific

```

Frontend:

```
apiVersion: networking.gke.io/v1beta1
kind: FrontendConfig
metadata:
  name: argo-frontend-config
  namespace: argo
spec:
  redirectToHttps:
    enabled: true

```

Backend:

```
apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
  name: argo-backend-config
  namespace: argo
spec:
  healthCheck:
    checkIntervalSec: 30
    timeoutSec: 5
    healthyThreshold: 1
    unhealthyThreshold: 2
    type: HTTPS
    requestPath: /
    port: 2746

```

Service:

```
apiVersion: v1
kind: Service
metadata:
  name: argo-server-expose
  namespace: argo
  annotations:
    cloud.google.com/neg: '{"ingress": true}'
    cloud.google.com/backend-config: '{"ports": {"http":"argo-backend-config"}}'
spec:
  type: NodePort
  ports:
    - name: http
      port: 2746
      protocol: TCP
      targetPort: 2746
  selector:
    app: argo-server

```

# Answer
Do not use GKE ingress, it's a nightmare. Use nginx-controller helm package instead. You will have to configure something like this:

```
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: argo-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/backend-protocol: http
    cert-manager.io/cluster-issuer: "letsencrypt"
spec:
  rules:
    - host: argo.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: argo-system-argo-workflows-server
                port:
                  number: 2746
  tls:
  - hosts:
      - argo.example.com
    secretName: argo-tls-cert

```
> 0 votes

---
Tags: kubernetes, ingress, gcp, google-kubernetes-engine
---


# Question
Title: How do I prevent a given GitHub Workflow from being launched twice from two almost simultaneous events?
I have a repo, in which there is one folder at the root with a Makefile to build a C library. In the same repo, there is another folder at the root with a folder with another Makefile for building a node C addon via Node-API, and a small JS/TS API using said addon. The NAPI addon is dependent on the C library.

I have a workflow file for the C which runs when there is a commit on the C folder, and runs with any push or PR. I have another workflow file that runs when there is a commit+push/PR to the JS/TS folder, OR when the C workflow has completed successfully.

My goal is (of course) to only have the minimal amount of workflows run, depending on "how high in the stack the commits were". If commits touch only the C, I want both the C and NAPI addon to be checked. If commits touch only the NAPI addon, I want only the NAPI addon to be checked. If commits touch both the C and NAPI addon, I want both the C and NAPI to be checked; but for NAPI to only be checked once.

For now, when the commits touch both the C and NAPI addon, the CI for the NAPI addon is launched twice: once for the commits affecting the NAPI folder, once for the successful completion of the C workflow.

How do I fix this ?

NB: I've tried using `fkirc/skip-duplicate-actions@master`, but it didn't do the job.

# Answer
We have similar situations whereby you want the action to do different things based on the files changed within the commit. You could generate a list of files modified within the PR/commit (within your action) using something like (simplified for ease of reading):

```
# export a list of changed files

export FILESCHANGED=$( curl --header 'authorization: Bearer ${{ secrets.GITHUB_TOKEN }}' --header 'content-type: application/json' --silent https://api.github.com/repos/${{ github.repository }}/commits/${{ github.sha }} | jq '.files[].filename')

# writes changed files into a text file whilst removing quotes and substituting a space for a newline (each file will appear on a separate line).

echo ${FILESCHANGED} | tr -d [\"] | sed 's/ /\n/g' >> files_changed.txt

# display the list of changed files for visibility in the workflow

echo "Files changed in this commit"

cat files_changed.txt

```

you could then examine the list of files that are part of the commit/PR and run your logic accordingly in steps in a simple script

```

if C makefile is contained in the list
  run C makefile
else
  do not run C makefile.

if C makefile is contained and/or if NAPI makefile is contained
  run NAPI makefile
else if neither exist in list
  do not run NAPI makefile.

```

Logic will then dictate what runs based on what is contained in the commit also this would turn this into a single workflow from what I can see.

Hope that helps.
> 1 votes

# Answer
So in the same repo you have separate Makefiles for both components (the core library and the add-on), but one of those components (the library) is also actually a dependency to the other.

I suggest merging these into a single Makefile. In the GitHub workflow, run `make all` and use GitHub action caching so that *make* only rebuilds components if their dependencies have been modified.

Then, use the **concurrency** feature in GitHub workflows, if you don’t want multiple runs to overlap simultaneously.

https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#concurrency
> 0 votes

---
Tags: github, workflow, github-actions
---


# Question
Title: how to decrease the docker image layers latency during the image build step?
I am following the Microsoft document to create a custom image of vsts-agent on top of ubuntu and install the required capabilities. Since our environment is not having internet enabled, we used certain debian packages to install the capabilities and which resulted in multiple tasks in Docker file such as"COPY,"ENV setup for different apps,

Since we have these much steps, each is getting created in separate layer and which is causing vey slow image build.

```
FROM ubuntu:18.04
#2-Enable Ubuntu Packages
COPY ./sources.list /etc/apt/
#3- Install basic Softwares
ENV DEBIAN_FRONTEND=noninteractive
RUN echo "APT::Get::Assume-Yes \"true\";" > /etc/apt/apt.conf.d/90assumeyes
RUN apt-get update && apt-get install -y --no-install-recommends \
    ca-certificates \
    curl \
    wget \
    jq \
    git \
    iputils-ping \
    libcurl4 \
    libicu60 \
    libunwind8 \
    netcat \
    telnet \
    libssl1.0 \
    python \
    python3 \
    nodejs \
    python3-setuptools \
    python3-pip
    vim \
    openjdk-11-jdk-headless \
    gnupg \
    make \
    yarn\
    apt-transport-https \
    lsb-release \
  && rm -rf /var/lib/apt/lists/*
    #4. Copy Offline Packages
    COPY ./sw/* /tmp/
    #4-Install AzureCLI
    RUN  curl -sL https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor | tee /etc/apt/trusted.gpg.d/microsoft.gpg > /dev/null
    RUN AZ_REPO=$(lsb_release -cs) \
      && echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ $AZ_REPO main" | tee /etc/apt/sources.list.d/azure-cli.list \
      && apt-get update \
      &&  apt-get install azure-cli
    #Installl helm,docker, googlechrome and kubectl
    RUN  tar zxvf /tmp/helm-v3.8.2-linux-amd64.tar.gz  && mv /linux-amd64/helm /usr/local/bin/ && mv /tmp/kubectl /usr/local/bin/ \
         &&  apt install /tmp/google-chrome-stable_current_amd64.deb \
         &&  apt install /tmp/containerd.io_1.6.9-1_amd64.deb \
         &&  apt install /tmp/docker-ce-rootless-extras_20.10.9_3-0_ubuntu-bionic_amd64.deb \
         &&  apt install /tmp/libslirp0_4.6.1-1build1_amd64.deb      \
         &&  apt install /tmp/docker-ce_20.10.9_3-0_ubuntu-bionic_amd64.deb \
         &&  apt install /tmp/docker-compose-plugin_2.6.0_ubuntu-bionic_amd64.deb \
         &&  apt install /tmp/pigz_2.6-1_amd64.deb \
         &&  apt install /tmp/docker-ce-cli_20.10.9_3-0_ubuntu-bionic_amd64.deb \
         &&  apt install /tmp/docker-scan-plugin_0.9.0_ubuntu-bionic_amd64.deb \
         &&  apt install /tmp/slirp4netns_1.0.1-2_amd64.deb
    #8-install maven 3.8.6
    RUN mkdir -p /usr/share/maven /usr/share/maven/ref \
      && tar -xzf /tmp/apache-maven-3.8.6-bin.tar.gz -C /usr/share/maven --strip-components=1 \
      && echo "Cleaning and setting links" \
      && rm -f /tmp/apache-maven.tar.gz \
      && ln -s /usr/share/maven/bin/mvn /usr/bin/mvn
        # 8.1- Define environmental variables required by Maven, like Maven_Home directory and where the maven repo is located
        ENV MAVEN_HOME /usr/share/maven \
            TARGETARCH=linux-x64 \
            MAVEN_CONFIG "$USER_HOME_DIR/.m2" \
            JAVA_HOME /usr/lib/jvm/java-11-openjdk-amd64
        #11- Agent Installation
        WORKDIR /azp
        COPY ./vstsagent/ .
        COPY ./start.sh .
        COPY ./docker.sh .
        RUN chmod +x start.sh docker.sh
        ENV TARGETARCH=linux-x64
        # Can be 'linux-x64', 'linux-arm64', 'linux-arm', 'rhel.6-x64'.
        ENV TARGETARCH=linux-x64
        ENV JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
        ENV JAVA_HOME_11_X64=/usr/lib/jvm/java-11-openjdk-amd64
        ENV MAVEN_HOME=/usr/share/maven
        COPY policy-rc.d /usr/sbin/policy-rc.d
        RUN dpkg -i /tmp/docker-ce_20.10.9_3-0_ubuntu-bionic_amd64.deb
        COPY ./extensions/ /root/.azure/
        RUN dpkg -i /tmp/google-chrome-stable_current_amd64.deb
        RUN apt install -y --fix-broken
        COPY ./sw/zip/* /tmp/zip/
        WORKDIR /tmp/zip/
        RUN dpkg -i zip_3.0-11_amd64.deb unzip_6.0-21ubuntu1_amd64.deb
        WORKDIR /azp/_work
        COPY ./tools .
        RUN mkdir /root/.m2
        WORKDIR /root/.m2
        COPY ./settings.xml .
        WORKDIR /root
        COPY ./.npmrc .
        COPY ./jf .
        COPY ./config /root/.kube/
        COPY ./docker-compose /usr/local/bin
        COPY ./daemon.json /etc/docker/
        RUN sed -i 's/"$@"/"$@" --no-sandbox/' /opt/google/chrome/google-chrome
        ENV JFROG_CLI_HOME /root
        ENV TESTCONTAINERS_RYUK_DISABLED=true
        WORKDIR /azp
        CMD ["./docker.sh"]
        ENTRYPOINT ["./start.sh"]

```

So in the below sample, can we segregate repeating tasks to one layer so that the imagebuild step will be fast ?

* Also is it allowed to start the vsts-agent without root user? if yes, how I can achieve the same using normal user?

# Answer
> 1 votes
First build an image with all the dependencie that want change and are not related to your code.

```
FROM ubuntu:18.04
#2-Enable Ubuntu Packages
COPY ./sources.list /etc/apt/
#3- Install basic Softwares
ENV DEBIAN_FRONTEND=noninteractive
RUN echo "APT::Get::Assume-Yes \"true\";" > /etc/apt/apt.conf.d/90assumeyes
RUN apt-get update && apt-get install -y --no-install-recommends \
    ca-certificates \
    curl \
    wget \
    jq \
    git \
    iputils-ping \
    libcurl4 \
    libicu60 \
    libunwind8 \
    netcat \
    telnet \
    libssl1.0 \
    python \
    python3 \
    nodejs \
    python3-setuptools \
    python3-pip
    vim \
    openjdk-11-jdk-headless \
    gnupg \
    make \
    yarn\
    apt-transport-https \
    lsb-release \
  && rm -rf /var/lib/apt/lists/*
    #4. Copy Offline Packages
    COPY ./sw/* /tmp/
    #4-Install AzureCLI
    RUN  curl -sL https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor | tee /etc/apt/trusted.gpg.d/microsoft.gpg > /dev/null
    RUN AZ_REPO=$(lsb_release -cs) \
      && echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ $AZ_REPO main" | tee /etc/apt/sources.list.d/azure-cli.list \
      && apt-get update \
      &&  apt-get install azure-cli
    #Installl helm,docker, googlechrome and kubectl
    RUN  tar zxvf /tmp/helm-v3.8.2-linux-amd64.tar.gz  && mv /linux-amd64/helm /usr/local/bin/ && mv /tmp/kubectl /usr/local/bin/ \
         &&  apt install /tmp/google-chrome-stable_current_amd64.deb \
         &&  apt install /tmp/containerd.io_1.6.9-1_amd64.deb \
         &&  apt install /tmp/docker-ce-rootless-extras_20.10.9_3-0_ubuntu-bionic_amd64.deb \
         &&  apt install /tmp/libslirp0_4.6.1-1build1_amd64.deb      \
         &&  apt install /tmp/docker-ce_20.10.9_3-0_ubuntu-bionic_amd64.deb \
         &&  apt install /tmp/docker-compose-plugin_2.6.0_ubuntu-bionic_amd64.deb \
         &&  apt install /tmp/pigz_2.6-1_amd64.deb \
         &&  apt install /tmp/docker-ce-cli_20.10.9_3-0_ubuntu-bionic_amd64.deb \
         &&  apt install /tmp/docker-scan-plugin_0.9.0_ubuntu-bionic_amd64.deb \
         &&  apt install /tmp/slirp4netns_1.0.1-2_amd64.deb
    #8-install maven 3.8.6
    RUN mkdir -p /usr/share/maven /usr/share/maven/ref \
      && tar -xzf /tmp/apache-maven-3.8.6-bin.tar.gz -C /usr/share/maven --strip-components=1 \
      && echo "Cleaning and setting links" \
      && rm -f /tmp/apache-maven.tar.gz \
      && ln -s /usr/share/maven/bin/mvn /usr/bin/mvn
        # 8.1- Define environmental variables required by Maven, like Maven_Home directory and where the maven repo is located
        ENV MAVEN_HOME /usr/share/maven \
            TARGETARCH=linux-x64 \
            MAVEN_CONFIG "$USER_HOME_DIR/.m2" \
            JAVA_HOME /usr/lib/jvm/java-11-openjdk-amd64
        #11- Agent Installation

```

Build that image as your base image and push it in a dockerregistry.

```
docker build -t mybaseimage:latest .
docker image tag mybaseimage:latest registry-domain/mypath/mybaseimage:latest

docker image push registry-domain/mypath/mybaseimage:latest

```

Now your mybaseimage image is in the registry and you can use it and rebuild your image with your code from here.

```
FROM mybaseimage

       WORKDIR /azp
        COPY ./vstsagent/ .
        COPY ./start.sh .
        COPY ./docker.sh .
        RUN chmod +x start.sh docker.sh
        ENV TARGETARCH=linux-x64
        # Can be 'linux-x64', 'linux-arm64', 'linux-arm', 'rhel.6-x64'.
        ENV TARGETARCH=linux-x64
        ENV JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
        ENV JAVA_HOME_11_X64=/usr/lib/jvm/java-11-openjdk-amd64
        ENV MAVEN_HOME=/usr/share/maven
        COPY policy-rc.d /usr/sbin/policy-rc.d
        RUN dpkg -i /tmp/docker-ce_20.10.9_3-0_ubuntu-bionic_amd64.deb
        COPY ./extensions/ /root/.azure/
        RUN dpkg -i /tmp/google-chrome-stable_current_amd64.deb
        RUN apt install -y --fix-broken
        COPY ./sw/zip/* /tmp/zip/
        WORKDIR /tmp/zip/
        RUN dpkg -i zip_3.0-11_amd64.deb unzip_6.0-21ubuntu1_amd64.deb
        WORKDIR /azp/_work
        COPY ./tools .
        RUN mkdir /root/.m2
        WORKDIR /root/.m2
        COPY ./settings.xml .
        WORKDIR /root
        COPY ./.npmrc .
        COPY ./jf .
        COPY ./config /root/.kube/
        COPY ./docker-compose /usr/local/bin
        COPY ./daemon.json /etc/docker/
        RUN sed -i 's/"$@"/"$@" --no-sandbox/' /opt/google/chrome/google-chrome
        ENV JFROG_CLI_HOME /root
        ENV TESTCONTAINERS_RYUK_DISABLED=true
        WORKDIR /azp
        CMD ["./docker.sh"]
        ENTRYPOINT ["./start.sh"]

```

Note: registry-domain/mypath/ could be docker.io/myaccountname or your private registry if you have one.

---
Tags: docker, docker-compose, dockerfile, dockerhub
---


# Question
Title: What is the best way to reverse port forward information from a Kubernetes cluster to localhost?
I am attempting to reverse port forward information from a Kubernetes pod **192.168.49.2:30085** to my **locoalhost:8000** but I am having issues.

The goal is to get the same response as `curl 192.168.49.2:30085` on localhost:8000

One thing I have tried has been using ktunnel with the command `ktunnel expose <service_name> 30085:8000` to try to forward this info. Doing this I get the output

```
INFO[0000] Exposed service's cluster ip is: 10.101.77.99
.INFO[0000] waiting for deployment to be ready
.....
INFO[0001] port forwarding to https://192.168.49.2:8443/api/v1/namespaces/default/pods/thisisatest-59d5584c84-b5qrc/portforward
INFO[0001] Waiting for port forward to finish
INFO[0001] Forwarding from 127.0.0.1:28688 -> 28688
INFO[2022-08-03 20:23:48.605] starting tcp tunnel from source 30085 to target 8000

```

Which seems normal and I am able to get a response when using `curl http://10.101.77.99:30085` on local host 8000 but not the correct response.

I have also attempted to run a tcp server with the command `nc 127.0.0.1 8000 | kubectl exec -i <pod_name> tcpserver 127.0.0.1 30085 cat` But am having poor results as well. Am I using these services incorrectly? Or is there a better way to do this all together?

# Answer
If you have access to `kubectl`, try

`kubectl port-forward service/<servicename> 8000:30085`.
> 1 votes

# Answer
Expose SSH on the pod and use SSH to do the reverse port forward

or

Use a service on the cluster pointing at your machine but that assumes the networking is connected some how

Those 2 anwsers were taken from this duplicate question. https://stackoverflow.com/questions/66666273/forward-traffic-from-kubernetes-pod-to-local-server

I think a third better approach might be to use somekinda mesh VPN like https://tailscale.com/
> 0 votes

---
Tags: docker, kubernetes, ports
---


# Question
Title: Ansible Unsupported Parameters for using handler
I'm trying to run a ansible task but I keep getting an error.

This is the play I want to run:

```
- name: use of handlers
  hosts: all
  tasks:
    - name: install packages
      yum:
        name: httpd
        state: latest
        notify: restart package
  handlers:
    - name: restart package
      service:
        name: httpd
        state: restarted

```

But this is the error that ansible gives me:

> fatal: \[serverb.local\]: FAILED! =\> {"changed": false, "msg": "Unsupported parameters for (ansible.legacy.dnf) module: notify. Supported parameters include: allowerasing, allow\_downgrade, name (pkg), enablerepo, disable\_excludes, conf\_file, list, disable\_plugin, installroot, enable\_plugin, disable\_gpg\_check, download\_dir, update\_cache (expire-cache), releasever, state, nobest, bugfix, install\_weak\_deps, update\_only, autoremove, exclude, download\_only, cacheonly, security, lock\_timeout, skip\_broken, disablerepo, validate\_certs, install\_repoquery."}

# Answer
The indentation is wrong. Correct syntax is

```
     - name: install packages
       yum:
         name: httpd
         state: latest
       notify: restart package

```

The error says it:

```
"Unsupported parameters for (ansible.legacy.dnf) module: notify.

```

*notify* is not a parameter of the module *dnf* or *yum*. It is a *playbook keyword* that can be applied to a block or a task. In the above case, it must be indented as a task keyword. See Notifying handlers.
> 3 votes

---
Tags: ansible, linux
---


# Question
Title: When is it not appropriate to use serverless technology such as AWS Lambda?
With serverless computing using AWS lambda, Amazon(AWS) is taking the ownership of architecture, administration & deployment aspect of AWS services(VPC/EC2/ELB/AScaling/Rtables/SGroups/etc...) for the application to run.

With this approach, AWS user losing control on designing an optimal VPC architecture that fine tunes with application

So, when not to think about serverless computing?

# Answer
> 4 votes
There is likely no general answer to this question, but some reasons to not consider Lambda (or other serverless platforms) include:

* App requires a language which is not available (C++, Haskell, Erlang, etc.)
* Organization has existing substantial investment in containerized (or other) app development model, including developer knowledge, build and deployment pipelines
* Portability is concern; i.e. the organization wants/needs to keep app in a state which is portable across platforms (AWS, Azure, various k8s implementations, and so on).
* Cost. Do costs of running the Lambda function(s), as well as associated resources (API Gateway, VPC components) exceed budget? This is unlikely if the org. is already running in AWS, but can be a factor when compared against existing (and paid-off) on-premise resources.

# Answer
> 3 votes
I wrote a bit about this on my LinkedIn post which was based on the article by Andrea Passwater.

In brief, **do not go serverless, if**

1. You want complete control of the infrastructure and latest system patches
2. If your microservices are complex, and thus have some latency in starting up and computing output
3. You are not prepared for a heavy reliance on event-based architecture to monitor systems

# Answer
> 1 votes
If your system doesnt have events which you want to handle or have something getting triggered then don't use lambda. I won't say that language should be a concern here since you can always rewrite that part into python or golang code. Lambda is difficult to handle for complex systems. Unless you are already using a web framework like rails or django which have their inbuilt way of handling routes, you do want to make use of aws lambda.

---
Tags: amazon-web-services, cloud, aws-lambda
---


# Question
Title: proxy in a docker container implemented only with iptables - refuses connection
We need to implement a proxy only by using iptables in a container and all traffic from other containers use this container to reach internet. Additionally in this proxy, drop trafffic to yahoo.com domain and allow the rest.

I've a Oracle VM Box in windows that hosts a Ubuntu VM. In this VM, I've installed docker with 4 containers having Ubuntu image as shown here:

Requirement is any traffic from avant3, avant4 or avant2 should use avant1 as proxy to reach internet. avant1 should only implement iptables rules to become a tcp proxy.

In avant1 container in interactive mode, did the following as per this link:

```
iptables --table nat --append POSTROUTING --out-interface eth0 -j MASQUERADE
iptables --append FORWARD --in-interface eth1 -j ACCEPT

```

Going into avant3 and executing curl and specifying avant1 as proxy, getting this response:

```
root@340787f0b409:/# curl -x 172.19.0.3 "www.google.com" -I
curl: (7) Failed to connect to 172.19.0.3 port 1080 after 0 ms: Connection refused
root@340787f0b409:/# curl -x 172.19.0.3:80 "www.google.com" -I
curl: (7) Failed to connect to 172.19.0.3 port 80 after 0 ms: Connection refused
root@340787f0b409:/# 18.0

```

Can anybody guide me on the mistake I'm doing to achieve my requirement?

# Answer
Rather than a proxy solution, I thought of a different approach, avant1 will act as the gateway to all containers in localnet & testnet bridge networks. Either while creating the bridge networks you can specify a custom gateway as 172.18.0.2 & 172.19.0.3 for localnet and testnet respectively or use ip route command to replace the default gateways in individual containers.

In my case, added a FORWARD drop rule for domain yahoo.com in avant1. Entering into avant1 container interactively, did the following:

```
iptables --append FORWARD -d yahoo.com -j DROP

```

& launching avant3 container interactively, executed

```
ip route replace default via 172.19.0.3 dev eth0

```

then,

```
curl yahoo.com -I

```

will not allow the traffic anymore in avant3 while allowing rest to passthrough
> 0 votes

---
Tags: docker, api-gateway
---


# Question
Title: Local dev, online test/prod - best approach?
My teams develop on their laptops (using the wonderful K3D) with a local repo. But, the test and prod environments are in Azure. Locally we configure a docker image repository (using k3d), and in Azure we use ACR. My teams' internet connections are flakey at best, so we cannot all be online all the time.

Currently, I use an environment variable to manage this. In my k8s config files I use the env-var name template pattern:

```
image: $K8S_REPO_URL/dev/tenants

```

In my makefile I have this:

```
@cat k8s.yaml | envsubst | sed "$$CERT" | kubectl apply -f -

```

Which substitutes the current variable value into the k8s file before applying it to k8s. I use the same pattern in my azure clusters (with Tekton) where I configure the tekton pod with env-vars that are populated from a config map.

This means that I can use the local registry when working locally and I can use ACR when running in azure, and the only change is in the environment.

I think this is quite 'linuxy' and it works, though it does have its flaws (it's quite easy to make a frustrating but non-fatal mistake by not sourcing the correct environment before one begins development). And, the registry isn't the only difference (it's just the one I'm using here); we also have differences with certs (prod is all tls, dev is no tls), and ingress-class (nginx `locally`, `addon-http-application-routing` in azure), which has led to some more complex processing before applying my configs in places.

I'm worried that this is becoming too complicated. It's just about manageable right now, but if I add a couple more environment-based differences it'll start getting out of hand. Does anyone have a better suggestion that isn't "Use Helm"? I'm not opposed to helm at all, it's just that I already know that I should be using helm but it seems like such a heavyweight solution to simply swapping a couple of strings and deleting a line or two.

# Answer
> 0 votes
My suggestion is "use kustomize", and then create a development overlay that overrides the images. Using the image transformer, you would end up with something like:

```
# This assumes that your deployment manifests are all locaed in the
# directory ../production. What we're doing here is including all
# of those manifests and then applying some transformations.
resources:
  - ../production/

images:
  - name: appimage
    newName: local_repo_url/appimage

```

This would replace all references to an image named `appimage` (in Pods, Deployments, Statefulsets, Daemonsets, Jobs, etc) with `local_repo_url/appimage`. You can also transform the tag, etc.

I really like working with kustomize (I use it both for work and personal projects), but it does require to be disciplined with respect to how you lay out your manifests.

---
Tags: kubernetes, environments
---


# Question
Title: How to programmatically generate docker-compose file?
I am writing an Ansible playbook to deploy services to a clustered environment. There are 3 types of hosts (Application, Load balancer, database). Services are deployed via docker-compose on each host separately.

Example of playbook to deploy to a database host.

```
- name:           Deploy database node.
  hosts:          databases
  become:         yes
  become_method:  sudo

  roles:
  - common
  - zabbix-agent
  - database
  - backup

```

The Ansible play needs to dynamically generate a docker-compose file based on which roles have been invoked on that host. So this example would need to generate a docker-compose which lists these roles.

**How do i programmatically generate a docker-compose.yml file, based on which roles have been invoked?**

I have considered:

* Ansible facts
* Tags
* global boolean variables \[yuck\] but none of these seems like the right way to me.

If there is a better approach to achieving my objective I'm open to any other suggestions too.

**Edit:**

> Can you explain what is in the container here?

'common' role currently installs Docker and docker-compose onto the target host. 'database' starts a mariadb container. 'zabbix-agent' installs..well, zabbix agent on the host. 'backup' copies some shell scripts over.

> Is the database in the container, or is the container running next to the database?

the database is a mariadb Docker container. So my compose file will need to include a 'mariadb' section.

> What would a good docker-compose.yml look like for you?

Good question. Something like this:

```
version:                            '2.2'
services:

  terracotta:
    image:                          terracotta/terracotta-server-oss:5.5.1
    container_name:                 terracotta
    restart:                        always
    networks:
      - skyNet
    logging:
      driver:                       "json-file"
      options:
        max-size:                   "10m"
        max-file:                   "5"

  mariadb:
    image:                          mariadb/server:{{docker_mariadb_version}}
    container_name:                 mariadb
    restart:                        always
    logging:
      driver:                       "json-file"
      options:
        max-size:                   "10m"
        max-file:                   "5"
    networks:
      - skyNet
    ports:
    - 127.0.0.1:3306:3306/tcp
    volumes:
      - /opt/mariadb/data:/var/lib/mysql
    environment:
      MARIADB_ROOT_PASSWORD:        "{{database_root_password}}"

# Network
networks:
  skyNet:
    driver:                         bridge

```

I need to be able to need to be able to toggle which services are added to the compose file via Tags or something. Maybe I should change the phrasing of my question?

# Answer
> 0 votes
> How do i programmatically generate a docker-compose.yml file, based on which roles have been invoked?

Reading your question, I don't think you need to do that. You're already deploying things on an explicit network; just make this external to the individual compose files. That is, perform the equivalent of:

```
docker network create skyNet

```

And then modify your compose configuration to reference that as an `external` network:

```
services:

  mariadb:
    image:                          mariadb/server:{{docker_mariadb_version}}
    container_name:                 mariadb
    restart:                        always
    logging:
      driver:                       "json-file"
      options:
        max-size:                   "10m"
        max-file:                   "5"
    networks:
      - skyNet
    ports:
    - 127.0.0.1:3306:3306/tcp
    volumes:
      - /opt/mariadb/data:/var/lib/mysql
    environment:
      MARIADB_ROOT_PASSWORD:        "{{database_root_password}}"

networks:
  skyNet:
    external: true

```

You can have each role manage it's own `docker-compose.yaml` file (and be responsible for bring up the stack). Because you're sharing a network between them, name lookups will work as expected between services, even though they're defined in different files.

---
Tags: ansible, docker-compose, jinja
---


# Question
Title: How to get a list of deployments that only have a certain label in the spec section
I know that I can perform this command to see all the pods that have a certain label : `kubectl get pods -l importance=normal`

Is there a command that would allow me to get the deployments that create the pods with the label `importance=normal`?

I want to find all deployments what have the label `importance=normal` found in the spec/template/metadata/labels (see my deployment yaml below).

I found this link(https://kubernetes.io/docs/tasks/access-application-cluster/list-all-running-container-images/#list-container-images-filtering-by-pod-label) and started to create the following command but I could not get it to work -

kubectl get deployments -namespace prod -o jsonpath='{range .items[*]}{"\n"}{.spec.template.metadata.labels}{":\t"}{", "}{end}{end}' |\ sort```
</code></pre>

My deployment yaml looks like the following:

```
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    app.kubernetes.io/managed-by: Helm
    deployment.kubernetes.io/revision: "11"
    meta.helm.sh/release-name: service-qa
    meta.helm.sh/release-namespace: prod
    source: https://gitlab.services.com/service
  creationTimestamp: "2019-04-07T12:51:03Z"
  generation: 22
  labels:
    app.kubernetes.io/managed-by: Helm
    chart: service-0.1.0
  name: service-deployment
  namespace: qa
  resourceVersion: "577488289"
  uid: cd2fdeb4-5933-11e9-ad6d-02211a607320
spec:
  progressDeadlineSeconds: 600
  replicas: 2
  revisionHistoryLimit: 3
  selector:
    matchLabels:
      app: service-selector
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
    type: RollingUpdate
  template:
    metadata:
      annotations:
        kubectl.kubernetes.io/restartedAt: "2022-11-13T08:19:36-06:00"
      creationTimestamp: null
      labels:
        app: service-selector
        importance: normal
        version: current

```

# Answer
The easiest solution is probably using `jq`. To get a list of Deployments that deploy Pods with the label `importance: normal`, you could do this:

```
kubectl get deploy -o json |
jq '.items[]|select(.spec.template.metadata.labels.importance == "normal")'

```

I would argue that you should rewrite your manifests so that the Deployment shares labels with the Pod so that you can just run:

```
kubectl get deploy -l importance=normal

```

That's much simpler. Using a tool like kustomize can simplify the work involved in keeping these labels consistent.
> 3 votes

---
Tags: kubernetes, kubectl
---