# Question
Title: Failed to create pod sandbox: rpc error: code = DeadlineExceeded desc = failed to get sandbox image "registry.k8s.io/pause:3.6
Today I found some events in kubernetes(`v1.28.3`) cluster, this error event look like this:

```
(combined from similar events): Failed to create pod sandbox: rpc error: code = DeadlineExceeded desc = failed to get sandbox image "registry.k8s.io/pause:3.6": failed to pull image "registry.k8s.io/pause:3.6": failed to pull and unpack image "registry.k8s.io/pause:3.6": failed to resolve reference "registry.k8s.io/pause:3.6": failed to do request: Head "https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/pause/manifests/3.6": dial tcp 74.125.204.82:443: i/o timeout

```

I know it may be a network issue when download the `registry.k8s.io/pause:3.6` image, but I could not found the image `registry.k8s.io/pause:3.6` config in the deamon set, this is the yaml define:

```
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: calico-node
  namespace: calico-system
  annotations:
    deprecated.daemonset.template.generation: '1'
status:
  currentNumberScheduled: 2
  numberMisscheduled: 0
  desiredNumberScheduled: 2
  numberReady: 1
  observedGeneration: 1
  updatedNumberScheduled: 2
  numberAvailable: 1
  numberUnavailable: 1
spec:
  selector:
    matchLabels:
      k8s-app: calico-node
  template:
    metadata:
      creationTimestamp: null
      labels:
        app.kubernetes.io/name: calico-node
        k8s-app: calico-node
      annotations:
        hash.operator.tigera.io/cni-config: 9f0a12e03c58671de56ed3876cb88f1c43cef5dc
        hash.operator.tigera.io/system: bb4746872201725da2dea19756c475aa67d9c1e9
        hash.operator.tigera.io/tigera-ca-private: 8766b9b7cef0ba4722da2eb519044c7c3b68df33
    spec:
      volumes:
        - name: lib-modules
          hostPath:
            path: /lib/modules
            type: ''
        - name: xtables-lock
          hostPath:
            path: /run/xtables.lock
            type: FileOrCreate
        - name: policysync
          hostPath:
            path: /var/run/nodeagent
            type: DirectoryOrCreate
        - name: tigera-ca-bundle
          configMap:
            name: tigera-ca-bundle
            defaultMode: 420
        - name: node-certs
          secret:
            secretName: node-certs
            defaultMode: 420
        - name: var-run-calico
          hostPath:
            path: /var/run/calico
            type: ''
        - name: var-lib-calico
          hostPath:
            path: /var/lib/calico
            type: ''
        - name: cni-bin-dir
          hostPath:
            path: /opt/cni/bin
            type: ''
        - name: cni-net-dir
          hostPath:
            path: /etc/cni/net.d
            type: ''
        - name: cni-log-dir
          hostPath:
            path: /var/log/calico/cni
            type: ''
        - name: flexvol-driver-host
          hostPath:
            path: /usr/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds
            type: DirectoryOrCreate
      initContainers:
        - name: flexvol-driver
          image: docker.io/calico/pod2daemon-flexvol:v3.26.1
          resources: {}
          volumeMounts:
            - name: flexvol-driver-host
              mountPath: /host/driver
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: IfNotPresent
          securityContext:
            capabilities:
              drop:
                - ALL
            privileged: true
            runAsUser: 0
            runAsGroup: 0
            runAsNonRoot: false
            allowPrivilegeEscalation: true
            seccompProfile:
              type: RuntimeDefault
        - name: install-cni
          image: docker.io/calico/cni:v3.26.1
          command:
            - /opt/cni/bin/install
          env:
            - name: CNI_CONF_NAME
              value: 10-calico.conflist
            - name: SLEEP
              value: 'false'
            - name: CNI_NET_DIR
              value: /etc/cni/net.d
            - name: CNI_NETWORK_CONFIG
              valueFrom:
                configMapKeyRef:
                  name: cni-config
                  key: config
            - name: KUBERNETES_SERVICE_HOST
              value: 10.96.0.1
            - name: KUBERNETES_SERVICE_PORT
              value: '443'
          resources: {}
          volumeMounts:
            - name: cni-bin-dir
              mountPath: /host/opt/cni/bin
            - name: cni-net-dir
              mountPath: /host/etc/cni/net.d
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: IfNotPresent
          securityContext:
            capabilities:
              drop:
                - ALL
            privileged: true
            runAsUser: 0
            runAsGroup: 0
            runAsNonRoot: false
            allowPrivilegeEscalation: true
            seccompProfile:
              type: RuntimeDefault
      containers:
        - name: calico-node
          image: docker.io/calico/node:v3.26.1
          env:
            - name: DATASTORE_TYPE
              value: kubernetes
            - name: WAIT_FOR_DATASTORE
              value: 'true'
            - name: CLUSTER_TYPE
              value: k8s,operator,bgp
            - name: CALICO_DISABLE_FILE_LOGGING
              value: 'false'
            - name: FELIX_DEFAULTENDPOINTTOHOSTACTION
              value: ACCEPT
            - name: FELIX_HEALTHENABLED
              value: 'true'
            - name: FELIX_HEALTHPORT
              value: '9099'
            - name: NODENAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: spec.nodeName
            - name: NAMESPACE
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.namespace
            - name: FELIX_TYPHAK8SNAMESPACE
              value: calico-system
            - name: FELIX_TYPHAK8SSERVICENAME
              value: calico-typha
            - name: FELIX_TYPHACAFILE
              value: /etc/pki/tls/certs/tigera-ca-bundle.crt
            - name: FELIX_TYPHACERTFILE
              value: /node-certs/tls.crt
            - name: FELIX_TYPHAKEYFILE
              value: /node-certs/tls.key
            - name: FIPS_MODE_ENABLED
              value: 'false'
            - name: FELIX_TYPHACN
              value: typha-server
            - name: CALICO_MANAGE_CNI
              value: 'true'
            - name: CALICO_IPV4POOL_CIDR
              value: 10.96.0.0/12
            - name: CALICO_IPV4POOL_VXLAN
              value: CrossSubnet
            - name: CALICO_IPV4POOL_BLOCK_SIZE
              value: '26'
            - name: CALICO_IPV4POOL_NODE_SELECTOR
              value: all()
            - name: CALICO_IPV4POOL_DISABLE_BGP_EXPORT
              value: 'false'
            - name: CALICO_NETWORKING_BACKEND
              value: bird
            - name: IP
              value: autodetect
            - name: IP_AUTODETECTION_METHOD
              value: first-found
            - name: IP6
              value: none
            - name: FELIX_IPV6SUPPORT
              value: 'false'
            - name: KUBERNETES_SERVICE_HOST
              value: 10.96.0.1
            - name: KUBERNETES_SERVICE_PORT
              value: '443'
          resources: {}
          volumeMounts:
            - name: tigera-ca-bundle
              readOnly: true
              mountPath: /etc/pki/tls/certs
            - name: tigera-ca-bundle
              readOnly: true
              mountPath: /etc/pki/tls/cert.pem
              subPath: ca-bundle.crt
            - name: lib-modules
              readOnly: true
              mountPath: /lib/modules
            - name: xtables-lock
              mountPath: /run/xtables.lock
            - name: policysync
              mountPath: /var/run/nodeagent
            - name: node-certs
              readOnly: true
              mountPath: /node-certs
            - name: var-run-calico
              mountPath: /var/run/calico
            - name: var-lib-calico
              mountPath: /var/lib/calico
            - name: cni-log-dir
              mountPath: /var/log/calico/cni
            - name: cni-net-dir
              mountPath: /host/etc/cni/net.d
          livenessProbe:
            httpGet:
              path: /liveness
              port: 9099
              host: localhost
              scheme: HTTP
            timeoutSeconds: 10
            periodSeconds: 10
            successThreshold: 1
            failureThreshold: 3
          readinessProbe:
            exec:
              command:
                - /bin/calico-node
                - '-bird-ready'
                - '-felix-ready'
            timeoutSeconds: 5
            periodSeconds: 10
            successThreshold: 1
            failureThreshold: 3
          lifecycle:
            preStop:
              exec:
                command:
                  - /bin/calico-node
                  - '-shutdown'
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: IfNotPresent
          securityContext:
            capabilities:
              drop:
                - ALL
            privileged: true
            runAsUser: 0
            runAsGroup: 0
            runAsNonRoot: false
            allowPrivilegeEscalation: true
            seccompProfile:
              type: RuntimeDefault
      restartPolicy: Always
      terminationGracePeriodSeconds: 5
      dnsPolicy: ClusterFirst
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: calico-node
      serviceAccount: calico-node
      hostNetwork: true
      securityContext: {}
      schedulerName: default-scheduler
      tolerations:
        - key: CriticalAddonsOnly
          operator: Exists
        - operator: Exists
          effect: NoSchedule
        - operator: Exists
          effect: NoExecute
      priorityClassName: system-node-critical
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
      maxSurge: 0
  revisionHistoryLimit: 10

```

where is the pause image configuration? what should I do to find out? I have tried to change the kubelet config and add mirror address like this:

```
[root@k8sslave01 kubelet]# systemctl status kubelet --full
● kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: active (running) since Mon 2024-02-19 12:26:30 CST; 33min ago
     Docs: https://kubernetes.io/docs/
 Main PID: 30876 (kubelet)
    Tasks: 11
   Memory: 84.6M
   CGroup: /system.slice/kubelet.service
           └─30876 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime-endpoint=unix:///var/run/containerd/containerd.sock --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.6

```

the `registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.6` is the new address. It seems did not work. I have already test the container is accessable:

```
[root@k8sslave01 kubelet]# crictl pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.6
Image is up to date for sha256:6270bb605e12e581514ada5fd5b3216f727db55dc87d5889c790e4c760683fee

```

# Answer
I also catch this issue, steps as bellow can fix it.

1. docker pull docker/desktop-kubernetes-pause:3.8
2. docker images you'll see result like bellow:

> docker/desktop-kubernetes-pause 3.8 4873874c08ef 19 months ago 711kB

// tag pause to registry.k8s.io/pause:3.8
3. docker tag 4873874c08ef registry.k8s.io/pause:3.8

// import pause to containerd
4. ctr -n k8s.io images import pause.tar
5. sudo ctr -n k8s.io images import pause.tar

Then restart kubelet, after kubelet daemon is active and healthy, you can reset k8s cluster, and reinstall k8s again.
> 1 votes

# Answer
I found another way to solve this issue by change the containerd sandbox image:

```
version = 2
[plugins."io.containerd.grpc.v1.cri"]
  #sandbox_image = "k8s.m.daocloud.io/pause:3.6"
  sandbox_image = "registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.6"

```

replace the `sandbox_image` to any mirror address you can access.
> 0 votes

---
Tags: kubernetes
---


# Question
Title: How to configurate keycloak healthcheck in docker compose?
I need my BFF service to wait for keycloak to be fully launched using condition `service_health`, and then start its launch. I can't set up a health check for `keycloak`. When searching for an answer, I come across either strange or outdated answers related to `curl`, which is no longer in the new version of `keycloak`. How do I set up the `healthcheck` keycloak correctly?

```
services:
  gateway:
    image: 'my-service-bff:0.0.1-SNAPSHOT'
    container_name: 'my-service-bff'
    ports:
      - '8080:8080'
    depends_on:
      keycloak:
        condition: service_healthy
  postgres:
    image: 'postgres:latest'
    container_name: 'postgres-db'
    volumes:
      - postgres_data:/var/lib/postgresql/data
    environment:
      POSTGRES_DB: keycloakdb
      POSTGRES_USER: keycloakuser
      POSTGRES_PASSWORD: keycloakpass
    healthcheck:
      test: pg_isready -U keycloakuser -d keycloakdb
      interval: 10s
      timeout: 3s
      retries: 3
      start_period: 5s
    networks:
      - keycloak-network
  keycloak:
    container_name: keycloak
    image: quay.io/keycloak/keycloak:23.0.4
    environment:
      KC_HEALTH_ENABLED: true
      DB_VENDOR: POSTGRES
      DB_ADDR: postgres
      DB_DATABASE: keycloakdb
      DB_SCHEMA: public
      KEYCLOAK_USER: user
      KEYCLOAK_PASSWORD: user
      KEYCLOAK_ADMIN: admin
      KEYCLOAK_ADMIN_PASSWORD: admin
    ports:
      - "9090:8080"
    command:
      - "start-dev"
    networks:
      - 'keycloak-network'
    depends_on:
      postgres:
        condition: service_healthy
    healthcheck:
      # Unsuccessful attempts:
      # test: [ "CMD", "curl", "-f", "http://0.0.0.0:8080/realms/master" ]
      # test: ['CMD-SHELL', '[ -f /tmp/HealthCheck.java ] || echo "public class HealthCheck { public static void main(String[] args) throws java.lang.Throwable { System.exit(java.net.HttpURLConnection.HTTP_OK == ((java.net.HttpURLConnection)new java.net.URI.create(args[0]).toURL().openConnection()).getResponseCode() ? 0 : 1); } }" > /tmp/HealthCheck.java && java /tmp/HealthCheck.java http://localhost:8080/health/live']
      start_period: 10s
      interval: 30s
      retries: 3
      timeout: 5s

networks:
  keycloak-network:
    driver: 'bridge'

volumes:
  postgres_data:
    driver: 'local'

```

# Answer
> 2 votes
You could just add Curl by using the KeyCloak image in a Dockerfile and reference that Dockerfile in your `docker-compose.yaml` file.

To do that I create a directory inside the `docker-compose.yaml` directory and call it something simple like `KeyCloakDockerfile` and in that create your \`Dockerfile so structure should look like this:

```
_
 |_ docker-compose.yaml
 |_ KeyCloakDockerfile/
   |_ Dockerfile

```

You would create the Dockerfile like so *(Assuming it's RHEL or some RedHat derivative)*:

```

###### Using As Base Image ######
FROM quay.io/keycloak/keycloak:23.0.4

###### Non-Interactive Installations #########
ENV DAEMON_OPTS=-nonat
ENV TERM=xterm-256color

######### Update & Upgrade the Image ##########
RUN yum update -y && yum upgrade -y

###### Install Curl ######
RUN yum install -y curl

###### Use The ENTRYPOINT From The Original Images Dockerfile. ######
###### The Original Dockerfile https://quay.io/repository/keycloak/keycloak/manifest/sha256:122ff5badfb88afe1027bc8e1a9c85532148ee813bee30c01173947a445d4fe7 ######

ENTRYPOINT ["/opt/keycloak/bin/kc.sh"]

```

Then in your `docker-compose.yaml` you can change you KeyCloak service to look like this:

```
  keycloak:
    container_name: keycloak
    build:
      context: .
      dockerfile: ./KeyCloakDockerfile/Dockerfile
    environment:
      KC_HEALTH_ENABLED: true
      DB_VENDOR: POSTGRES
      DB_ADDR: postgres
      DB_DATABASE: keycloakdb
      DB_SCHEMA: public
      KEYCLOAK_USER: user
      KEYCLOAK_PASSWORD: user
      KEYCLOAK_ADMIN: admin
      KEYCLOAK_ADMIN_PASSWORD: admin
    ports:
      - "9090:8080"
    command:
      - "start-dev"
    networks:
      - 'keycloak-network'
    depends_on:
      postgres:
        condition: service_healthy
    healthcheck:
      # Unsuccessful attempts:
      test: [ "CMD", "curl", "-f", "http://0.0.0.0:8080/realms/master" ]
      start_period: 10s
      interval: 30s
      retries: 3
      timeout: 5s

```

This way when you run `docker compose up -d` if the image has not been built it will build it then run it with the other Services defined in the `docker-compose.yaml`.

**SECURITY SIDENOTE !**

Looking the the repository that Image of KeyCloak we're using has some vulerabilities present as is labeled here:

https://quay.io/repository/keycloak/keycloak/manifest/sha256:122ff5badfb88afe1027bc8e1a9c85532148ee813bee30c01173947a445d4fe7?tab=vulnerabilities

It states that version **3.90.0-4.el9\_3** should be upgraded to version **0:3.90.0-6.el9\_3**. This should be paid attentions to when using KeyCloak as good practise.

---
Tags: docker, docker-compose, keycloak, healthcheck
---


# Question
Title: Setting ingress port with helm parameter
I am runnning portainer on a k3s cluster, and installing/updating portainer using the command

```
helm upgrade --install --create-namespace -n portainer portainer portainer/portainer\
     --set enterpriseEdition.enabled=true\
     --set tls.force=true\
     --set tls.existingSecret=portainer-tls-secret

```

I would like to be able to access the portainer install on

https://portainer.k8s.mydomain.com/#!/1/kubernetes/dashboard
but I have to use
https://portainer.k8s.mydomain.com:30779/#!/1/kubernetes/dashboard

I assume that the issue is a setting in the helm chart, but I would prefer not to have to make a custom chart.

I assume that I can add a parameter. I am hestiant, because as I see it,  `--set tls.force=true` is translated to `--http-disabled`

Output of `kubectl describe deploy portainer --namespace portainer` :

> ```
> Name:               portainer
> Namespace:          portainer
> CreationTimestamp:  Sat, 23 Sep 2023 19:16:03 +0000
> Labels:             app.kubernetes.io/instance=portainer
>                     app.kubernetes.io/managed-by=Helm
>                     app.kubernetes.io/name=portainer
>                     app.kubernetes.io/version=ce-latest-ee-2.19.4
>                     helm.sh/chart=portainer-1.0.49
>                     io.portainer.kubernetes.application.stack=portainer
> Annotations:        deployment.kubernetes.io/revision: 9
>                     meta.helm.sh/release-name: portainer
>                     meta.helm.sh/release-namespace: portainer
> Selector:           app.kubernetes.io/instance=portainer,app.kubernetes.io/name=portainer
> Replicas:           1 desired | 1 updated | 1 total | 1 available | 0 unavailable
> StrategyType:       Recreate
> MinReadySeconds:    0
> Pod Template:
>   Labels:           app.kubernetes.io/instance=portainer
>                     app.kubernetes.io/name=portainer
>   Annotations:      kubectl.kubernetes.io/restartedAt: 2024-02-20T19:51:36Z
>   Service Account:  portainer-sa-clusteradmin
>   Containers:
>    portainer:
>     Image:       portainer/portainer-ee:2.19.4
>     Ports:       9443/TCP, 8000/TCP
>     Host Ports:  0/TCP, 0/TCP
>     Args:
>       --http-disabled
>       --sslcert=/certs/tls.crt
>       --sslkey=/certs/tls.key
>       --tunnel-port=30776
>     Liveness:     http-get https://:9443/ delay=45s timeout=1s period=10s #success=1 #failure=3
>     Readiness:    http-get https://:9443/ delay=45s timeout=1s period=10s #success=1 #failure=3
>     Environment:  <none>
>     Mounts:
>       /certs from certs (ro)
>       /data from data (rw)
>   Volumes:
>    data:
>     Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
>     ClaimName:  portainer
>     ReadOnly:   false
>    certs:
>     Type:        Secret (a volume populated by a Secret)
>     SecretName:  portainer-tls-secret
>     Optional:    false
> Conditions:
>   Type           Status  Reason
>   ----           ------  ------
>   Available      True    MinimumReplicasAvailable
>   Progressing    True    NewReplicaSetAvailable
>
> ```

# Answer
> 1 votes
By taking a look at portainer's documentation https://portainer.github.io/k8s/charts/portainer/, you can find values to set for configuring port, such as

service.httpPort

ingress.hosts.paths.port

---
Tags: kubernetes, helm
---


# Question
Title: How to design AWS infrastructure for my online course selling website?
I am designing an AWS architecture for hosting a website that sells online courses and merchandise products. For the purpose, I am planning to use the following architecture design:

* 1 VPC
    + 1 Application (RDS database + PHP web application)
        - Under Application, there can be 1 EC2 instance, 1 RDS database
        - S3 bucket
            * To store all static files, images, stylesheets, javascript files etc.
            * To upload all dynamic files, images, etc. (to be uploaded by users & website admin)
            * A CDN domain (e.g. cdn.mydomain.com) will be created to serve all files from S3 bucket.
    + 1 CloudFront (for content hosting & delivery)
    + AWS Router 53 (for domain & subdomain)

As per my knowledge and understanding, CloudFront should be used to deliver the content from S3 bucket that's why I am thinking of this.

**Question #1**: Should I create a VPC and encapsulate all AWS services/resources under it like S3, Application, RDS Database?

**Question #2**: Should CloudFront be used for the purpose?

**Question #3**: Is there any better approach for achieving this?

**Note**: *Please note that I am trying to utilize the resources under Free Tier as much as possible to avoid cost as this is an initial phase of the website and I will be selling my courses through this as of now. And, there will be videos for each session/lecture where each video can be of 2GB or more in size and for 1 course there can be 40-50 videos that students can access only after making a purchase.* *Also, I would love to know if I should add anything that I missed.*

Happy to hear experts' advise/suggestions on this.

# Answer
**Question #1: Should I create a VPC and encapsulate all AWS services/resources under it like S3, Application, RDS Database?**

If you use Elastic Beanstalk, you shouldn't have to worry too much about this, since it is a PaaS service (platform-as-a-service) that would take care of it for you. Be careful about the free tier limits and platform locking.

**Question #2: Should CloudFront be used for the purpose?**

You don't need CloudFront to serve your S3 content. It will be too pricey for what you want to achieve. Every object in your S3 bucket has a unique URL that you can use directly. Please read : https://stackoverflow.com/questions/3327425/when-to-use-amazon-cloudfront-or-s3

* S3 are not VPC bound by default. You can decide to do it for security reasons but it's optional.

**Question #3: Is there any better approach for achieving this?**

There are many different approaches available. It depends on your requirements : number of users, proximity of users, High Availability requirements, size of the content served, etc. You can read about System Design to get more information about this.
> 1 votes

---
Tags: amazon-web-services, amazon-ec2, amazon-s3, architecture, amazon-cloudfront
---


title: Docker gets stuck when running 16 tests

text:
# Question
Title: Docker gets stuck when running 16 tests
I'm running my dotnet 7.0 tests using "testcontainers" on windows machine. The tests are parallelized integration tests which use SQL Server inside a Linux docker container. Each test creates its own database.

When I run less than 16 tests at the same time, they all pass. But when I run more than that (and my IDE limits it to 16) it gets stuck and nothing will happen. If I wait for a day it's still trying to run the first 16 tests.

Same thing happens on VS and Rider.

There is nothing interesting in docker logs and I don't know where else to look.

When I open the task manager I can see that docker keeps restarting some its services, as it's constantly alternating between these two states:

## Update:

I found a solution by limiting the parallel tests in my infrastructure by running batches of 10. But I'm still wondering why this happens in the first place. Is it testcontainers bug or docker's? Anyone else has a similar experience?

# Answer
It might be a memory issue, I found using Docker in Windows with WSL ate RAM to the point of crashing and I had to restart the Docker daemon, I want only creating 6 containers, and I have 16GB of RAM!

If your using Docker Compose you can just run `docker compose up` and it will launch all the containers and you will get the output of each program that starts running in each container. You can then output it to a logfile for further analysis.

Or if your just using Dockerfile for this I would assume it's either RAM or a DB concurrency issue or maybe if you log into the DB with the same user it has a connection limit set. If you could share your docker-compose.yaml or Dockerfile that could help us fix the issue.
> 0 votes

---
Tags: docker, windows, testing, ubuntu, dotnet
---


# Question
Title: Nexus2: How can I get a list of all artifacts from a specific repository via REST API?
I want to get a list (json) of all artifacts in one specific repository via the REST API.

I've already tried:

```
curl -X GET http://localhost/nexus/service/local/respositories/<reponame>/content/

```

This works, but it deliveres me a json listing all my subdirectories in this repo. But what I want is a list with all the artifacts contained in all subdirectories.

Someone knows how to do this? Is there a way to iterate through all folders? I was thinking about the search keyword, but I wouldn't know for what to search.

# Answer
Try this:

```
# Get status
curl http://localhost:8081/nexus/service/local/status
# List of repositories
curl http://localhost:8081/nexus/service/local/repositories

```

For the json parameter:

```
curl -i -H "Accept: application/json" <url>

```

More information, see this link
> 2 votes

# Answer
For nexu3 this how to list artifact using curl:

curl -X GET "http://localhost:8081/nexus/service/rest/v1/search?repository={repository\_name}&format=maven2&maven.groupId={groupeID}&maven.artifactId={ArtifactID}&maven.extension={package\_type}" -H "accept: application/json"
> 0 votes

---
Tags: api, nexus, n3dr
---


# Question
Title: docker-compose invalid type, it should be a string
I have the following environment variable configured in a docker-compose.yml file:

```
version: '3'
services:
  server:
    ports:
     - 13045:3000
    environment:
     - NODE_CONFIG: '{"DATABASE_URL":"http://db:5984"}'

```

When trying to run `docker-compose up`, I'm getting this error:

```
services.server.environment contains {"NODE_CONFIG": "{\"DATABASE_URL\":\"http://db:5984\"}"}, which is an invalid type, it should be a string

```

I need the environment variable to be set to a JSON string (see https://github.com/lorenwest/node-config/wiki/Environment-Variables#node\_config)

Am I doing something wrong here? Can I get this to work somehow?

# Answer
The Docker Compose file reference states, that environment variables are defined as `VARIABLE=value` array elements. For your case, the docker-compose.yml file would need to be changed to this:

```
version: '3'
services:
  server:
    ports:
     - 13045:3000
    environment:
     - NODE_CONFIG='{"DATABASE_URL":"http://db:5984"}'

```
> 17 votes

# Answer
You need to remove dash in front of variable. Use syntax like that:

```
   environment:
     NODE_CONFIG: '{"DATABASE_URL":"http://db:5984"}'

```
> 15 votes

# Answer
The args elements themselves need to be a string

```
  args:
    - 'MAVEN_USER=$MAVEN_USER'
    - 'MAVEN_PASSWORD=$MAVEN_PASSWORD'

```
> 8 votes

# Answer
```
version: '3'
services:
  server:
    ports:
     - 13045:3000
    environment:
     NODE_CONFIG: '{"DATABASE_URL":"http://db:5984"}'

```

Just take the dash before NODE\_CONFIG out. For some weird reason this dash looks to not be accepted under environment.

The following example is from official postgres docker hub https://hub.docker.com/\_/postgres

```
# Use postgres/example user/password credentials
version: '3.1'

services:

  db:
    image: postgres
    restart: always
    environment:
      POSTGRES_PASSWORD: example

  adminer:
    image: adminer
    restart: always
    ports:
      - 8080:8080

```

**QQ edit:**

Basically, dashes are lists and no dashes are mappings, so that is why environment variable must not have dashes

References: https://stackoverflow.com/questions/64275361/docker-compose-yml-dash-syntax-in-yaml https://docs.ansible.com/ansible/latest/reference\_appendices/YAMLSyntax.html
> 2 votes

# Answer
You have to use = sign instead of :

example:

```
     - NODE_CONFIG= '{"DATABASE_URL":"http://db:5984"}'

```
> 0 votes

---
Tags: docker, docker-compose
---


# Question
Title: Ansible - 'loop' requires a list and got a list, but doesn't work
Minimal reproducible example:

`role/defaults/main.yaml`:

```
node_sysctl:
  kernel.shmall: 197951838
  kernel.shmmax: 810810728448
  kernel.shmmni: 4096

```

`role/tasks/main.yaml`:

```
- name: Tune sysctl
  sysctl:
    name: "{{ item.key }}"
    value: "{{ item.value }}"
    reload: true
  loop: "{{ lookup('dict', node_sysctl, wantlist=True) }}"
  become: true

```

I'm trying to run this task against an Ubuntu 18.04 host. When I do, I get the following ansible error:

```
TASK [../../../../ansible/roles/test-role : Tune sysctl] *********************************************************************************************************
fatal: [coordinator]: FAILED! => {"msg": "Invalid data passed to 'loop', it requires a list, got this instead:
 [{'key': 'kernel.shmall', 'value': 197951838}, {'key': 'kernel.shmmax', 'value': 810810728448}, {'key': 'kernel.shmmni', 'value': 4096}, {'key': 'vm.overcommit_memory', 'value': 2}, {'key': 'vm.overcommit_ratio', 'value': 95}, {'key': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'}, {'key': 'kernel.sem', 'value': '250 2048000 200 8192'}, {'key': 'kernel.sysrq', 'value': 1}]. Hint: If you passed a list/dict of just one element, try adding wantlist=True to your lookup invocation or use q/query instead of lookup."}

```

`loop` is clearly receiving a list here, and I don't understand why it won't work.

Running with python 3.9 on the local machine, and ansible:

```
ansible --version

  import pkg_resources
ansible 2.7.10
  config file = None
  configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/user/.local/share/virtualenvs/linux-p5PCkeXK/lib/python3.9/site-packages/ansible
  executable location = /home/user/.local/share/virtualenvs/linux-p5PCkeXK/bin/ansible
  python version = 3.9.18 (main, Nov 28 2023, 14:57:43) [GCC 13.2.1 20231011 (Red Hat 13.2.1-4)]

```

I also tried using `with_items` instead of `loop`, which also produces an error:

```
FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'key'

```

# Answer
> 0 votes
It appears that the root cause was some internal incompatibility between the ancient ansible I'm forced to use and python 3.9.

Simply switching the virtual environment on my machine to use **python 3.7** resolved the issue and now the task runs successfully.

---
Tags: ansible, linux
---


# Question
Title: Ensure two node ports have the same IP address
I'm working on setting up a testing environment for my application. Without getting into all of the details, one requirement is that I stand up a container that is listening on two TCP ports. It is imperative that the ports be hosted by the same IP address. That is, an `nmap` of the IP address should reveal both ports.

This is a situation which docker-compose would solve very easily. However, my company's GitLab runners don't support docker-compose and everyone uses Kubernetes (via OpenShift, if it matters). I know that I can create NodePort services for the two ports. However, if I'm understanding Kubernetes networking correctly, there's no guarantee that the two services would be on the same IP address as far as the outside world is concerned.

Is there any way to force Kubernetes to do this?

# Answer
This can be accomplished quite easily by, instead of creating two different services, creating one service with two ports:

```
spec:
  ports:
    - name: port1
      port: 5000
      targetPort: 5000
    - name: port2
      port: 5001
      targetPort: 5001

```

There will be only one domain name for the service and hence both ports will be open on the same IP address.
> 1 votes

---
Tags: kubernetes, networking
---


# Question
Title: How to design an ansible workflow over multiple hosts?
I am new to ansible and I am wondering how I should design my workflow with ansible in an ansible kind of way.

**The Task:** I have a workflow/process (not ansible yet) for setting an application up. This workflow has to do serveral tasks on different hosts and to store some configuration in git repositories. Depending on the tenend (e.g. A and B) of my application, it has to setup different stuff on different hosts. My workflow from top to down:

| # | host | tenand | task description |
| --- | --- | --- | --- |
| 1 | localhost | A & B | Create configurations and store it in an git repo and push to bitbucket |
| 2 | localhost | A & B | Setup an git repo with basic configs and push to bitbucket |
| 3 | localhost | A & B | Setup some other configs and post it with HTTP to a service |
| 4.1 | host a,b,c | A | Create a basic filesystem (directory and files) |
| 4.2 | host a,d | B | Create a basic filesystem (directory and files) |
| 5.1 | host a,b,c | A | Restart a deamon |
| 5.2 | host a,d | B | Restart a deamon |

**Thoughts**: I did create a playbook in exactly this manner and tried to differ between the hosts depending on the tenand an in its own configuration file. However thats exactly the way I am used to by an workflow orchestraton tool. Is it the ansible way too?
Do I have to change my perspective to the hosts and should create a playbook for each host and decide during the runtime if there are tasks for this host?

# Answer
It depends mostly on How one build his inventory and has organized hosts and group variables. As well what kind of services and configuration needs to be addressed and if there are dependencies between them.

One of many possibilities is for an inventory file **`example.com`**

```
[tenants:children]
A
B

[A]
a.example.com
b.example.com
c.example.com

[B]
a.example.com
d.example.com

```

a minimal example playbook

```
---
- hosts: localhost
  become: false
  gather_facts: false

  pre_tasks:

  - name: Create configurations, store it in a Git repo and push it into Bitbucket
    debug:
      msg: "1"

  - name: Setup a Git repo with basic configs and push it into Bitbucket
    debug:
      msg: "2"

  - name: Setup some other configs and post it with HTTP to a service
    debug:
      msg: "3"

- hosts: A
  become: false
  gather_facts: false

  tasks:

  - name: Create a basic filesystem (directory and files)
    debug:
      msg: "4.1"

  - name: Restart a daemon
    debug:
      msg: "5.1"

- hosts: B
  become: false
  gather_facts: false

  tasks:

  - name: Create a basic filesystem (directory and files)
    debug:
      msg: "4.2"

  - name: Restart a daemon
    debug:
      msg: "5.2"

```

Called via

```
ansible-playbook --inventory example.com main.yml

```

it will result into an output of

```
PLAY [localhost] ************************************************************************************************************************************************

TASK [Create configurations, store it in a Git repo and push it into Bitbucket] *********************************************************************************
ok: [localhost] =>
  msg: '1'

TASK [Setup a Git repo with basic configs and push it into Bitbucket] *******************************************************************************************
ok: [localhost] =>
  msg: '2'

TASK [Setup some other configs and post it with HTTP to a service] **********************************************************************************************
ok: [localhost] =>
  msg: '3'

PLAY [A] ********************************************************************************************************************************************************

TASK [Create a basic filesystem (directory and files)] **********************************************************************************************************
ok: [a.example.com] =>
  msg: '4.1'
ok: [b.example.com] =>
  msg: '4.1'
ok: [c.example.com] =>
  msg: '4.1'

TASK [Restart a daemon] *****************************************************************************************************************************************
ok: [a.example.com] =>
  msg: '5.1'
ok: [b.example.com] =>
  msg: '5.1'
ok: [c.example.com] =>
  msg: '5.1'

PLAY [B] ********************************************************************************************************************************************************

TASK [Create a basic filesystem (directory and files)] **********************************************************************************************************
ok: [a.example.com] =>
  msg: '4.2'
ok: [d.example.com] =>
  msg: '4.2'

TASK [Restart a daemon] *****************************************************************************************************************************************
ok: [a.example.com] =>
  msg: '5.2'
ok: [d.example.com] =>
  msg: '5.2'

```
> 0 votes

---
Tags: ansible
---


# Question
Title: What is the difference between CMD and terminal in Windows machine?
While using the Windows machine we face soo much trouble running the command line scripts on CMD(command prompt). Few commands run successfully but some are not. Why is there any difference between terminal and CMD.

# Answer
The question needs details, like what type of script you're attempting to run, and the specific errors you're getting.

Generally speaking, by default Windows Terminal launches a PowerShell interpreter (but can launch others), whereas cmd.exe launches a command interpreter that's much older and has no PowerShell functionality.

If you're attempting to run a PowerShell script, it needs to be in a PowerShell session (start either PowerShell, PowerShell ISE, Windows Terminal with a PowerShell tab, or if you really want to, you can run in a cmd.exe window by invoking `powershell.exe -File MyScript.ps1`).

If it's a batch file, either interpreter can run it.

If it's a shell script, it's most likely meant to be executed on Linux.
> 0 votes

# Answer
In my opinion, the major difference between Windows 'cmd/PS' and Linux 'terminal' is the command line or shell language intepreters.

They are both shells but have different script language, features and modules or packages.

Windows CMD/PS Scripts are usually saved as 'file.bat' or 'file.ps1'. Note PS Scripts such as 'file.ps1' would only run using PowerShell interface while a 'file.bat' can run on both CMD and PowerShell interfaces.

Linux Scripts files are usually saved as 'file.sh' and can only be run on terminal interface.
> 0 votes

---
Tags: windows, shell-script, powershell
---


# Question
Title: How to deploy AKS without exposing cluster API to the internet?
How to deploy Azure AKS kubernetes in a private network mode without exposing cluster API to the internet? I need to deploy React with Github actions and test that it's working

# Answer
What about this :

> In a private cluster, the control plane or API server has internal IP addresses that are defined in the RFC1918 - Address Allocation for Private Internet document. By using a private cluster, you can ensure network traffic between your API server and your node pools remains on the private network only.
>
> The control plane or API server is in an Azure Kubernetes Service (AKS)-managed Azure resource group. Your cluster or node pool is in your resource group. The server and the cluster or node pool can communicate with each other through the Azure Private Link service in the API server virtual network and a private endpoint that's exposed on the subnet of your AKS cluster.
>
> When you provision a private AKS cluster, AKS by default creates a private FQDN with a private DNS zone and an additional public FQDN with a corresponding A record in Azure public DNS. The agent nodes continue to use the A record in the private DNS zone to resolve the private IP address of the private endpoint for communication to the API server.

https://learn.microsoft.com/en-us/azure/aks/private-clusters?tabs=azure-portal
> 0 votes

---
Tags: kubernetes, azure-devops, github-actions
---


# Question
Title: Is SSL/HTTPS Needed Between Docker Containers on the Same Network?
As the title of the questions asks, is it important/needed to have ssl/https working between docker containers on the same network?

Our setup is an Nginx container that proxies two Tomcat containers.

The Nginx container performs ssl termination for the internet cloud.

Then the http request is forwarded to one of two tomcat instances in the same docker network.

For security/safety should the communication between the Nginx container and the two Tomcat containers happen over ssl?

Thank you for your time.

# Answer
Generally - no, I don't think you need TLS between your reverse proxy and service containers within same Docker network (or possibly 2 Docker networks: 1 internal for workers, 1 with internet access where reverse proxy sits and bridges the gap into internal network).

A wider answer would primarily depend on whether you trust your Docker host and services on it. Then consider how containers address each other, do they need to leave docker network (like public name resolution) at any time. And what you'd be looking for from TLS: identity verification or transport encryption. Each of those considerations would have some caveats.
> 3 votes

# Answer
As security cannot be high enough, using SSL between containers *can* be implemented, but:

* eavesdropping on the traffic among containers is nearly impossible as your containers are isolated from other networks, so you are already protected
* HTTPS is slower to 4x-10x than HTTP
* and from an IT security perspective it is not worth the price and resources it would take to implement the eavesdropping in your system (except if you store the next week's winning lottery numbers there).

So in short, it is unnecessary.
> 1 votes

# Answer
It depends on the security stance of your project. If you're simply serving stickers to a chat app, then no, you probably don't need to encrypt data in transit.

But, if you're developing an application that runs in a bank, then generally the answer is yes, you must have tls between the service in a cluster. As another answer mentioned, there is a significant cost to this, so that cost must be weighed against the risk of doing nothing and the impact if it all goes horribly wrong.
> 0 votes

---
Tags: docker, nginx, ssl, tomcat
---


# Question
Title: how to stop "docker run -it certbot/certbot"?
I'm newbie in docker. I use below command to run the certbot image twice:

```
docker run --rm certbot/certbot

```

I'd like to stop it but I can't. This command has started to produce many containers and these containers are automatically closed and produced again.

I have done the following:

1.Remove and kill containers

```
docker rm -f $(docker ps -a -q)
docker kill $(docker ps -q)

```

2.Through the following command, I tried to get the ID process and kill it:

```
ps aux | grep <containerId> | awk '{print $1 $2}'

```

then

```
kill -9 <processId>

```

But the ID process keeps changing.

3.I restarted Docker, even rebooted the whole server but still the problem persists.

I posted a screen record gif that shows that containers are constantly generated with different IDs and are automatically deleted.

How to resolve this problem?

Docker version 24.0.2, build cb74dfc
CentOS Linux release 7.9.2009 (Core)

# Answer
I have created a swarm service, it will keep recreating the container everytime it fails, which is expected and The certbot service is constantly failing.

I disabled swarm mode with below command and re-initialized swarm again and my problem has been solved.

```
docker swarm leave --force
docker swarm init

```
> 0 votes

# Answer
I would check the logs and see if something is notable there first.

```
 # docker logs 459979a55425
    [2024-01-30 23:56:18 +0000] [1] [INFO] Starting gunicorn 21.2.0
    [2024-01-30 23:56:18 +0000] [1] [INFO] Listening at: http://0.0.0.0:5000 (1)
    [2024-01-30 23:56:18 +0000] [1] [INFO] Using worker: sync
    ...

```

And try monitoring `events` while the container is active. This would need to be run in another terminal:

```
# docker events
2024-02-29T06:59:09.512867172-06:00 network connect ...(container=...b7fda34f83a, name=bridge, type=bridge)
...

```

The check if there is a restart policy for some reason (https://docs.docker.com/config/containers/start-containers-automatically/).

```
# docker inspect 413071a51305
           ...
           "RestartPolicy": {
                "Name": "no",
                "MaximumRetryCount": 0
            },
            ...

```

If restart is set to `always` or similar, you can change it with something like below. There is more in the link above:

```
docker update --restart=no 459979a55425

```
> 0 votes

---
Tags: docker, docker-swarm, images, centos, certbot
---


# Question
Title: Ansible - Configuring Supermicro servers via Redfish API
I plan to configure Supermicro servers via the Redfish API. Ansible provides the following modules for the Redfish API:

Supermicro offers this Redfish Reference Guide, which describes its Redfish REST API in detail. However, the Ansible modules use some sort of *commands* to do the configurations, instead of URIs.

Can someone explain how to map the API options to the Ansible modules? E. g., how can I use the Ansible modules to set the hostname or to provide NTP settings?

# Answer
As outlined in U880D's helpful answer, you currently really have to dig down into the code to get working examples for Ansible's Redfish modules.

As a rule of thumb, I recommend to look at the info you can *get*, to get an idea of info you can probably *set*. Use both the Supermicro Redfish docs and the redfish\_info module to see what you can get. Be aware, the results are not identical and neither the former nor the latter is more complete than the other. The two examples below will highlight some differences.

The "commands" used by the Ansible modules cannot be found in the Supermicro/Redfish docs. They are synonyms for certain API URIs. You have to look them up in the code, e. g. here and try to map them to the Supermicro/Redfish docs. Or dig down deeper to get to the functions that compose the URLs.

# Hostname

From the Supermicro docs, you can see that there is a property called `HostName`:

> ```
> URI: /redfish/v1/Managers/1/EthernetInterfaces/1
> Method: GET
> Response: 200
> {
>     ...
>     "MTUSize": 1500,
>     "HostName": "local",
>     "FQDN": "local.supermicro.com",
>     ...
> }
>
> ```

FYI, querying the same info with Ansible, e. g.:

```
- name: Get nic info
  community.general.redfish_info:
    category: Manager
    command: GetManagerNicInventory
    baseuri: "{{ baseuri }}"
    username: "{{ username }}"
    password: "{{ password }}"
  register: result

- debug:
    var: result

```

will not return a "hostname" property (output omitted). So a win for the Supermicro docs...

Fortunately, you can still *set* the hostname with Ansible:

```
- name: Set hostname
  community.general.redfish_config:
    category: Manager
    command: SetManagerNic
    nic_config:
      HostName: MyHostname
    baseuri: "{{ baseuri }}"
    username: "{{ username }}"
    password: "{{ password }}"

```

# NTP

The Supermicro docs show the following NTP properties:

> ```
> URI: /redfish/v1/Managers/1/Oem/Supermicro/NTP
> Method: PATCH
> Payload:
> {
>     "NTPEnable": true,
>     "PrimaryNTPServer":"127.0.0.1",
>     "SecondaryNTPServer":"127.0.0.1",
>     "DaylightSavingTime": false
> }
>
> ```

But with the following Ansible tasks:

```
- name: Get services info
  community.general.redfish_info:
    category: Manager
    command: GetNetworkProtocols
    baseuri: "{{ baseuri }}"
    username: "{{ username }}"
    password: "{{ password }}"
  register: result

- debug:
    var: result

```

you will get different NTP properties:

```
...
NTPServers:
- 127.0.0.1
- 127.0.0.1
NTPServers@odata.count: 2
ProtocolEnabled: true
...

```

And this time it is a win for redfish\_info as it turns out you have to use the properties from its output. Example:

```
- name: Set NTP
  community.general.redfish_config:
    category: Manager
    command: SetNetworkProtocols
    network_protocols:
      NTPServers:
        - 127.0.0.1
        - 127.0.0.1
      ProtocolEnabled: true
    baseuri: "{{ baseuri }}"
    username: "{{ username }}"
    password: "{{ password }}"

```

# Further reading

Besides the examples of the Ansible modules, you can get more examples here:
> 1 votes

# Answer
> *Supermicro offers this Redfish Reference Guide, which describes its Redfish REST API in detail.*

Right. According the documentation you've referenced, Ch. 4 "*Using RESTful APIs*" p. 13 ff., the structure of the REST API call seems to be

```
https://${URL}/redfish/v1/${CATEGORY} ...

```

followed by further actions, commands, tasks or payloads.

> *Can someone explain how to map the API options to the Ansible modules?*

After looking up the module documentation examples and the source code

it turns out that the modules using

as a wrapper the utilize the REST API. It maps the REST API structure to the parameter structure of the modules (`category`, `command`, `payload`). For a better understanding you may have a look into the mentioned source.

> *How can I use the Ansible modules to set the hostname or to provide NTP settings?*

According the source it seems to be the `manager_services` for NTP. Because of the lack of the specific environment I'm not able to investigate this further and to do tests, therefore for how to use it you may also have a look into the source.

To get familar with the Redfish RESTful API you may also have a look into DMTF Redfishtool it

> *was originally written during the development of the Redfish specification to help find ambiguities in the spec.*

**Further Examples**

There are some explanations and a general programming example from a bigger production environment available

**Documentation**
> 2 votes

---
Tags: ansible
---


# Question
Title: EC2 can't get to internet? SG and ACL seem OK
I've applied the terraform code below (`variables.tf` and `main.tf` are, of course, separate IRL) to setup an AWS `VPC`, private and public subnets, routes, NAT gateway, Internet Gateway, Network ACLs, an EC2 instance (in the public subnet) with a Security Group attached.

I can ssh into the EC2 just fine, but `apt update` won't complete. `tcpdump` shows `SYN` packets going out `eth0` but nothing coming back. DNS resolution works, and I can ping `10.12.100.1` which is something upstream from `eth0`, I'm not sure what, but that's about it. The EC2 has a private IP on `eth0` which is in the `10.12.100.x` CIDR and an IPv6 address which god only knows what IP is on that.

I've spent the better part of a day redoing `aws_route` and `aws_route_table` resources trying to figure out if I have something wrong there and comparing my resources to the terraform `vpc` module code and I just can't figure it out.

It seems like it should be an `SG` or `ACL` issue but when I look at those configurations in the AWS console, they are as configured, and allowing all egress traffic.

Anyone know what's missing, or how I can troubleshoot this more efficiently? Thanks

```
#############################################################################
#                         variables.tf
#############################################################################

variable "aws_region" {
  description = "AWS region where resources will be created"
  type        = string
  default     = "us-east-2"
}

variable "project" {
  description = "Project tag value for resources"
  type        = string
  default     = "vpc-test"
}

variable "environment" {
  description = "Environment tag value for resources"
  type        = string
  default     = "test"
}

variable "owner" {
  description = "The owner of the project"
  type        = string
  default     = "xyz"  # initials
}

variable "ssh_pub_key" {
  description = "pubkey of the SSH key pair for EC2 instances"
  type        = string
  default     = "ssh-ed25519 AAAAC...9"
}

variable "ingress_cidr_blocks" {
  description = "CIDR blocks to allow for ingress traffic"
  type        = list(string)
  default     = ["63.9.0.0/16", "64.10.1.1/32"]
}

variable "vpc_dns_support" {
  description = "Enable DNS support for the VPC"
  type        = bool
  default     = true
}

variable "vpc_dns_hostnames" {
  description = "Enable DNS hostnames for the VPC"
  type        = bool
  default     = true
}

variable "vpc_cidr_block" {
  description = "IPv4 CIDR block for the VPC"
  type        = string
  default     = "10.12.0.0/16"
}

variable "subnet_cidr_public" {
  description = "CIDR block for the public end of the subnet"
  type        = string
  default     = "10.12.100.0/24"
}

variable "subnet_cidr_private" {
  description = "CIDR block for the private end of the subnet"
  type        = string
  default     = "10.12.1.0/24"
}

variable "availability_zone" {
  description = "Availability zone for the subnet"
  type        = string
  default     = "us-east-2a"
}

variable "instance_type" {
  description = "EC2 instance type"
  type        = string
  default     = "t2.micro"  # Free tier - could be a policy in spacelift
}

variable "ami_instance" {
  description  = "AMI Instance identifier"
  type         = string
  default      = "ami-024e6efaf93d85776" # Ubuntu 22.04 Server in us-east-2
}

variable "ec2_webserver_data" {
  description = "User data script to initialize EC2 instances"
  type        = string
  default     = <<-EOF
    #!/bin/bash

    test ! -d /var/www/html && mkdir -p /var/www/html

    # Update package repositories and install tmux and vim
    apt-get update -y
    apt-get install apache2 tmux vim dig -y

    echo "<b>Private appserver instance B</b>" > /var/www/html/index.html
    systemctl enable apache2

    # Disable web caching in apache config so subsequent page loads don't conflict
    cat <<EOT >> /etc/apache2/apache2.conf
    <Directory /var/www/html>
        <IfModule mod_headers.c>
            Header unset ETag
            Header set Cache-Control "max-age=0, no-cache, no-store, must-revalidate"
            Header set Pragma "no-cache"
            Header set Expires "Wed, 11 Jan 1984 05:00:00 GMT"
        </IfModule>
    </Directory>
    EOT

    systemctl start apache2
    EOF
}

variable "ec2_std_data" {
  description = "User data script to initialize EC2 instances"
  type        = string
  default     = <<-EOF
    #!/bin/bash

    # Update package repositories and install tmux and vim
    apt-get update -y
    apt-get install apache2 tmux vim dig -y
    EOF
}

#############################################################################
#                         main.tf
#############################################################################

# Define the provider (AWS)
provider "aws" {
  region = var.aws_region
}

# the ssh pub key to use --  get from environment variable TV_VAR_ssh_pub_key
resource "aws_key_pair" "my_key_pair" {
  key_name   = "${var.project}-key-pair"  # Replace with the desired key name
  public_key = var.ssh_pub_key
}

# VPC
resource "aws_vpc" "my_vpc" {
  cidr_block           = var.vpc_cidr_block
  enable_dns_support   = var.vpc_dns_support
  enable_dns_hostnames = var.vpc_dns_hostnames

  tags = {
    Name        = "${var.project}-TEST-VPC"
    Environment = var.environment
    Project     = var.project
    Owner         = var.owner
  }
}

# private subnet within the VPC
resource "aws_subnet" "private_subnet" {
  vpc_id                  = aws_vpc.my_vpc.id
  cidr_block              = var.subnet_cidr_private
  availability_zone       = var.availability_zone
  map_public_ip_on_launch = false

  tags = {
    Name        = "${var.project}-PRIV-SUB"
    Environment = var.environment
    Project     = var.project
    Owner         = var.owner
  }
}

# public subnet within VPC
resource "aws_subnet" "public_subnet" {
  vpc_id                  = aws_vpc.my_vpc.id
  cidr_block              = var.subnet_cidr_public
  availability_zone       = var.availability_zone
  map_public_ip_on_launch = true     # For public subnet

  tags = {
    Name        = "${var.project}-PUB-SUB"
    Environment = var.environment
    Project     = var.project
    Owner         = var.owner
  }
}

resource "aws_network_acl" "priv_sub_acl" {
  vpc_id      = aws_vpc.my_vpc.id
  subnet_ids  = [aws_subnet.private_subnet.id]

  # Define ingress for trusted networks
  ingress {
    rule_no       = 100
    protocol      = "-1"
    action        = "allow"
    cidr_block    = aws_vpc.my_vpc.cidr_block
    from_port     = 0
    to_port       = 0
  }

  # Define ingress (inbound) rules
  egress {
    rule_no       = 100
    protocol      = "-1"
    action        = "allow"
    cidr_block    = "0.0.0.0/0"
    from_port     = 0
    to_port       = 0
  }

  # Tags
  tags = {
    Name          = "${var.project}-PRIV-ACL"
    Environment   = var.environment
    Project       = var.project
    Owner         = var.owner
  }
}

resource "aws_network_acl" "pub_sub_acl" {
  vpc_id      = aws_vpc.my_vpc.id
  subnet_ids  = [aws_subnet.public_subnet.id]

  # allow trusted networks ingress
  ingress {
    rule_no       = 100
    protocol      = "-1"
    action        = "allow"
    cidr_block    = var.ingress_cidr_blocks[0]
    from_port     = 0
    to_port       = 0
  }

  # dynamic config for multiple elements of var.ingress_cidr_blocks
  dynamic "ingress" {
    for_each  = range(length(var.ingress_cidr_blocks))
    iterator = i
    content {
      rule_no       = 100 + i.value * 100
      protocol      = "-1"
      action        = "allow"
      cidr_block    = var.ingress_cidr_blocks[i.value]
      from_port     = 0
      to_port       = 0
    }
  }

  # Define egress
  egress {
    rule_no       = 100
    protocol      = "-1"
    action        = "allow"
    cidr_block    = "0.0.0.0/0"
    from_port     = 0
    to_port       = 0
  }

  # Define IPv6 egress rule
  egress {
    rule_no             = 101
    protocol            = "-1"
    action              = "allow"
    ipv6_cidr_block     = "::/0"  # IPv6 CIDR block
    from_port           = 0
    to_port             = 0
  }

  # Tags
  tags = {
    Name          = "${var.project}-PUB-ACL"
    Environment   = var.environment
    Project       = var.project
    Owner         = var.owner
  }
}

# Create an Internet Gateway
resource "aws_internet_gateway" "my_igw" {
  vpc_id       = aws_vpc.my_vpc.id

  tags = {
    Name        = "${var.project}-IGW"
    Environment = var.environment
    Project     = var.project
    Owner         = var.owner
  }
}

# Elastic IP for the NAT Gateway
resource "aws_eip" "nat_eip" {
  domain           = "vpc"

  # Tags
  tags = {
    Name          = "${var.project}-NATG-EIP"
    Environment   = var.environment
    Project       = var.project
    Owner         = var.owner
  }
}

# NAT Gateway in the public subnet for private resources to reach internet resources
resource "aws_nat_gateway" "my_natg" {
  allocation_id     = aws_eip.nat_eip.id
  subnet_id         = aws_subnet.public_subnet.id  #< NATG needs to reside in pub sub

  # Tags
  tags = {
    Name          = "${var.project}-NATG"
    Environment   = var.environment
    Project       = var.project
    Owner         = var.owner
  }

  depends_on = [aws_internet_gateway.my_igw]
}

# route for private subnet - uses NATG
resource "aws_route_table" "private_route_table" {
  vpc_id = aws_vpc.my_vpc.id

  route {
    cidr_block = "0.0.0.0/0"
    nat_gateway_id = aws_nat_gateway.my_natg.id
  }

  tags = {
   Name = "${var.project}-PRIV-RT"
    Environment = var.environment
    Project     = var.project
    Owner       = var.owner
  }
}

# route to internet for vpc resources
resource "aws_route_table" "public_route_table" {
  vpc_id = aws_vpc.my_vpc.id

  route {
      cidr_block = "0.0.0.0/0"
      gateway_id = aws_internet_gateway.my_igw.id
  }

  tags = {
    Name        = "${var.project}-PUB-RT"
    Environment = var.environment
    Project     = var.project
    Owner       = var.owner
  }
}

resource "aws_default_route_table" "default_route_table" {
  default_route_table_id = aws_vpc.my_vpc.default_route_table_id

  tags = {
    Name        = "${var.project}-DEF-RT"
    Environment = var.environment
    Project     = var.project
    Owner       = var.owner
  }
}

# Associate pubsub with pub route
resource "aws_route_table_association" "public_association" {
  subnet_id      = aws_subnet.public_subnet.id
  route_table_id = aws_route_table.public_route_table.id

  # no tags
}

# Associate privsub with priv route
resource "aws_route_table_association" "private_association" {
  subnet_id      = aws_subnet.private_subnet.id
  route_table_id = aws_route_table.private_route_table.id

  # no tags
}

# Define a security group to allow incoming SSH traffic (adjust rules as needed)
resource "aws_security_group" "ec2_sg" {
  name                = "${var.project}-SG"
  description         = "Security group allowing SSH traffic from campus only, egress anywhere"
  vpc_id              = aws_vpc.my_vpc.id

  # Ingress rule for SSH
  ingress {
    from_port         = 22
    to_port           = 22
    protocol          = "tcp"
    cidr_blocks       = var.ingress_cidr_blocks
  }

  # Egress rule for all IPv4 traffic anywhere
  egress {
    from_port         = 0
    to_port           = 0
    protocol          = "-1"
    cidr_blocks       = ["0.0.0.0/0"]
  }

  # Egress rule for all IPv6 traffic anywhere
  egress {
    from_port         = 0
    to_port           = 0
    protocol          = "-1"
    ipv6_cidr_blocks  = ["::/0"]
  }

  # Tags
  tags = {
    Name          = "${var.project}-EC2-SG"
    Environment   = var.environment
    Project       = var.project
    Owner         = var.owner
  }
}

/*

# Create an EC2 instance in privsub
resource "aws_instance" "my_ec2_instance" {
  ami                     = var.ami_instance
  instance_type           = var.instance_type
  key_name                = aws_key_pair.my_key_pair.key_name
  user_data               = var.ec2_std_data
  vpc_security_group_ids  = [aws_security_group.ec2_sg.id]
  subnet_id               = aws_subnet.private_subnet.id

  metadata_options {
    http_tokens          = "required"
  }

  # Tags
  tags = {
    Name          = "${var.project}-priv1-ec2"
    Environment   = var.environment
    Project       = var.project
    Owner         = var.owner

  }
}

# Create a second EC2 instance in privsub
resource "aws_instance" "my_ec2_instance_b" {
  ami                     = var.ami_instance
  instance_type           = var.instance_type
  key_name                = aws_key_pair.my_key_pair.key_name
  user_data               = var.ec2_std_data
  vpc_security_group_ids  = [aws_security_group.ec2_sg.id]
  subnet_id               = aws_subnet.private_subnet.id

  metadata_options {
    http_tokens          = "required"
  }

  # Tags
  tags = {
    Name          = "${var.project}-priv2-ec2"
    Environment   = var.environment
    Project       = var.project
    Owner         = var.owner
  }
}
*/

# Create a second EC2 instance in pubsub
resource "aws_instance" "my_ec2_instance_c" {
  ami                     = var.ami_instance
  instance_type           = var.instance_type
  key_name                = aws_key_pair.my_key_pair.key_name  # Reference the key name from the aws_key_pair
  user_data               = var.ec2_webserver_data
  vpc_security_group_ids  = [aws_security_group.ec2_sg.id]
  subnet_id               = aws_subnet.public_subnet.id

  metadata_options {
    http_tokens          = "required"
  }

  # Tags
  tags = {
    Name          = "${var.project}-pub1-ec2"
    Environment   = var.environment
    Project       = var.project
    Owner         = var.owner
  }
}

```

# Answer
> 0 votes
So, for anyone else who stumbles across this, the ACLs don't know that the SYN-ACK packets coming back from a connection initiated from within an EC2 is traffic that is related to that connection (stateless).

This means you can't just `apt update && apt dist-upgrade -y` without first opening `0/0` on the inbound side of the ACL.

---
Tags: amazon-web-services
---


# Question
Title: What would be the best approach or tool to deploy one-off SQL scripts?
A vendor is supplying one-off "hot fixes" and I am trying to streamline these deployments best I can. Currently I am going out to an FTP site, downloading the SQL scripts and manually deploying them against the various database environments. A few items to note:

1. Often these SQL scripts have to be executed against the database in a particular order according to the vendor, however this has caused problems in the past as sometimes manual deployments have led to errors as they were not executed in the correct order.
2. It is not given that these deployments will make it to production, often times deployments are not successful and redo scripts are then supplied from the vendor, effectively making these deployments throw away deployments.
3. I don't think there is an internal repo or CI/CD practice in place, as the belief is that this is the job of the vendor.
4. I would like to capture the output from each execution and send it to a MS Teams channel for the analyst to know the status of the deployment. (Something I've done in the past)

One thought was to create an Azure DevOps deployment and empowering the analyst to deploy these scripts when needed. My concern is the analyst would get confused on how to create a pull request in GitHub, so I'm not sure I can remove myself completely from the process as I probably would still have to download the scripts from the FTP site and drop them into GitHub and create the PR.

What can I do here to make this process easier and more reliant? These code changes almost happen daily and take a chunk of my time every day. Or am I just wasting my time trying to come up with something that doesn't might not have much value?

# Answer
The absolute first change in your process I would make is to add the files to source control. Maintaining the scripts in version control is probably your highest priority as this should be your main source of truth for your database scripts. I would add these scripts to source control before you run them. Then, having some type of cron job, ADO pipeline, etc. that downloads these files and pushes to source control is crucial.

> My concern is the analyst would get confused on how to create a pull request in GitHub.

It sounds like there are a lot of technical limitations to overcome here. Whether that is knowledge or boundaries elsewhere, I would say automating as much as you can to remove human error is the next step forward. Creating an ADO pipeline to run your deployments (even if having to be executed manually) is a great start.

> These code changes almost happen daily and take a chunk of my time every day. Or am I just wasting my time trying to come up with something that doesn't might not have much value?

I think you've answered your own question :) If this is a sizable chunk of your time daily, it's easily worth your time and effort to automate this workflow and make it more reliable. You will save hours a week and reduce the number of mistakes caused by repeating these manual deployments.
> 2 votes

---
Tags: azure-devops, continuous-deployment, github, continuous-delivery, github-actions
---


# Question
Title: Should I use monorepo for deploying projects?
I'm a newbie into devops tasks. I am working on a project in which I have the following parts: One frontend which provides info about the project (1), one frontend that is the actual web application (2), one frontend that provides authentication logic (3), one backend for the web application (4) and one backend for authentication (5). Each of them will not communicate with all the others, however there will be communications like 2-4, 3-5.

Now, each of them lives on its own git repo. I am starting to dockerize it, and I don't know if keeping them in separate repos is the best approach. I feel like it makes setting up docker a bit more difficult as each service will be run from a different docker compose file.

For the first steps, I guess having a single repo would be easier, however I don't feel like that is the best approach if the project would become larger and larger.

Any opinions with regard to this topic?

# Answer
> 0 votes
If you want to keep these components in separate repositories, you still can, and create a new, separate repository, similar to a monorepo, to hold all of these components. Check out git submodules or git subtrees? If you want to contribute back upstream from, you will want to use submodules, but your workflow becomes a little more complex. If you want to keep the same git workflow, and don't want to contribute back upstream, use subtrees.

---
Tags: docker, docker-compose, monorepo
---


# Question
Title: Difference between Sync Phases and Waves?
Argo CD can synchronize (i.e. apply manifests) in order. The ordering is determined by *phase*, *wave*, resource kind (e.g. `Namespace`, `Deployment`), and name.

There are 3 phases:

* `PreSync`
* `Sync`
* `PostSync`

Within the `Sync` phase can be one or more *waves*, assigned to resources using the `argocd.argoproj.io/sync-wave` annotation, and has integer values, with the lower-numbered waves being applied first.

The question is - what's the purpose of sync phases when the ordering can already be achieved using sync waves? Or - what's the difference assigning a resource to the `PreSync` phase vs. assigning to a low-numbered sync wave?

# Answer
> 0 votes
**tl;dr** phases are used to order the running of *scripts*; waves are used to order the `apply` of application resources.

---

The `PreSync` and `PostSync` sync phases are used to run resource hooks \- scripts that are supposed to run to completion. For example:

* `PreSync` \- can be used to run data migration *scripts*
* `PostSync` \- can be used to run a smoke test *script*, or a *script* to send a Slack message

Because hooks are scripts, they tend to be `Pod`, `Job` or Argo Workflows.

**Hooks are meant to be deleted** after it runs; if and when exactly depends on the hook deletion policy.

On the other hand, resources applied within a sync wave are not scripts, but objects/services/config that are **meant to remain as part of the application after the sync** is over. Sync waves exists so you can dictate the order these resources are applied (e.g. make sure one service is healthy before starting another one).

---
Tags: argocd
---


title: Simple metrics of code in Azure Devops

text:
# Question
Title: Simple metrics of code in Azure Devops
I have just got the responsibility for around 100 different repositories (applications), distributed on about 20 project in Azure DevOps.

I want to get some (very) basic metrics on each project:

* Numbers and names of repositories in each project
* Number of branches in a repository
* Number of files in a branch
* Number of code lines in a branch
* Names of developers which has worked on a project
* Number of developers which has worked on a project
* First and last commit/pull request from a developer on a project
* etc.

Just simple counting and listing. Nothing fancy as test coverage etc. So far all my searching has only given results about code analysis and other advanced stuff. It's not where I am in my tasks right now.

# Answer
To gather basic metrics on your Azure DevOps projects and repositories, you can use the Azure DevOps REST API. Here’s a general approach to getting the metrics you need:

List of repositories in each project: Use the GET request to {organization}/{project}/\_apis/git/repositories?api-version=6.0. This will give you the names and IDs of all repositories in a project.

Number of branches in a repository: For each repository, make a GET request to {organization}/{project}/\_apis/git/repositories/{repositoryId}/refs?api-version=6.0. This will list all branches in the repository.

Number of files in a branch: This is trickier because there's no direct API to count files per branch. You might need to clone the repository and use a Git command like git ls-tree -r branchName --name-only | wc -l to list and count files.

Number of code lines in a branch: Similarly, after cloning the repository, you can use a command like git ls-files | xargs wc -l in the branch to get the total lines of code.

Names and number of developers who worked on a project: Use the GET request to {organization}/{project}/\_apis/git/repositories/{repositoryId}/commits?api-version=6.0 to fetch commits. From the commit data, you can extract the authors (developers).

First and last commit/pull request from a developer on a project: With the commit data fetched above, you can filter by author and sort by date to find the first and last commit for each developer.

You will need to write scripts that automate these API calls and process the responses to generate your metrics. For authentication, you might use a personal access token (PAT) with your requests to access the Azure DevOps REST API.
> 0 votes

---
Tags: azure, metrics, repository
---


# Question
Title: Custom Rule issues in Azure DevOps Services
I'm trying to customise the Scrum process in Azure DevOps to make it fit our needs better.

When we had TFS we were able to prevent a Developer from changing the state of a Product Backlog Item from "Quality Control" to "Ready for Test" if that developer was already in a "Developed\_By" field.

How do we mimic that behaviour in Azure DevOps, the rules seem to be quite limiting (we can't define reasons for state movement, we can't do conditional rules based on security)?

# Answer
Customizing the Scrum process in Azure DevOps to enforce specific rules, like preventing certain state transitions based on user roles or field values, can be challenging because Azure DevOps provides limited inbuilt customization options for process rules. However, there are some strategies you can employ to achieve similar functionality:

Process Customization: While Azure DevOps allows for some process customizations, such as adding or modifying work item types, fields, and states, it doesn't natively support complex conditional logic based on user roles directly within the state transition rules. You can check if there are any updates or extensions that might offer more advanced process customization options.

Azure DevOps Services Hooks: Use service hooks or webhooks to integrate with external services. You can set up a service hook that triggers on work item updates and then use an external service (like a custom web application or Azure Functions) to check the conditions you want to enforce (like the "Developed\_By" field). If the condition is not met, you can revert the change or notify the responsible parties to take appropriate action.

Custom Development: Develop a custom application or script that uses the Azure DevOps REST API to monitor work item changes and enforces the rules you need. This application can periodically check the state of work items and revert changes or notify users if their actions violate your process rules.

To mimic the specific behavior you described, you would likely need a combination of a service hook or webhook with a custom logic handler (like an Azure Function or a web service) that checks the "Developed\_By" field and validates the state transition based on your rules.
> 1 votes

---
Tags: azure, process
---


# Question
Title: Why are the outputs variables that I am passing from one job to another showing empty in Azure pipelines?
I have a pipeline with multiple jobs. I would like to send the output that I am setting in a powershell script to another job in the pipeline.

I have followed several examples to do this, like: 'https://medium.com/microsoftazure/how-to-pass-variables-in-azure-pipelines-yaml-tasks-5c81c5d31763' and 'https://devops.stackexchange.com/questions/17604/how-do-i-refer-in-azure-pipelines-to-an-output-variable-defined-in-a-previous-st'.

I can access the output variables that I am setting if the tasks are in the same job, but in different jobs the variables are empty.

This is my current pipeline:

```
- stage: DeploytoDev
  displayName: Deploy to Dev
  condition: ${{ eq(variables['Build.SourceBranch'], 'refs/heads/develop') }}
  jobs:
    - deployment: DeployInfraDev
      environment: env
      strategy:
        runOnce:
          deploy:
            steps:
            - task: AzureResourceManagerTemplateDeployment@3
              displayName: 'Infra Deployment: Storage Account, Function App Develop'

              inputs:
                deploymentScope: 'Subscription'
                azureResourceManagerConnection: '$(serviceConnectionDev)'
                action: 'Create Or Update Resource Group'
                subscriptionName: '$(subscriptionDev)'
                resourceGroupName: '$(resourceGroup)'
                location: '$(location)'
                templateLocation: 'Linked artifact'
                csmFile: '$(Pipeline.Workspace)/$(infraArtifactName)/templates/main.bicep'
                overrideParameters: >-
                  -appName $(appName)
                  -stage dev
                  -location $(location)
                deploymentMode: 'Incremental'
                deploymentName: 'DeployPipelineTemplate'
                deploymentOutputs: 'deploymentOutputs'
              name: 'deployment'

            - task: PowerShell@2
              name: 'outputVariables'
              displayName: 'Set Deployment Output Variables'
              inputs:
                targetType: inline
                script: |
                  $armOutputObj = '$(deploymentOutputs)' | ConvertFrom-Json

                  $armOutputObj.PSObject.Properties | ForEach-Object {
                    $keyname = $_.Name
                    $value = $_.Value.value

                    # Creates a standard pipeline variable
                    Write-Output "##vso[task.setvariable variable=$keyName;isOutput=true]$value"

                    # Display keys in pipeline
                    Write-Output "output variable: $keyName $value"
                  }
                  Write-Output "##vso[task.setvariable variable=test;isOutput=true]test"

                pwsh: true

            - bash: echo "$(outputVariables.test) | $(outputVariables.storageAccountName)"

    - job: BuildWebsite
      dependsOn: DeployInfraDev
      variables:
        storageAccountName: $[ dependencies.DeployInfraDev.outputs['outputVariables.storageAccountName'] ]
        storageAccountNameFoo: $[ dependencies.DeployInfraDev.outputs['outputVariables.test'] ]
      steps:
        - bash: echo "$(storageAccountName) | $(storageAccountNameFoo)"
        - bash: echo "$storageAccountName | $storageAccountNameFoo"
          env:
            storageAccountName: $(storageAccountName)
            storageAccountNameFoo: $(storageAccountNameFoo)

```

Image of the Bash inline scripts of the jobs:

* Same job bash script:
* Different Job without setting env variables in task
* Different Job while setting env variables in task

I was expecting that in both bash scripts in the 'BuildWebsite' Job would print the same values as the bash script from the 'DeployInfraDev' Job.

# Answer
For deployment jobs check this format `$[dependencies.<job-name>.outputs['<job-name>.<step-name>.<variable-name>']]`. Documentation: Support for output variables
> 0 votes

# Answer
While executing deployment strategies, you can access output variables across jobs using the following syntax.

For runOnce strategy: $\[dependencies..outputs\['..'\]\] (for example, $\[dependencies.JobA.outputs\['JobA.StepA.VariableA'\]\]) For runOnce strategy plus a resourceType: $\[dependencies..outputs\['*..'\]\]. (for example, $\[dependencies.JobA.outputs\['Deploy\_VM1.StepA.VariableA'\]\]) For canary strategy: $\[dependencies..outputs\['*..'\]\] For rolling strategy: $\[dependencies..outputs\['\_..'\]\]

https://learn.microsoft.com/en-us/azure/devops/pipelines/process/deployment-jobs?view=azure-devops#support-for-output-variables
> 0 votes

---
Tags: azure-devops, azure, azure-pipelines, pipeline
---


# Question
Title: Why Azure App Service auto serve my site on port 443?
## Project Setup

* My React app image is stored on Azure Container Registry (ACR), with a `Dockerfile` as below,

```
FROM node:16.19.1-alpine
WORKDIR /frontend
COPY ./frontend/package*.json .
RUN npm install
COPY ./frontend .
EXPOSE 5173
CMD npm run dev

```
* On Azure App Service, I configured a `docker-compose.yml` setup since there'll be multiple images needed to be deployed on this app service (e.g., backend service), the compose file as below,

```
version: '3.8'

services:
  fe:
    image: bugman195.azurecr.io/traddo-fe:latest
    build:
      context: .
      dockerfile: ./fe/Dockerfile
    ports:
      - '5173:5173' -> This port config is where I'm confused

```

## My question:

Why is my frontend code being served on port **443** automatically? Shouldn't it be served on port **5173** as my `docker-compose.yml` configured?

Is there some sort of load balancing mechanism automatically done by Azure to direct port 443 to my container's 5173 port?

# Answer
> 0 votes
The ports directive in your docker-compose.yml ('5173:5173') usually specifies port mappings for local development and is often ignored by hosting environments like Azure App Service, which have their own mechanisms for routing and exposing ports.

Your frontend code is served on port 443 because Azure App Service automatically handles the routing and SSL termination for your containerized applications, directing external HTTPS traffic to the internal port that your container exposes. This simplification is part of the managed service offering, allowing you to focus on the application development rather than the infrastructure configuration.

---
Tags: docker, docker-compose, azure
---


# Question
Title: Best artifactory type for Azure release pipeline
We've just started to use Azure DevOps platform and having some doubts regarding artifactory type for release pipeline.

Here's the situation: we get complete delivery in zip file from Dev team which is ready to be deployed to the target machines, which means that build pipeline is unnecessary here. Therefore, there's no `publish` step.

We tried to create feed and connect Universal Packages to it which wasn't right solution for us because it supports only Semantic versioning 2.0 (`MAJOR.MINOR.PATCH`), but we need to have `MAJOR.MINOR.PATCH.HOTFIX` versioning.

The only working solution was to push zip delivery to Azure Repos Git and add it as an artifact to the release pipeline which seems little bit awkward.

Is there any better way to solve this problem? Maybe something like creating service connection to Jfrog or something similar and keep our artifacts there?

Thanks in advance.

# Answer
> 0 votes
Using Azure Repos Git to store binary artifacts like a zip file can indeed be awkward and not the best use of a source control system. There are better ways to manage binary artifacts that allow for more flexible versioning and better storage management. Here are some alternatives you can consider:

1. Artifactory by JFrog JFrog Artifactory is a popular choice for storing binary artifacts and can support any versioning scheme you prefer, including MAJOR.MINOR.PATCH.HOTFIX. Artifactory integrates well with Azure DevOps, and you can use it to store your zip files. You can set up a service connection in Azure DevOps to Artifactory and then use this connection in your release pipeline to fetch the artifacts for deployment.

Steps to Integrate: Set up Artifactory: First, ensure you have Artifactory set up and accessible. Upload Artifacts: Configure your process to upload the zip files to Artifactory, with the versioning scheme you need. Create Service Connection in Azure DevOps: Go to your Azure DevOps project settings and set up a service connection to your Artifactory instance. Use in Release Pipeline: In your Azure DevOps release pipeline, use the Artifactory service connection to download the specific artifact version you want to deploy. 2. Azure Blob Storage If you don't require the full feature set of a dedicated artifact repository, you can use Azure Blob Storage to store your zip files. Azure Blob Storage is cost-effective and supports storing large files without any specific versioning constraints.

Steps to Use: Upload to Blob Storage: Automate the upload of your zip files to a Blob Storage container, possibly with a versioning scheme in the file name or path. Access in Azure DevOps: You can access these blobs in your release pipeline either by using Azure CLI tasks, PowerShell scripts, or custom tasks designed to interact with Azure Blob Storage.

---
Tags: azure, azure-pipelines, release, azure-artifacts
---


# Question
Title: How to specify a filter for Kubectl -o flag=custom-column
I want to use `kubectl get -o=custom-columns` to extract information with specific columns. The cropped yaml response is like this

```
{
    "apiVersion": "v1",
    "items": [
        {
            "apiVersion": "v1",
            "kind": "Pod",
            "metadata": {
                "creationTimestamp": "2018-11-27T14:37:46Z",
                "generateName": "kubernetes-bootcamp-5c69669756-",
                "labels": {
                    "pod-template-hash": "1725225312",
                    "run": "kubernetes-bootcamp"
                },
                "name": "kubernetes-bootcamp-5c69669756-cxhwf",
                "namespace": "default",
                "ownerReferences": [
                    {
                        "apiVersion": "extensions/v1beta1",
                        "blockOwnerDeletion": true,
                        "controller": true,
                        "kind": "ReplicaSet",
                        "name": "kubernetes-bootcamp-5c69669756",
                        "uid": "950aac8e-f24f-11e8-bf5d-065b6ba59aa8"
                    }
                ],
                "resourceVersion": "118035",
                "selfLink": "/api/v1/namespaces/default/pods/kubernetes-bootcamp-5c69669756-cxhwf",
                "uid": "0190c234-f252-11e8-bf5d-065b6ba59aa8"
            },

```

I want to extract the **name** from items\[0\].metadata.name, but I don't know the syntax.

My questions:

1. What is the query syntax used in this situation. Is it JSON path? I don't see it documented.
2. What is the specific query in my case?

# Answer
> 2 votes
Since the result is JSON, pipe the result to **jq** and go from there.

```
kubectl get | jq '.items[0].metadata.name'

```

yields

```
➜ cat abc.json | jq '.items[0].metadata.name'
"kubernetes-bootcamp-5c69669756-cxhwf"

```

# Answer
> 0 votes
The syntax of the command is as follows;

```
$ kubectl get pods <optional_pod_name> --output custom-columns="NAME:.metadata.name"
NAME
kubernetes-bootcamp-5c69669756-cxhwf

```

You can add more columns by separating them with commas. For example;

```
$ kubectl get pods <optional_pod_name> -o custom-columns="NAME:.metadata.name,STATUS:.status.phase"

NAME                                   STATUS
kubernetes-bootcamp-5c69669756-cxhwf   Running

```

# Answer
> 0 votes
You can also use JSONPath:

```
kubectl get nodes --output=custom-columns='$COLUMN_NAME:$JSON_PATH'
kubectl get nodes --output=custom-columns='NAME:{.metadata.name}' # optional curly bracket
kubectl get nodes --output=custom-columns='NAME:.metadata.name,POD_CIDR:{.spec.podCIDR}'
kubectl get nodes --output=jsonpath='{range .items[*]}{.metadata.name}{" "}{.spec.podCIDR}{"\n"}{end}'

```

---
Tags: kubernetes
---


title: A case for exceeding docker's max depth

text:
# Question
Title: A case for exceeding docker's max depth
I am building a common library inside a docker container. A number of applications depend on the library. They build inside the common container and the compiled files are copied in a "slim" application container.

The common library `Dockerfile` looks as such:

```
FROM common:latest
COPY sourcecode
RUN build steps for library

```

The reason to iterate over the latest image is to keep multiple versions of the same library in the latest image. Then the services which depend on the library can build inside latest common image and find the correct version of the library on which they depend.

Eventually, the max layer depth of docker builds is exceeded.

**Complication:** The common image actually contains multiple packages, each of which is versioned. All the apps that depend on the libraries are also versioned and depend of specific versions of the packages. All the builds happen automatically in a CI pipeline.

**Solution 1:** Don't build the common libraries in docker. Instead clone the common repository build the correct versions, build your app on top of it's dependencies and copy the artifact in an image. However, now your CI agent is building the same version of the libraries again and again.

**Solution 2:** Start versioning the common image. Find a way to correlate the app version with the common image tag. Pass the tag in a `--build-arg` to `docker build`. This solution is complicated. Versioning the common image is doable. For instance, the build digest can be used to tag. Now you can't correlate the app versions with the common bundle in a straight forward way.

Correlating is possible in my special case. The agreement is that functionality is only added to the common libraries, (whether this may be a terrible idea). In this case, every time the a common library is iterated, I can make a small commit in the app repositories, saving the docker common image tag and bumping some dependencies in the application source code.

Then this saved tag can serve the `docker build` of the applications until the next iteration of common. No matter how much any staging environment lags behind, I can use the tag that lives in the application repository branch from which the environment is deployed. This also allows patching against the tag in need of hot-fixes.

**Question:** How are you solving this problem? What are the common practices?

*P.S. This is not hypothetical. I exceeded the max depth today. Fortunately, someone made docker-squash.*

# Answer
> 1 votes
I ended up using docker-squash.

Install: `pip install docker-squash`

Example usage, squash up to `<LAYER>`:

```
docker-squash -f <LAYER> -t <IMAGE>:squashed <IMAGE>:<TAG>

```

---
Tags: docker, versioning
---


# Question
Title: How to determine the color of a shell script task in Ansible?
There are various Ansible modules which can execute my own-written script (ansible.builtin.script, ansible.builtin.shell, ansible.builtin.command). However, I would like to determine the color of the task as usual:

* red if execution was not successful
* yellow on successful execution resulted a modification of the system
* and green if successed an no change was needed.

I think, it could be encoded most easily in the exit code of the script, but any other scriptable option is to me okay.

Unfortunately, all the relevant modules I have found until now, simply completely ignore the result.

What to do?

# Answer
> ***... it could be encoded most easily in the exit code of the script ...***

Right, the documentation about Error handling in playbooks show how one Define failure or Define `changed`. This will work for `script`, `shell`, `command`, as well Custom Modules.

> ***... but any other scriptable option is to me okay.***

That is what the `diy` callback – Customize the output was made for. The Examples show how to implement and use it then.
> 1 votes

---
Tags: ansible, shell-script
---