# Question
Title: Azure Pipelines: How to install Xamarin components (for iOS & Android) on MacOS13 to make msbuild work in terms of building a Xamarin/C# project
Essentially what the subject says. I need Xamarin components in order to build some Xamarin/iOS libs that can only be build on MacOS (using sharpie etc). Currently I'm using MacOS12 (which does have Xamarin iOS + Android components pre-installed) but once I tried to move to MacOS13 I noticed that alot of stuff in the Xamarin side have been removed:

https://github.com/actions/runner-images/blob/main/images/macos/macos-12-Readme.md

vs

https://github.com/actions/runner-images/blob/main/images/macos/macos-13-Readme.md

I'm just wondering whether there's an easy way to reintroduce the Xamarin stuff (iOS and Android) manually from the command line in the MacOS13 image. For example I managed to install mono like so:

```
wget    https://download.mono-project.com/archive/6.12.0/macos-10-universal/MonoFramework-MDK-6.12.0.182.macos10.xamarin.universal.pkg

sudo    installer    -target /    -pkg MonoFramework-*.pkg

sudo   sh -c   "echo   '\n\nexport PATH=\"/Library/Frameworks/Mono.framework/Versions/Current/bin:\$PATH\"\n\n'   >> ~/.bash_profile"

source    ~/.bash_profile

```

How can I achieve something similar for the Xamarin iOS+Android components?

My C# .csproj essentially employs this bit:

```
<Project Sdk="MSBuild.Sdk.Extras/3.0.44">
    <TargetFrameworks>monoandroid12.0;xamarin.ios10;</TargetFrameworks>
</Project>

```

I have already enriched the agent with a demand to install Xamarin.Android like so:

```
pool:
    name: Azure Pipelines
    demands:
        - msbuild
        - Xamarin.Android

```

But this doesn't do the trick - I still get an error during my msbuild about android and ios xamarin stuff missing:

```
/Users/runner/.nuget/packages/msbuild.sdk.extras/3.0.44/Build/LanguageTargets/CheckMissing.targets(44,5): error : The specified language targets for monoandroid12.0 is missing. Ensure correct tooling is installed for 'monoandroid'. Missing: '/Library/Frameworks/Mono.framework/Versions/6.12.0/lib/mono/xbuild/Xamarin/Android/Xamarin.Android.Bindings.targets' [/Users/runner/work/1/s/bindings/foo.csproj]

```

# Answer
It looks like your problem references this issue #7635.

According to this post you can use:

```
VERSION=6.12.0.182
MONO_FOLDER_NAME=$(echo $VERSION | cut -d. -f 1,2,3)
PKG_URL="https://download.mono-project.com/archive/${MONO_FOLDER_NAME}/macos-10-universal/MonoFramework-MDK-${VERSION}.macos10.xamarin.universal.pkg"
PKG_NAME=${PKG_URL##*/}
curl -4fsLo "${PKG_NAME}" "${PKG_URL}"
sudo installer -pkg "${PKG_NAME}" -target /

```

Please share your experience if you managed to solve the problem.
> 1 votes

---
Tags: azure-devops, azure, azure-pipelines, macos
---


title: terraform remote state bucket on s3

text:
# Question
Title: terraform remote state bucket on s3
I want to use a remote state bucket on s3 for my terraform tfstate files (described here: https://developer.hashicorp.com/terraform/language/settings/backends/s3). It looks pretty basic, but in practice not so straightforward. I tried just defining the bucket like any other s3 resource but terraform won't init/plan anything since the state bucket doesn't exist. So, I used aws-cli to create the state bucket and was planning on dropping all the details into `variables.tf` but then `init` produced:

```
# Error: Variables not allowed

```

My entire root module/project references `variables.tf` for everything. From tags, to subnets to AWS region - everything. I've got like 30 resources defined for this project. The path forward that I see here is quite a mess. literal values in the state bucket code, multiple files to maintain the state bucket which are detached from the "project" and probably in a separate git repo, then the bucket and IAM polices and dynamoDB config... ugh.. At this point, the remote state bucket creates more problems than it solves. I'm to the point where a makefile pulling/pushing the `.tfstate` files back-and-forth to a regular s3 bucket makes more sense. Can someone point me in the right direction here, or is this really the way it is?

state bucket code:

```
resource "aws_s3_bucket" "my_tf_state_bucket" {
  bucket = var.state_bucket_name

   tags           = {
      Name              = "${var.my_project}-tf-state-bucket"
      project           = var.my_project
      environment       = var.my_environment
   }

}

terraform {
  backend "s3" {
    bucket         = var.state_bucket_name  # Replace with your S3 bucket name
    key            = "terraform.tfstate"
    region         = var.my_region
    encrypt        = true
    dynamodb_table = "${var.my_tf_state_bucket}-terraform-locking"

  }
}

```

---

**update:** In case it's useful for anyone else, and since this was a particularly nasty way of configuring a working state bucket, here is what I ended up doing.

I created `s3-state-bucket.tf` with the contents below. This allows me to keep the configuration for the state bucket in the same directory as the rest of my terraform files for the project and the commands to create the state bucket can be a shell script or pasted in from comments left in the `.tf` file. The script can be checked into git/hub/lab with the rest of the project.

**Use bash to create a unique bucket name, enable object lock, SSL connections, bucket key + encryption, versioning, block public access**

```
UNIQUE_ID=$((RANDOM % 899 + 100)); MY_BUCKET="S3-uniquely-named-state-bucket-${UNIQUE_ID}"; aws s3api create-bucket --bucket ${MY_BUCKET} --region $AWS_REGION
--create-bucket-configuration LocationConstraint=$AWS_REGION --object-lock-enabled-for-bucket

```

**Enable bucket key and SSE with AWS managed keys to save on costs for KMS key calls**

```
aws s3api put-bucket-encryption --bucket ${MY_BUCKET} --server-side-encryption-configuration '{"Rules": [{"ApplyServerSideEncryptionByDefault": {"SSEAlgorithm": "AES256"},"BucketKeyEnabled": true}]}'

```

**Enable object lock so tfstate files can't be accidentally modified without explicit locking. Include retention policy for data**

```
aws s3api put-object-lock-configuration --bucket ${MY_BUCKET} --object-lock-configuration '{"ObjectLockEnabled":"Enabled","Rule":{"DefaultRetention":{"Mode":"COMPLIANCE","Days":30}}}'

```

**Ensure public access is specifically disabled, which seems to be an evolving default for AWS but whatever...**

```
aws s3api put-public-access-block --bucket ${MY_BUCKET} --public-access-block-configuration "BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true"

```

**Add a bucket policy to only allow SSL connections to keep compliance happy**

```
aws s3api put-bucket-policy --bucket  ${MY_BUCKET} --policy  "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Deny\",\"Principal\":\"*\",\"Action\":\"*\",\"Resource\":[\"arn:aws:s3:::${MY_BUCKET}\",\"arn:aws:s3:::${MY_BUCKET}/*\"],\"Condition\":{\"Bool\":{\"aws:SecureTransport\":\"false\"}}}]}"

```

**Add tags to the bucket as this aids with cost and resouce managment, monitoring, automation and other processes**

```
aws s3api put-bucket-tagging --bucket ${MY_BUCKET} --tagging 'TagSet=[{Key=environment,Value=production},{Key=project,Value=tf-state-bucket-for-project},{Key=Name,Value='${MY_BUCKET}'}]'

```

\**Added to variables.tf for applying additional access/security controls or configuration \**

```
// the name of the terraform state bucket created using aws-cli so we can apply additional config to the bucket
variable "my_tf_state_bucket" {
  type        = string
  description = "Terraform state bucket created using aws-cli commands"
  default     = "S3-uniquely-named-state-bucket-448"
}

```

**..if you want to remove it all in one shot**

```
aws s3 rm s3://${MY_BUCKET} --recursive; aws s3 rb s3://${MY_BUCKET}

```

**dynamo db table for state locking - not sure if this is optional or not**

```
aws dynamodb create-table --table-name dynamodb-${MY_BUCKET}-locking-table --attribute-definitions AttributeName=LockID,AttributeType=S --key-schema AttributeName=LockID,KeyType=HASH --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5

```

**The s3-state-bucket.tf file**

```
#---------------------------------------------------------------
#
# BE SURE TO EDIT THE VALUES BELOW IF RE-CREATING THE BUCKET
# successful tf init contains this:
# $ tf init
#
# Initializing the backend...
#
# Successfully configured the backend "s3"! Terraform will automatically
# use this backend unless the backend configuration changes.
# Initializing modules...
#
#
# According to `terraform init` "Variables may not be used here."
terraform {
  backend "s3" {
    bucket         = "S3-uniquely-named-state-bucket-448"                        # <----<<< Replace with latest value of ${MY_BUCKET}
    key            = "terraform.tfstate"
    region         = "us-west-2"                                                 # <----<<< Replace with the region you want... not sure if variable works here
    encrypt        = true
    dynamodb_table = "dynamodb-S3-uniquely-named-state-bucket-448-locking-table" # <----<<< Replace with latest value of ${MY_BUCKET}

  }
}

```

# Answer
`A backend block cannot refer to named values (like input variables, locals, or data source attributes).` This is from the official documentation Terraform Backend. If you want to use variables in your backend block, you may want to consider using Terragrunt. We personally don't use it but we are considering adding it as there are multiple backends that we need to handle. My advice is don't use variables in the backend as it will not work. If you do not want to add another tool that compliments Terraform, just stick with adding the values to the backend directly.

You can also check this article.

Example:

```
resource "aws_s3_bucket" "my_tf_state_bucket" {
  bucket = var.state_bucket_name

   tags           = {
      Name              = "${var.my_project}-tf-state-bucket"
      project           = var.my_project
      environment       = var.my_environment
   }

}

terraform {
  backend "s3" {
    bucket         = s3_bucket_name
    key            = "terraform.tfstate"
    region         = "eu-west-1"
    encrypt        = true
    dynamodb_table = "terraform-locking"

  }
}

```
> 0 votes

---
Tags: amazon-web-services, terraform
---


# Question
Title: What is the difference between The Shared repository model and fork and pull model?
I am trying to understand what is the difference between the two. The Shared repository model: uses ‘topic’ branches, which are reviewed, approved, and merged into the main branch.

Fork and pull model: a repo is forked, changes are made, a pull request is created, the request is reviewed, and if approved, merged into the main branch.

Can someone explain with example?

# Answer
The most obvious difference is the permissions model. With forking, you don't have to grant someone access to your repository. In open source communities, this is a huge benefit. Within an organization, this becomes less of an issue.

With a branching model, keep in mind branches are still a part of your repository. Example, someone creates a branch, and pushes a large file to their branch, that can have performance impacts on your repo. If a secret is pushed to a branch, that is exposed in your repository.
> 2 votes

---
Tags: git, branch
---


# Question
Title: s3: reading parquet files with a subset of columns as argument; does this reduce data transfer?
I have set of large tables stored as parquets on s3. In python, I'm using:

```
pd.read_parquet(...,columns=columns)

```

I'm reading the files directly from s3, without any database engine whatsoever for preprocessing in between.

My question is, will the columns argument allow me to reduce data transfer to my remote dask cluster workers by just specifying a subset of the columns I'm interested in, or will they load the full parquet, at first, and then extract the columns? I suspect the latter is the case.

Looking for a solution, I found S3 select: https://docs.aws.amazon.com/AmazonS3/latest/userguide/selecting-content-from-objects.html

I think, I could use boto3 and sql syntax to read subsets of columns directly on s3, similar as done here: https://www.msp360.com/resources/blog/how-to-use-s3-select-feature-amazon/

But what I would really like to have is a version of pd.read\_parquet that does this in the background. What I found here is the awswrangler library: https://aws-sdk-pandas.readthedocs.io/en/stable/stubs/awswrangler.s3.read\_parquet.html

I suspect that awswrangler does exactly what I want, but I did not find an example that shows this. Does anybody know how it works?

Thanks!

# Answer
> 0 votes
Here's what I do using a custom defined `read_parquets`. It seems to reduce the data transfer size.

> ```
> df1 = read_parquets(s3_path)
>
> ```
>
> Time taken 4m 8.0s

> ```
> df2 = pd.read_parquet(s3_path, columns = ['col1'])
>
> ```
>
> Time taken 25.0s

> ```
> df3 = read_parquets(s3_path, columns = ['col1'])
>
> ```
>
> Time taken 15.2s

Supporting functions:

```
from functools import lru_cache
import os
import re

import boto3
import boto3
from botocore.errorfactory import ClientError
import json

import logging
_logs_file_ops = logging.getLogger(__name__)

s3 = boto3.client('s3')
s3_resource = boto3.resource('s3')

def list_distribute_into_blocks(input_list: list, num_lists: int = 0) -> list[list]:
    """Equally distributes an input list in to a list of specified number of lists

    Args:
        input_list (list): A python flat list
        block_size (int): Size of blocks
    """
    if num_lists == 0:
        return input_list
    try:
        list_length = len(input_list)
        sublist_size = list_length // num_lists  # Calculate the size of each sublist
        remaining_elements = list_length % num_lists  # Calculate the number of remaining elements

        sublists = []
        index = 0

        for i in range(num_lists):
            sublist = input_list[index: index + sublist_size]
            index += sublist_size
            # If there are remaining elements, distribute them across sublists
            if remaining_elements > 0:
                sublist.append(input_list[index])
                index += 1
                remaining_elements -= 1

            sublists.append(sublist)

        sublists = [_ for _ in sublists if len(_) > 0]
        return sublists
    except Exception as e:
        _logs_common_func.error(f'Distribution to list of {num_lists} lists failed due to: {e}')
        return input_list

def listdir(folder_path):
    """Returns a list of full paths to the contents of a specified location. Also accepts s3 locations
    """
    if 's3://' in folder_path:
        folder_path = folder_path.replace('\\', '/')
        # create S3 client
        # specify bucket and prefix (folder path)
        # bucket_name = 'boulevarddataplatform'
        bucket_name, prefix = _bucket_prefix(folder_path)
        if prefix[-1] != '/':
            prefix += '/'

        keys = []
        kwargs = dict(Bucket=bucket_name, Prefix=prefix, Delimiter='/')
        while True:
            resp = s3.list_objects_v2(**kwargs)
            files, folders = resp.get('Contents', []), resp.get('CommonPrefixes', [])
            for obj in files:
                keys.append(obj['Key'])
            for obj in folders:
                keys.append(obj['Prefix'])
            try:
                kwargs['ContinuationToken'] = resp['NextContinuationToken']
            except KeyError:
                break

        _logs_file_ops.info(f'{len(files)} files + {len(folders)} folders in {bucket_name}/{prefix}')
        return [f's3://{bucket_name}/{_}' for _ in keys]
    else:
        return [os.path.join(folder_path, _) for _ in os.listdir(folder_path)]

```

Function to read parquets in parallel. You can pass a s3 or local disk directory from which to read parquet files:

```
def read_parquets(
        path:str | list[str], columns:list[str] | None = None,
        threads:int=10, max_retries:int = 5, replace_npnan:bool = False
    ):
    """Reads the specified path (s3 or local) as a pandas dataframe

    Args:
        path (str | list[str]): A folder or a list of parquet files
        columns (list[str] | None, optional): Specify columns to be read from the parquet file
        threads (int, optional): Number of parallel threads reading. Do NOT exceed 10
        max_retries (int, optional): Max times to retry
        replace_npnan (bool, optional): Replace nan values?

    Returns:
        pandas.DataFrame: A pandas dataframe with all the parquet files concatenated
    """
    import pandas as pd
    import time
    import random
    import numpy as np
    from tqdm import tqdm
    from concurrent.futures import ThreadPoolExecutor
    from app.utils.common_functions import list_distribute_into_blocks
    import pyarrow.parquet as pq

    def _read_parquet_file(file_path, columns=None, retry_number = 0):
        """Read a single parquet file
        """
        if retry_number >= max_retries:
            _logs_file_ops.error(f'Failed for {file_path} after {retry_number} retries')
            return None
        try:
            parquet_file  = pq.ParquetFile(file_path)
            if parquet_file.metadata.num_rows == 0:
                return None
            if columns is not None:
                columns = [_ for _ in columns if _ in parquet_file.schema.names]
            return (parquet_file.read(columns=columns).to_pandas())
        except Exception as e:
            _logs_file_ops.warning(f'Retrying: {file_path}: {e}')
            # Full jitter wait
            temp = min(10, 2**retry_number)
            wait_time = temp/2 + random.uniform(0, temp)
            time.sleep(wait_time)
            # Retry
            return _read_parquet_file(file_path, columns, retry_number+1)

    def _read_list_of_parquet_files(file_list: list[str], columns=None):
        """Read a list of parquet files serially using `_read_parquet_file`
        """
        df = pd.concat([
                _read_parquet_file(file_path=f, columns=columns)
                for f in tqdm(file_list, desc = 'Reading...', leave = False)
            ])
        if replace_npnan is False:
            return df
        else:
            _logs_file_ops.warning(f'[!] Replace np.nan with {replace_npnan}')
            return df.replace(np.nan, replace_npnan)

    if isinstance(path, str):
        if '.parquet' in path:
            return pd.read_parquet(path=path, columns=columns)
        else:
            list_of_files = [_ for _ in listdir(path) if '.parquet' in _]
    elif isinstance(path, list):
        list_of_files = path

    if len(list_of_files) == 0:
        _logs_file_ops.warning(f'No .parquet files found in {path}')
        return None

    file_list_of_lists = list_distribute_into_blocks(list_of_files, threads)
    with ThreadPoolExecutor(max_workers=threads) as executor:
        futures = [
            executor.submit(_read_list_of_parquet_files, file_list, columns)
            for file_list in file_list_of_lists
        ]
        sublists = [future.result() for future in futures]

    return pd.concat(sublists)

```

---
Tags: amazon-web-services, python, amazon-s3, distributed-systems, s3
---


title: Run the latest image from ECR on EKS

text:
# Question
Title: Run the latest image from ECR on EKS
I have a CircleCI set-up to push docker images to ECR, similar to this question here: What is the idiomatic way to reload EKS/Kube configs when a new image is pushed to ECR

My question is two-fold:

1. I've read that the `latest` tag is an anti-pattern and should be avoided and instead simply use the git commit id or something like that to tag the image. How does ECR identify which is the latest image if it is not explicitly tagged `latest`?
2. How should I trigger EKS to pick the most recent image - have things changed since that last question was posted? Does it need the `latest` tag ?

# Answer
1. ECR is just a registry - so it only stores images - it does not identify latest images, nor provides any additional information beyond tags and sha256 image digests. So it is up to you to identify the sequence, common way is to use meaningful versioning system and tag with that. I.e., if you use SemVer, you can tag something like myimage:1.0.1, myimage:1.0.2 - which makes it pretty obvious what is newer.
2. Things have not changed in principle since the last question. Generally speaking, promotion of a newer image to a Kubernetes cluster is a separate process (CD) from creating and storing such image (CI). So you need some other trigger to make the promotion happen - you can automate this as well, and add such trigger directly at the end of the CI step if that is what you want for now - though note that usually there is more complex approval logic here in production settings.
> 1 votes

---
Tags: kubernetes, eks, aws-ecr, circleci
---


# Question
Title: Getting error while trying to communicate with apiserver in kube-state-metrics pod
I am getting error:

```
# kubectl logs  kube-state-metrics-fdc49658-g98sl -n monitoring
I0523 11:30:25.206556       1 main.go:86] Using default collectors
I0523 11:30:25.206766       1 main.go:98] Using all namespace
I0523 11:30:25.206800       1 main.go:139] metric white-blacklisting: blacklisting the following items:
W0523 11:30:25.206886       1 client_config.go:543] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0523 11:30:25.216265       1 main.go:184] Testing communication with server
F0523 11:30:55.217773       1 main.go:147] Failed to create client: error while trying to communicate with apiserver: Get "https://10.96.0.1:443/version?timeout=32s": dial tcp 10.96.0.1:443: i/o timeout

```

Here is the manifest for Kube-state-metrics:

```
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kube-state-metrics
spec:
  selector:
    matchLabels:
      app: kube-state-metrics
  replicas: 1
  template:
    metadata:
      labels:
        app: kube-state-metrics
    spec:
      nodeSelector:
        boardType: aarch64
      serviceAccountName: prometheus
      containers:
      - name: kube-state-metrics
        image: carlosedp/kube-state-metrics:v1.9.6
        ports:
        - containerPort: 8080
          name: monitoring
---
kind: Service
apiVersion: v1
metadata:
  name: kube-state-metrics
spec:
  selector:
    app: kube-state-metrics
  type: LoadBalancer
  ports:
  - protocol: TCP
    port: 8080
    targetPort: 8080

```

# Answer
This doesn't look like `kube-state-metrics` but your node configuration/networking issue. You will need to debug node to node communication. I would start by checking on which nodes the `kube-state-metrics` is scheduled and launch a temporary pod on that node to see if you can access the API Server pod correctly. The error could be in subnet, security groups/ firewalls, or other networking and security-related components.
> 1 votes

# Answer
The problem I encountered was that I had adjusted the secrets before the certificate was deleted.

```
 kubectl -n monitoring delete pods kube-state-metrics-xxxxx

```
> 0 votes

---
Tags: kubernetes, prometheus, metrics
---


# Question
Title: How to set correct mount permissions of Minikube grafana pod
I have a minikube cluster with a grafana pod running in it.

I have mounted a directory for grafana to use as a persistant volume to store. `minikube mount /var/tmp/minikube-folder1:/folder1`

I receive the output

```
  Mounting host path /var/tmp/minikube-folder1 into VM as /folder1 ...
    ▪ Mount type:   9p
    ▪ User ID:      docker
    ▪ Group ID:     docker
    ▪ Version:      9p2000.L
    ▪ Message Size: 262144
    ▪ Options:      map[]
    ▪ Bind Address: 127.0.0.1:64221
  Userspace file server: ufs starting
✅  Successfully mounted /var/tmp/minikube-folder1 to /folder1

```

i bind to the folder as such in my grafana yaml

```
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-grafana
spec:
  volumeName: grafana-pv
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: grafana-pv
spec:
  storageClassName: standard
  accessModes:
    - ReadWriteMany
  capacity:
    storage: 2Gi
  hostPath:
    path: /folder1

```

and eventually mount it on my container as such

```
   [spec..tempalate..spec above]

    containers:
      - name: mygrafana
        image: grafana/grafana
        ports:
          - containerPort: 3000
        volumeMounts:
          - name: grafana-storage
            mountPath: /var/lib/grafana

     volumes:
       - name: grafana-storage
         persistentVolumeClaim:
            claimName: pvc-grafana

```

my grafana pod cant launch and always complains the permissions arent correct on `/var/lib/grafana`

I can remedy it by manually going into `/var/tmp/minikube-folder1/` and `chmod 777`ing the folders each time a new one appears (ex /var/lib/grafana/alerting), but this seems like an inefficient and wrong strategy.

Is there someway to not have the minikube setup have these filesystem permission conflicts?

# Answer
> 0 votes
You have multiples solutions

### 1 - Set a uid and gid when creating the mount

This imply knowing the uid and gid of the user running inside your pod. To get it you could do something like this :

```
$ kubectl run -ti debug --image grafana/grafana --command bash
If you don't see a command prompt, try pressing enter.
debug:/usr/share/grafana$ id
uid=472(grafana) gid=0(root) groups=0(root)

```

Then when creating the mount with minikube, set the uid and gid

```
$ minikube mount /var/tmp/minikube-folder1:/folder1 --uid 472 --gid 0

```

Now in your pod you can see that the `/var/lib/grafana/` permissions are correct

```
grafana-57d4899dd7-fsp5z:/usr/share/grafana$ id
uid=472(grafana) gid=0(root) groups=0(root),47
grafana-57d4899dd7-fsp5z:/usr/share/grafana$ ls -lnd /var/lib/grafana/
drwxrwxr-x    1 472      0             4096 Aug 11 14:19 /var/lib/grafana/

```

### 2 - Use a security context to run the pod

Localy your folder is created with your user, you can tell your pod to use the same uid as your local user.

First you need to get the uid of the user who created `/var/tmp/minikube-folder1`

```
$ ls -lnd /var/tmp/minikube-folder1
total 4
drwxrwxr-x 2 1000 1000 4096 août  11 16:06 /var/tmp/minikube-folder1

```

Let say the uid is 1000.

In your deployment configuration (grafana yaml), you can set the `runAsUser` directive :

```
...
spec:
  template:
    spec:
      securityContext:
        runAsUser: 1000
      containers:
...

```

Now if you check in your started pod you can see that it's started with uid 1000

```
$ kc exec -ti grafana-7c895564cc-w26p6 -- bash
grafana-7c895564cc-w26p6:/usr/share/grafana$ id
uid=1000 gid=0(root) groups=0(root),472

```

And that's the same user as the `/var/lib/grafana/` folder in the pod :

```
grafana-5f75d8ff65-dhbb5:/usr/share/grafana$ ls -lnd /var/lib/grafana/
drwxrwxr-x    1 1000     999           4096 Aug 11 14:16 /var/lib/grafana/

```

Hope this is clear and that it helps !

---
Tags: kubernetes, grafana, persistent-volumes
---


# Question
Title: Azure Devops: How to order query by custom backlog order?
In Azure DevOps you can custom order items in a field called *Order*. How do you order a Query by this order? What is the *Order* field name?

# Answer
> 0 votes
Order in this context is just a line count.

You can drag items to different lines and the 'Order' will stay the same.

---
Tags: azure-devops
---


# Question
Title: How to specify ANY for tags in the yaml file for pipelines
In my pipeline definition, for getting resources from other pipeline, I'd like to define a default tags value to use, such as "release candidate", but allow to select to use any tags or even without tag. Is there a simple way to do so?

Here's my attempt but seems like "\*" doesn't work:(

```
parameters:
- name: TagsToUse
  displayName: 'Select the tags to use for artifact from pipeline xxxx'
  type: string
  default: 'release candidate'
  values:
  - 'release candidate'
  - '*'

resources:
  pipelines:
  - pipeline: xxxx
    project: 'MyProject'
    source: 'TargetSource'
    branch: 'master'
    tags: ${{ parameters.TagsToUse }}
...
...
...

```

# Answer
Using `values:` is a way to restrict what can be passed into the parameters. For this reason, there isn't a way to wildcard an input value.

Since what you want to do is allow anything but default to 'release candidate' if nothing is provided, you really only need `default:` and can remove the `values:` restriction.
> 0 votes

---
Tags: azure-devops, azure-pipelines, build-pipeline, yaml, pipeline
---


# Question
Title: Azure Devops pipeline not listing artifact created by it
I am building and pushing Docker images to my Azure Container Registry from Azure DevOps using the `azure-pipelines.yml` file below.

This works great **except** that there doesn't seem to be any relation created between the created image and the pipeline that created it. There is no artifacts listed on the executed pipeline:

The artifact is created correctly such that I can use it when running a release pipeline and if I inspect the image locally I can confirm that it has the labels on it created by the Docker task as described here.

```
# Docker
# Build and push an image to Azure Container Registry
# https://docs.microsoft.com/azure/devops/pipelines/languages/docker

trigger:
- none

resources:
- repo: self

variables:
  # Container registry service connection established during pipeline creation
  dockerRegistryServiceConnection: '<my connection>'
  imageRepository: '<my repo>'
  containerRegistry: '<my registry>'
  dockerfilePath: '$(Build.SourcesDirectory)/Dockerfile'
  tag: '$(Build.BuildNumber)'
  DOCKER_BUILDKIT: 1

  # Agent VM image name
  vmImageName: 'ubuntu-latest'

stages:
- stage: Build
  displayName: Build and push stage
  jobs:
  - job: Build
    displayName: Build
    pool:
      vmImage: $(vmImageName)
    steps:
      - task: Docker@2
        displayName: Build
        inputs:
          command: build
          repository: $(imageRepository)
          dockerfile: $(dockerfilePath)
          arguments: '--target prod'
          containerRegistry: $(dockerRegistryServiceConnection)
          tags: |
            latest
            $(tag)
      - task: Docker@2
        displayName: Push
        inputs:
          command: push
          repository: $(imageRepository)
          dockerfile: $(dockerfilePath)
          containerRegistry: $(dockerRegistryServiceConnection)
          tags: |
            latest
            $(tag)

```

# Answer
You should not see that. Artifact is something published from PublishBuildArtifacts@1 task. In your case, you publish to Docker Container Registry (not to azure build artifacts).

All other items (like tags and new image) you can find through docker cli.
> 0 votes

# Answer
As my experience, Artifact of the Build pipeline in a Docker solution are normally a Docker Compose file, or manifest file... that contains information about the images have been built. Then, in the Release pipeline we can use these files for deploying corresponding images to the server. You have to pusblish these files as an Artifact by yourself.
> 0 votes

# Answer
Shamrai is correct that the build and push tasks for Docker don't publish an artifact to the pipeline and instead are pushing that artifact to ACR.

If you want to see the artifact in the pipeline you will have to use the PublishBuildArtifacts task.

However, if you only want a link between the build and the image and do not want to store a second copy in the build, you could tag the image with the build id.

```
- task: Docker@2
        displayName: Push
        inputs:
          command: push
          repository: $(imageRepository)
          dockerfile: $(dockerfilePath)
          containerRegistry: $(dockerRegistryServiceConnection)
          tags: |
            latest
            $(tag)
            $(Build.BuildId)

```
> 0 votes

---
Tags: docker, azure-devops, docker-registry, artifacts
---


# Question
Title: Azure DevOps - How Do I Change the Service Connection for a Build Pipeline
I have a build pipeline already configured using a .yml file. I transferred the repository over to a different organization on GitHub, added all of the necessary Azure Pipelines integrations and permissions in GitHub, and added the service connection to Azure Devops.

Now, when I go to run my pipeline I get the error: `Service connection {id} not found in the project {projectId}`

This is true. I don't want it to use the old service connection. I want to tell Azure DevOps to use the new one. How do I change this?

The only workaround I can find is to create a new pipeline. Fortunately, this links right up with GitHub and I don't have to rewrite the yaml. Unfortunately, I'd have to plugin all the pipeline variables again.

# Answer
> 0 votes
You can click on `Edit`. The UI will give the notification that the connection is not available and only classic editor is available. The connection can be changed to a different one in the classic editor.

# Answer
> 0 votes
You can change out the service connections for your DevOps project by navigating to the `Project Settings`

and then selecting `Service Connections`

# Answer
> 0 votes
I am assuming this is also a new organization/project in Azure DevOps.

In that case, you will have to create that service connection in Azure DevOps.

If you do not want to edit the yaml, you will need to find the name of the service connection being used in the pipeline and create the service connection with that exact same name.

If you don't check the box to 'Grant access permission to all pipelines' when creating the service connection, then you will also get a pop-up on the first pipeline run to 'Authorize Resources' before the pipeline will actually run.

---
Tags: azure-devops
---


# Question
Title: New install of helm shows traefik and traefik-crd, but why?
When I run `helm ls -A` on a new install of Helm, I see,

```
$ helm ls -A
NAME        NAMESPACE   REVISION    UPDATED                                 STATUS      CHART                       APP VERSION
traefik     kube-system 2           2023-08-12 20:08:51.802846904 +0000 UTC deployed    traefik-21.2.1+up21.2.0     v2.9.10
traefik-crd kube-system 2           2023-08-12 20:08:51.764773882 +0000 UTC deployed    traefik-crd-21.2.1+up21.2.0 v2.9.10

```

Why is it that Traefik is installed? Is this a dependency of helm?

# Answer
# k3s

This is actually a default of k3s which installs traefik by default and flags the install as Helm-managed.
> 0 votes

---
Tags: helm, k3s, traefik
---


# Question
Title: Can the K3s installed Traefik be updated independently through helm?
If k3s installs traefik and flags it as a helm chart, can it be updated through helm? When I run the commands below,

```
helm repo add traefik https://traefik.github.io/charts
helm repo update
helm upgrade traefik traefik/traefik -n kube-system

```

I get the following

> Error: UPGRADE FAILED: resource mapping not found for name: "traefik-dashboard" namespace: "kube-system" from "": no matches for kind "IngressRoute" in version "traefik.io/v1alpha1" ensure CRDs are installed first

# Answer
> 0 votes
## Install new CRDs and RBAC for Kube v1.16+

Yes, but because of the CRDs in Helm are managed by Rancher and not Traefik, and they were changed upstream,

> The `apiextensions.k8s.io/v1beta1` CustomResourceDefinition is deprecated in Kubernetes v1.16+ and will be removed in v1.22+. For Kubernetes v1.16+, please use the Traefik `apiextensions.k8s.io/v1` CRDs instead.

You must install the new CRDs and RBAC's.

```
# Install Traefik Resource Definitions:
kubectl apply -f https://raw.githubusercontent.com/traefik/traefik/v2.10/docs/content/reference/dynamic-configuration/kubernetes-crd-definition-v1.yml

# Install RBAC for Traefik:
kubectl apply -f https://raw.githubusercontent.com/traefik/traefik/v2.10/docs/content/reference/dynamic-configuration/kubernetes-crd-rbac.yml

```

## Upgrade Traefik

Then you can install the Traefik repos and upgrade Traefik,

```
helm repo add traefik https://traefik.github.io/charts
helm repo update
helm upgrade traefik traefik/traefik -n kube-system

```

Now you should see that the upgrade was complete,

```
❯ helm history traefik -n kube-system
REVISION    UPDATED                     STATUS      CHART                   APP VERSION DESCRIPTION
1           Sun Aug 13 18:45:21 2023    superseded  traefik-21.2.1+up21.2.0 v2.9.10     Install complete
2           Sun Aug 13 13:48:07 2023    deployed    traefik-24.0.0          v2.10.4     Upgrade complete

```

---

## Remove deprecated CRDs

Now you can optionally remove the `traefik-crd` that Rancher provides, but because the crd's are versioned, you can just keep it too.

```
helm delete -n kube-system traefik-crd

```

---
Tags: k3s, traefik
---


# Question
Title: Can't get Python with cx_oracle to connect outbound to Oracle server
When I inherently go to make the connection from the python app inside the docker file I get .. I can ping and nslookup from indide the docker container from the terminal....so I'm a bit confused..

> 2023-08-12 01:30:45.665974(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST='host.host.host')(PORT=1521))(CONNECT\_DATA=(SERVICE\_NAME='ADMTST')))DPY-6005: cannot connect to database. Connection failed with "\[Errno -2\] Name does not resolve"

Docker File:

```
FROM python:3-alpine

WORKDIR /app

COPY requirements.txt .
COPY package.json .
RUN apk add openldap-dev
RUN apk update && apk add python3-dev \
                        gcc \
                        libc-dev \
                        npm

RUN pip install --upgrade pip
RUN pip install --upgrade setuptools
RUN python3 -m pip install -U "Cython<3.0.0"
RUN python3 -m pip install --no-build-isolation "oracledb==1.2.2"

RUN pip install -r requirements.txt

RUN npm install

COPY . .
RUN mv node_modules ./static/
EXPOSE 8000
CMD ["gunicorn","--access-logfile","LOGS/access_logs","--error-logfile","LOGS/error_logs","--certfile","tempSSL/certificate.pem","--keyfile","tempSSL/server.key","--workers","4", "-b", "0.0.0.0:8000", "wsgi:app"]

```

Docker Compose

```
version: '3.8'
services:
  app:
    build: .
    ports:
      - 8000:8000
      - 1521:1521

```

# Answer
> 0 votes
Your problem in this case is very clear..

> 2023-08-12 01:30:45.665974(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(**HOST='host.host.host'**)(PORT=1521))(CONNECT\_DATA=(SERVICE\_NAME='ADMTST')))DPY-6005: cannot connect to database. Connection failed with **"\[Errno -2\] Name does not resolve"**

Whatever that code is being called (in python), you're trying to connect to a database called `host.host.host`. That means under the hood something is trying to resolve `host.host.host` to an ip address. There is nothing on your network or in your hosts file that can resolve `host.host.host`.

---
Tags: docker, python, network
---


# Question
Title: Is there an nginx that isn't an ingress controller?
I'm using traefik as an ingress controller. I'm looking to route traffic to a webserver. Is there a helm chart that packages up nginx not as an ingress controller but as a webserver?

# Answer
Bitnami packages software to helm charts. One of them is Nginx web server.

Bitnami Nginx Helm Chart from Artifacthub.io or you can pull it from Bitnami Official Bitnami Website A quick reference on the Bitnami site to get start

```
helm install my-release oci://registry-1.docker.io/bitnamicharts/nginx
or
helm pull oci://registry-1.docker.io/bitnamicharts/nginx

```

A cool thing about Bitnami is that nginx is running in non privilege port. Happy Helming !!
> 2 votes

---
Tags: kubernetes, helm, nginx
---


# Question
Title: How to use facts from one host to perform actions on another host?
In Ansible I'm trying to generate a variable from a task on one host and then use the variable to do something to another host. Specifically, I'm trying to create a Kubernetes cluster - On the control plane node I can save the join command into a variable, but this appears to be scoped to the source host only - I can't work out how to access that variable from another host.

```
- name: read join command from the controlPlane
  shell: kubeadm token create --print-join-command
  when: inventory_hostname in groups['controlPlane']
  register: joinCmd

- name: join worker nodes to cluster
  command: "{{ joinCmd.stdout }}"
  when: inventory_hostname in groups['workerNodes']

```

The above fails on the second task (*The task includes an option with an undefined variable. the error was: 'dict object' has no attribute 'stdout'*). The standard approach seems to be save the line to a file locally, copy the file to the worker nodes and execute it locally which seems cumbersome (and leaves the join token lying around).

How can I achieve this with variables only?

# Answer
> 2 votes
I found a solution - the trick here is *delegate\_to* and *delegate\_facts*:

```
- hosts: all
  remote_user: ansible
  become: false

  tasks:
  - name: read join line from controlPlane
    shell: kubeadm token create --print-join-command
    when: inventory_hostname in groups['workerNodes']
    register: joinCmd
    delegate_to: "{{ item }}"
    delegate_facts: true
    with_items: "{{ groups['controlPlane'] }}"

  - name: join all worker nodes to cluster
    command: "{{ joinCmd.results[0].stdout }}"
    become: yes
    when: inventory_hostname in groups['workerNodes']

```

The first task kind of reads backwards to my mind - *when* says 'run this task on all the workerNode hosts...' but *delegate\_to* says '..but actually run it on a different host'. It works for me though (I've only got 1 host in controlPlane group, not sure of outcome with more hosts).

# Answer
> 1 votes
Another option here would be to run tasks on the master node(s) to create the join command and just run the role for nodes joining based on the same conditions:

```
- hosts: cluster_masters,cluster_workers
  pre_tasks:
    - name: Create the token to join
      ansible.builtin.shell: |
        kubeadm token create --print-join-command
      register: join_command
      when: inventory_hostname in groups.cluster_masters
    - name: Set token as fact
      ansible.builtin.set_fact:
        join_command: "{{ join_command.stdout }}"
      run_once: true
      when: inventory_hostname in groups.cluster_masters
  roles:
    - role: applications/kubernetes/node
      when: inventory_hostname in groups.cluster_workers and join_command is defined

```

The join\_command is now availabe in the role

---
Tags: kubernetes, ansible
---


# Question
Title: How To Convert Jenkins String Parameter To Terraform Map Variable? (Invalid number literal)
Edit (8/16/2023): I figured it out!

I'm trying to pass string parameter(s) to a Terraform map variable, but receiving error "Invalid number literal".

Jenkinsfile:

```
pipeline {
    agent any

    parameters {
        string(name: 'IP1', defaultValue: '', description: 'Enter first IP address')
    }

    stages {
        stage('Git Checkout') {
            steps {
                git branch: 'branch1', credentialsId: '', url: 'http://<redacted>.git'
            }
        }
        stage('Deploy Terraform') {
            steps {
                script {
                        dir('Linux') {
                                sh """
                                terraform init
                                terraform plan
                                terraform apply -var 'vms=${params.IP1}' \
                                --auto-approve
                                """
                            }
                    }
                }
            }
        }
    }
}

```

variables.tf

```
variable "vm_map" {
  type = map(object({
    name = string
    ip   = string
  }))
  default = {
    "first" = {
      name = "ubuntu-jenkins1"
      ip   = "172.30.100.160"
    }
    "second" = {
      name = "ubuntu-jenkins2"
      ip   = "172.30.100.161"
    }
    "third" = {
      name = "ubuntu-jenkins3"
      ip   = "172.30.100.162"
    }
  }
}

```

# Answer
I figured it out! You can substitute 'vm1' for any value that identifies the first map object.

```
terraform apply -var vms='''{vm1: {name: "ubuntu",ip: "${params.IP1}"}}''' --auto-approve

```

If you also wanted to add a name string parameter for the vm name, it would look like

```
terraform apply -var vms='''{vm1: {name: "${params.VM_NAME1}",ip: "${params.IP1}"}}''' --auto-approve

```

I will leave this in case someone else runs into this problem.
> 1 votes

# Answer
Here's the terraform apply for multiple vm's

```
terraform apply -var 'vm_map={"first": {"name": "ubuntu-jenkins1", "ip": "172.30.100.160"}, \
                                                          "second": {"name": "ubuntu-jenkins2", "ip": "172.30.100.161"}, \
                                                          "third": {"name": "ubuntu-jenkins3", "ip": "172.30.100.162"}}' --auto-approve

```
> 0 votes

---
Tags: jenkins, terraform, jenkins-pipeline, jenkinsfile, infrastructure-as-code
---


# Question
Title: Monitoring tool to microservices on Kubernetes
I'm deploying an instance with kubernetes on Azure public cloud, and I need a tool to monitor the microservices and pods inside this instance, the question is, what tool are you using to do that?
I've been searching in a couple of places but without success.

# Answer
AKS

https://github.com/MicrosoftDocs/azure-docs/blob/master/articles/aks/tutorial-kubernetes-monitor.md

Looks interesting. I have not tried it myself, but based on the more than 300 github stars this looks interesting to me. The readme contains screen shots how to configure it in Azure.
> 1 votes

# Answer
There is a basic dashboard which give some informations on your cluster. You can monitor basics metrics and this is the entry-level for Kubernetes monitoring.

https://github.com/kubernetes/dashboard

If you want something more complete, you could use the stack: Prometheus with CAdvisor/node-exporter and Grafana
> 1 votes

# Answer
monitoring resources (pods, node, storages) is importance. There are time we have issue with kubernetes pod which is in Crash result in service interupption.

IMHO, there are two thing need to take priority: Metrics and Logging. We can use: **Elastic Stack (ELK):** Discover how Elasticsearch, Kibana, Beats, and Logstash lay the foundation for various use cases, starting with logging and stretching the boundaries of imagination.

**Prometheus:** Uncover the distinguishing features of Prometheus, including its multidimensional data model, versatile query language (PromQL), and real-time alerting capabilities.

**Grafana:** Explore the dynamic world of Grafana, a versatile platform that excels in data visualization, monitoring, and crafting insightful dashboards.

**Kube-state-metrics:** Understand how this tool simplifies the utilization of Kubernetes API metrics to monitor resource health and performance.

**Datadog:** Delve into Datadog's robust integration of traces, metrics, and logs, revolutionizing application and infrastructure observability.

**Sumo Logic:** Learn about Sumo Logic's cutting-edge machine data analytics tailored for cloud-native environments. I have documented an article about 7 Top Kubernetes Monitoring Tools hope can help
> 0 votes

---
Tags: kubernetes, monitoring, microservices
---


# Question
Title: docker: homedir does not belong to the user
I created a Dockerfile which creates a new user. When I run this Dockerfile I see that the user is created, his homedir is created as well, but the homedir is owned by root - and even "chown" can't change it.

What's wrong here? What should I do to make the homedir be owned by the user?

This is my Dockerfile:

```
FROM debian

RUN echo "START"
RUN whoami
RUN groupadd -g 1023 bolek
RUN useradd -u 1023 -m -g bolek -s /bin/bash bolek
RUN ls -ld /home/bolek
RUN chown bolek:bolek /home/bolek
RUN ls -ld /home/bolek

```

And this is the result of running it:

```
$ docker build .
Sending build context to Docker daemon  2.048kB
Step 1/8 : FROM debian
 ---> 278727474c29
Step 2/8 : RUN echo "START"
 ---> Running in ea2ef07f8cf7
START
Removing intermediate container ea2ef07f8cf7
 ---> 380cf07ef790
Step 3/8 : RUN whoami
 ---> Running in 310b27eb1667
root
Removing intermediate container 310b27eb1667
 ---> f2d3e944ba74
Step 4/8 : RUN groupadd -g 1023 bolek
 ---> Running in 7a6ed0dfa95f
Removing intermediate container 7a6ed0dfa95f
 ---> 00edbb567c42
Step 5/8 : RUN useradd -u 1023 -m -g bolek -s /bin/bash bolek
 ---> Running in 29df766891ca
Removing intermediate container 29df766891ca
 ---> a16b10eb3ed7
Step 6/8 : RUN ls -ld /home/bolek
 ---> Running in c13578f95134
drwxr-xr-x 2 root root 4096 Aug 13 05:42 /home/bolek
Removing intermediate container c13578f95134
 ---> 69dbc583fd7a
Step 7/8 : RUN chown bolek:bolek /home/bolek
 ---> Running in a0cd64572d84
Removing intermediate container a0cd64572d84
 ---> c3d50f8cdf97
Step 8/8 : RUN ls -ld /home/bolek
 ---> Running in 1cd57cf5f75b
drwxr-xr-x 1 root root 4096 Aug 13 05:42 /home/bolek
Removing intermediate container 1cd57cf5f75b
 ---> ceb5c78c2786
Successfully built ceb5c78c2786

```

I noticed that the problem happens only on some computers - on other ones the very same Dockerfile behaves correctly.

I could reproduce the problem on Linux Mint 20.3, where docker version is 20.10.25 (build 20.10.25-0ubuntu1~20.04.1) and docker's server version is also 20.10.25.

# Answer
Try this, instead (have not tested this):

```
FROM debian

RUN echo "START" \
&& whoami \
&& groupadd -g 1023 bolek \
&& useradd -u 1023 -m -g bolek -s /bin/bash bolek \
&& ls -ld /home/bolek \
&& chown bolek:bolek /home/bolek \
&& ls -ld /home/bolek

```
> 0 votes

---
Tags: docker
---


# Question
Title: Processes in "podman build" have lower file descriptor limit than processes in "podman run", how can I increase this?
I'm running podman 3.4.4 on Ubuntu 22.04 LTS server.

I have been having `podman build` issues that are not reproducible if I do those steps after the container build using `podman run`.

Eventually I traced it to this: the limits given to processes in `podman build` are completely different. Here's a sample Dockerfile:

```
FROM node:14-bullseye
WORKDIR /app
RUN echo "clear cache 1"
RUN bash -c 'cat /proc/$$/limits'
# Placeholder, not used in this example
CMD bash

```

If I run:

```
podman build -t project .

```

This prints (among other things):

```
Max open files            1024                 1024                 files

```

Gee, that's low.

If I become root in my shell and use `ulimit -n 131072`, then `cat /proc/self/limits` shows my increased limit in the shell. But if I then bump "Clear cache 1" to "Clear cache 2" in the Dockerfile (**this is important**) and run the build command, still in that root shell with the increased ulimit, I still get:

```
Max open files            1024                 1024                 files

```

So that's what I see in `podman build`. However if I then open a bash shell in a container using the new image using `podman run`, I get this result:

```
root@ubuntu:/home/ubuntu/opt/cloud/build# podman run -it project bash
root@17961323a1c9:/app# cat /proc/$$/limits
Max open files            1048576              1048576              files

```

The command has completely different results at build time and at "run" time.

I did try to modify `ulimit` inside a RUN statement, but no dice:

```
bash: line 1: ulimit: open files: cannot modify limit: Operation not permitted
Error: error building at STEP "RUN bash -c 'ulimit -n 131072 && cat /proc/$$/lim

```

Anyway it would be much better to increase this for the build command in general, or at least using an option to it, rather than for every RUN that might need it.

Practically speaking this appears to be behind a build failure I have with a larger Dockerfile, but I've boiled it down to this straightforward test case.

I have also tried using `docker` (real Docker Engine). The problem does not happen with Docker Engine, I get 1048576 files there during build.

How can I increase the file descriptor limit at build time?

Thank you!

**Update:** I'm told there is a `--ulimit` option to podman, and indeed there is. But podman 3.4.4, which is standard on Ubuntu 22.04, does not support the simple `--ulimit=host` version. I am trying to figure out what a valid option value looks like. I have tried:

```
--ulimit=n=131072:1048576
--ulimit=NOFILES=131072:1048576

```

Both throw errors of the form:

```
requires name=SOFT:HARD, failed to be parsed: invalid ulimit type: n

```

# Answer
> 2 votes
The following command works, cobbled together from friendly advice and additional googling based on that input:

```
podman build --ulimit=nofile=131072:1048576 [other arguments here]

```

The latest releases of `podman` apparently accept `--ulimit=host`, which is convenient if you want the same policy, but 3.4.4 does not. For 3.4.4, you'll need to know the specific limit name. These are not yet listed in podman documentation, but for the file descriptor limit the proper name is `nofiles`. This Red Hat article may be helpful with regard to other limit names.

---
Tags: containers, podman
---


# Question
Title: QA instability due to lack of stable UAT/staging environment
Our testing teams are facing extensive downtime due to testing in a shared QA environment.

When our front end teams perform end to end testing, their test (QA) environment points to the back end services QA environment, which in turn points to enterprise and mainframe service QA environments.

Because these downstream QA environments are themselves constantly deployed to and tested, the result is frequent instability for front end testers. Also, the front end and service code they are testing is not being tested against the same mainframe and enterprise services that they will interface with in production.

What is the solution here? Should each teams QA environment interface with a more stable clone of what is in production? Or perhaps a UAT environment?

# Answer
> 1 votes
If you need to manually test the whole system, there needs to be a stable environment for that. It should reflect the production environment, with the only difference being the component that is about to be put live.

For example, if production has the following:

* UI 1.0
* Commerce 1.0
* Tax 1.0
* Shipping 1.0

And the plan is to launch a new version of the Tax service... you should have a stable environment to test this:

* UI 1.0
* Commerce 1.0
* Tax **1.1**
* Shipping 1.0

The general idea is that it should look exactly how production will look after the deployment.

There are other ways to achieve this, but this sounds like the match for your context.

---
Tags: jenkins, continuous-integration, testing, software-engineering, mainframe
---


# Question
Title: How many deployment environments are appropriate for SPA + API architecture?
My company uses Azure as our cloud-infrastructure provider, and we are making use of their Azure Webapp PaaS offering to individually deploy our SPA (react.js) and corresponding API (Golang). Haven't felt the need to move to Kubernetes as of yet.

For CI/CD we utilize the GitHub Actions + Azure Web App Deployment slots - a relatively simple setup. Currently both our API and React SPA have three deployment slots in their Web App resource called:

1. Production (the web app itself)
2. Main
3. Stage

Merging on **stage branch** leads to deployment on **stage slot**, and merging on **main branch** leads to a deployment on **main slot**. When given the go-ahead the Main slot is swapped with the Production one and so now we have code from **main branch** in the **production slot** (environment). Swapping allows for zero downtime of production app and avoids directly deploying to the production slot.

For the react spa, all env variables need to be specified at build time; these includes: **api\_uri, auth\_system\_uris, etc**. This means when merging on the main branch we need to specify all env variables for production slot (not really the main slot). I understand that there shouldn't be the slightest bit of difference between these two deployments but still am confused about the overall architecture.

Do I even need three deployments for my React SPA, if so, how do I make it work? **Should main slot of my spa communicate with main slot of api** (but that wouldn't work due to the necessity of keeping things same between prod and main slots) ? It would be great help if someone could briefly describe how this architecture should best be set up. Thanks.

# Answer
My first thought when I see two components that depend on each other during a deployment - often the app and the database - I try to come up with a way to decouple them.

If you can deploy your UI and API independently, you'll avoid many problems that are caused when they don't match.

You can use similar patterns to update the API in ways that mean the UI can work with either the old or new version (for example the expand/contract pattern), or you can deploy versioned APIs so the UI always calls the version you trust to work (for example `/api/v1/operation`).

The one thing you don't want to rely on is the UI and API going live at the exact same nanosecond.

With this in mind, you UI and API should have zero knowledge of which slot to talk to. You don't want to manipulate the production slot to talk to the main slot or anything similar to that.

You can set up the configuration against the slot, to prevent having to reconfigure config files in version control.
> 1 votes

---
Tags: azure, deployment, github-actions, environments, environment-variables
---


# Question
Title: Chown inside docker container is not changing directory owner
I can't change ownership of newly created directory inside of docker container. It is not a volume and `chown` runs correctly for all my colleagues - the only difference is I'm running docker on linux while they are working on macs and windows (what should not matter, right?). My docker version: `Docker version 20.10.25, build 20.10.25-0ubuntu1~22.04.1`

I have simplified my use case to this most basic Dockerfile (php version doesn't matter, tried with 8.0 as well):

```
FROM php:7.1.33-fpm AS base
RUN mkdir /home/www-data
RUN ls -ld /home/*
RUN usermod -d /home/www-data www-data
RUN ls -ld /home/*
RUN chown -R www-data /home/www-data
RUN ls -ld /home/*
RUN chmod -R 0777 /home/www-data
RUN ls -ld /home/*

```

I run it with `docker build . --no-cache`, so there is no docker-compose used, anything cached etc.

My partial output:

```
Step 5/9 : RUN ls -ld /home/*
 ---> Running in faee10891512
drwxr-xr-x 2 root root 4096 Aug 17 08:46 /home/www-data
Removing intermediate container faee10891512
 ---> bd417136c193
Step 6/9 : RUN chown -R www-data /home/www-data
 ---> Running in 7656908de5e0
Removing intermediate container 7656908de5e0
 ---> e25a42b7f73b
Step 7/9 : RUN ls -ld /home/*
 ---> Running in e916fca31c51
drwxr-xr-x 1 root root 4096 Aug 17 08:46 /home/www-data
Removing intermediate container e916fca31c51
 ---> be3154bf3ad6

```

Expected output (just taken from docker run on windows):

```
#8 [5/9] RUN ls -ld /home/*
#8 sha256:ab73f23a9843a55ca731c83532c9f1b95e738525c2a8437f58ed7c17d0cf245e
#8 0.457 drwxr-xr-x 2 root root 4096 Aug 17 08:40 /home/www-data
#8 DONE 0.5s

#9 [6/9] RUN chown -R www-data /home/www-data
#9 sha256:686bd64c892edc235610feb148f1a9213bdf9238e62e74f1ca87da416c97eb99
#9 DONE 0.5s

#10 [7/9] RUN ls -ld /home/*
#10 sha256:e37fbdcf3bd0ae93dd19f8bc4b81d3c22fec176a3ef6d9b957c9c46787cb79a0
#10 0.434 drwxr-xr-x 1 www-data root 4096 Aug 17 08:40 /home/www-data
#10 DONE 0.4s

```

I am really puzzled about this, anyone has and idea what should I try next?

# Answer
I have solved this problem by removing docker completely and installing it as Docker Desktop from .deb package provided from . After that, chown started to work as it should.
> 0 votes

# Answer
Maybe this, instead (have not tested this):

```
FROM php:7.1.33-fpm AS base
RUN mkdir /home/www-data \
&& ls -ld /home/* \
&& usermod -d /home/www-data www-data \
&& ls -ld /home/* \
&& chown -R www-data /home/www-data \
&& ls -ld /home/* \
&& chmod -R 0777 /home/www-data \
&& ls -ld /home/*

```
> 0 votes

---
Tags: docker, linux
---


# Question
Title: Kubernetes ingress fails to fetch letsencrypt
I using a root server at `v2202111160479168516.hotsrv.de`. I set up a microk8s install and enabled the plugins:

```
    cert-manager         # (core) Cloud native certificate management
    dashboard            # (core) The Kubernetes dashboard
    dns                  # (core) CoreDNS
    ha-cluster           # (core) Configure high availability on the current node
    helm                 # (core) Helm - the package manager for Kubernetes
    helm3                # (core) Helm 3 - the package manager for Kubernetes
    host-access          # (core) Allow Pods connecting to Host services smoothly
    ingress              # (core) Ingress controller for external access
    metrics-server       # (core) K8s Metrics Server for API access to service metrics

```

Then I tried to create the ingress for my dashboard, but it doesn't seem to be working. When I try to go to the /.well-known/acme-challenge/ the response is an nginx 404. I already tried changing the endpoint of the dashboard to /dashboard, but it doesn't seem to work. As far I can see in the logs, everything else is working, I do get to the dashboard with a certificate complaint.

```
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt
  namespace: kube-system
spec:
  acme:
    # You must replace this email address with your own.
    # Let's Encrypt will use this to contact you about expiring
    # certificates, and issues related to your account.
    email: coen.hacking@gmail.com
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      # Secret resource that will be used to store the account's private key.
      name: letsencrypt-account-key
    # Add a single challenge solver, HTTP01 using nginx
    solvers:
      - http01:
          ingress:
            class: nginx
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    className: nginx
    cert-manager.io/cluster-issuer: letsencrypt
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
  name: kubernetes-dashboard
  namespace: kube-system
spec:
  rules:
    - host: v2202111160479168516.hotsrv.de
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: kubernetes-dashboard
                port:
                  number: 443
  tls:
    - hosts:
        - v2202111160479168516.hotsrv.de
      secretName: dashboard-ingress-cert

```

I'm just getting into Kubernetes, so I'm probably missing something. Does anybody know what the mistake in my configuration is?

# Answer
After quite some time, I found the issue. the `1.28` branch of `microk8s` appeared to have be the culprit. When I switched to `1.27.4` it was all solved.
> 0 votes

---
Tags: kubernetes, ingress-nginx
---


# Question
Title: How to set context to be default in kubectl?
When I do :

```
$ kubectl config get-contexts

CURRENT   NAME                          CLUSTER                                                            AUTHINFO                                                           NAMESPACE
          kubernetes-admin@kubernetes   kubernetes                                                         kubernetes-admin
*         qa-1                        arn:aws:eks:xxxxxx150:cluster/ocean-qa   xxx:xxx:cluster/xxxx-RH36mmA4   xx-xx-tulw

```

I have multiple contexts defined in the config file. I would like to set 'kubernetes-admin@kubernetes' cluster as the default each time I use kubectl.

I can't find the command ..

# Answer
> 0 votes
`kubectl config use-context kubernetes-admin@kubernetes` should be the command you are lookin for.

To check if it worked, use `kubectl config current-context`.

If in doubt, check its help with `kubectl config set-context --help`.

---
Tags: kubernetes, kubectl
---


# Question
Title: Testing for EL with Ansible facts; distribution or os_family?
Sometimes in an Ansible playbook, you'll want to condition running a task on whether or not the distribution is CentOS, RHEL, etc, or if it's something else. A very common example is when a package is named one thing on CentOS and something else on Debian.

I see a lot of Ansible playbooks (and have written a few myself) that will handle this by adding a "when:" to the task like this:

`when: ansible_distribution == 'CentOS' or ansible_distribution == 'RedHat'`

Is there a reason that that's so common instead of checking for os\_family? Why do I not see this more often?:

`when: ansible_os_family == 'RedHat'`

Besides being more direct and concise, that also allows for other EL Linuxes like Scientific Linux and Rocky Linux without having to continually add to a list of all the possible EL derivatives. Is there something I'm missing?

# Answer
> 2 votes
Maybe try using `lsb_release` to derive the information, setting it as a fact, then look up the family from an Ansible hash map populated with the associations. Here is a mention of `ansible_facts['os_family']` at https://docs.ansible.com/ansible/latest/user\_guide/playbooks\_conditionals.html#ansible-facts-os-family

# Answer
> 2 votes
Based on the link that mr. @mikequentel share

> https://docs.ansible.com/ansible/latest/user\_guide/playbooks\_conditionals.html#ansible-facts-os-family

There is a list of distro or OS name that listed on `ansible_distribution` but not on `ansible_os_family`. So if your OS or distro listed on both you can choose what `facts` do you wanted to use.

---
Tags: ansible
---


# Question
Title: How is memory and storage shared between tasks running stateful applications?
I've been reasearching docker swarms and found out that a swarm is a collection of nodes, nodes are computers that run services, services are schedulers of tasks, and tasks are running containers. Services are made out of a single container image and can be made out of multiple redundant tasks.

My question is: if tasks are instances of container images, how is memory and storage shared between them, assuming your service is running a stateful application like a database (storage) or a website backend (memory)?

* Are the resource requirements multiplied and sinchronized regularly or in real-time? By that logic shouldn't there be a "master task"?
* If not, then stateful applications should have at maximum 1 task replica per service, and each redundancy unit should be a new actively mantained service (like databases, for instance)?

The core of my question is that I assume that services are general-purpose, so most types of applications could benefit from this task replication thing. Otherwise, isn't this core feature only suitable for stateless applications like rest apis? I imagine core features of general-purpose tools to also be general-purpose.

# Answer
> 1 votes
A container is nothing more than a process that has been partitioned from the host system and other processes using some linux process isolation technology.

Docker - even swarm's - role in this is to setup the parameters for this isolation and then hand the execution over to the OS to handle - this is partially what makes containers so lightweight - docker literally adds nothing to the execution of a containerised app.

This does mean, out the box, swarm is very good at orchestrating stateless containers. And anything that's stateful needs to have native support for being run in some kind of clustered/multi node configuration because docker swarm is not implementing anything magical. If you want to run a clustered Postgres db for example, you figure out how to run 3 Postgres instances without docker, and then write a swarm stack that spins up 3 replicas configured that way, solving problems like how each instance is going to persist its data: Swarm assumes tasks are stateless and will assume it can re-start any Postgres instance on any node, so its up to you to use cluster aware volumes, or use constrains to pin specific instances to specific nodes so they always use the same mounted data volumes etc.

---
Tags: docker, docker-swarm
---


# Question
Title: Our code is behind a firewall and we don't want to put source code in cloud. Are there any good CI/CD using local agents for Azure/Github actions
Our source code is in our servers and we don't have approval to put it in cloud.

We don't have a seamless CI/CD. Can we use Azure pipeline agents or any local agents to build our code and only export the build artifacts to cloud and continue with CD?

We are deploying our applications in Azure but not the source code.

Please suggest a good CI/CD for us.

# Answer
Azure DevOps also has a solution called Azure DevOps Server. It allows you to host you source code on-prem, and also build on-prem. Depending on your needs you could store your artifact on-prem or in the cloud and either way, it would allow you to deploy in the cloud.

So, yes, it is possible, but it will take quite some effort. At least you'll need a server for Azure DevOps Server, depending on the size another one for your database, and another one for your build and release agents. It will also take some effort to put together the pipelines and other infrastructure required, but if you really need to keep your sourcecode and builds on-prem this will definitely work for you.

Let me know if you need to know anything else.
> 1 votes

# Answer
You can create your entire build pipeline inside your network, so your version control and build server are all running behind the firewall.

There is a reasonable amount of effort in doing this, as you are now running infrastructure and updating the tooling on top of writing the software. This is why so many organizations are using cloud offerings such as GitHub, GitLab, and Azure DevOps. It does work out far cheaper and the each provide a trust centre to help the organization make a decision.

You would just need to make sure that the final stage of the build pipeline could access the "external" location where you plan to store the artefact or deploy the software to, so there would still be some network rules to set up your allow list for the outbound part of this process.
> 0 votes

---
Tags: jenkins-pipeline, azure, continuous-deployment, software-recommendation, github-actions
---


# Question
Title: Windows 10 docker desktop keeps showing Kubernetes is starting
I checked the `Enable Kubernetes` in windows Docker Desktop and it keeps circling and showing `Kubernetes is starting...`

I searched and couldn't find any proper answer. Some advises are about deleting Docker Desktop and reinstalling it.

I want to know any other solution Before pushing kill switch.

# Answer
> 0 votes
I tried this and worked for me.

First of all open a new PowerShell. We need to have Kubernetes server installed and running. So execute this command:

> **kubectl version**

I faced this line in the bottom of response:

> **Unable to connect to the server: dial tcp 127.0.0.1:6443: connectex:** **No connection could be made because the target machine actively** **refused it.**

If you got some similar response, it means Windows cannot resolve the internal domain which should be set in the `hosts` file. So do as the following instructions:

1. Open the hidden path

> **C:\Windows\System32\drivers\etc\hosts**

`hosts` file has no extension. Open it with notepad or any text editor. Do it with **Adminisrator** privilege.

2. insert this line in the opened hosts file:

> **127.0.0.1 kubernetes.docker.internal**

3. Save it.
4. Run Docker Desktop again.

And that's it.

**For checking the Kubernetes installation success, follow below instructions:**

Docker Desktop will not show Kubernetes as **installed** until all of the Kubernetes components are running. To see what is happening with its components open a new PowerShell to have the necessary environment variables set and run:

Open PowerShell and run this command:

> **kubectl get all -n kube-system**

Check if the followings are true:

```
1) all of the pods have “Running” status.
2) deamonset.apps/kube-proxy has “1” in the “Ready” column
3) deployment.apps/coredns has “2/2” in the “Ready” column
4) replicaset.apps/coredns-HASH has “2” in the “Ready” column

```

If any of the above statements are not true, use `kubectl describe -n kube-system OBJECTNAME` for more details. For example:

> **kubectl describe -n kube-system pod/coredns-558bd4d5db-99qxd**

You could be found more in this complete **discussion**.

I am still eager for new answer and discussions.

---
Tags: docker, kubernetes, windows
---


title: Prometheus scrape from service vs pods

text:
# Question
Title: Prometheus scrape from service vs pods
Let's say I have multiple pods (replicas) of application under a service (with `targetPort` pointing to port where application keeps its metrics for Prometheus) named "app-monitor-service". Right now I am getting metrics from pods directly using kubernetes\_sd\_config and is working fine.

Can I use `static_config` and point Prometheus to scrape from that service?

Logically scraping from service may seem to work as service routes requests in round robin fashion, Prometheus should be able to scrape metrics from all pods, but want to validate, if I am correct.

# Answer
Scraping through any kind of load balancer in **not advised**. Even though you have round robin managing of requests, pod will have slightly different values of metrics, and this will only increase confusion around them.

Additionally things like counters and histograms will have particularly bad time in this setup:
you'll see unexpected resets, that couldn't be correctly accounted by functions like `rate` and `increase`.
> 2 votes

---
Tags: kubernetes, prometheus
---


# Question
Title: AWX Failed to pull image on a private registry
**Context:**
I got a K8s cluster where I setup AWX using the official Helm install. On this cluster, there I also setup a Sonatype Nexus repository with a docker(hosted) private registry configured in order to pull custom AWX EE image I built.

At the moment, the private registry works only with a HTTP listener configured on port 8080. I'm able to login and push/pull my custom image to this registry from another server using docker cli.

**Configuration/AWX Setup for image pulling:**
In order to pull my custom image into my AWX project, I configure AWX the following way :

1. Setup a custom EE
2. Create a Container Registry credentials
3. And finally setup Execution Environment on my templates.

**Problem:**

When i launch a template, it keeps running for a long time, and here's error i got from kubectl :

```
PS > kubectl describe pods automation-job-50-78mp5
Name:             automation-job-50-78mp5
Namespace:        awx
Priority:         0
Service Account:  default
Node:             XXXXXX
Start Time:       Tue, 22 Aug 2023 11:42:04 +0200
Labels:           ansible-awx=4b53c506-cd99-41df-a4d5-1ed0395e8906
                  ansible-awx-job-id=50
Annotations:      cni.projectcalico.org/containerID: 66b9d3f6852b82b5e1d3b37ad8fc199667170c207f5f79af2c2e5d3a67007d9e
                  cni.projectcalico.org/podIP: 172.20.5.208/32
                  cni.projectcalico.org/podIPs: 172.20.5.208/32
Status:           Pending
IP:               172.20.5.208
IPs:
  IP:  172.20.5.208
Containers:
  worker:
    Container ID:
    Image:         awx-repo.mydomain.com/awx-repo
    Image ID:
    Port:          <none>
    Host Port:     <none>
    Args:
      ansible-runner
      worker
      --private-data-dir=/runner
    State:          Waiting
      Reason:       ImagePullBackOff
    Ready:          False
    Restart Count:  0
    Requests:
      cpu:        250m
      memory:     100Mi
    Environment:  <none>
    Mounts:       <none>
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:            <none>
QoS Class:          Burstable
Node-Selectors:     <none>
Tolerations:        node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                    node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  20s                default-scheduler  Successfully assigned awx/automation-job-50-78mp5 to frhd01cor003-ebeb10-worker-3
  Normal   BackOff    19s (x2 over 20s)  kubelet            Back-off pulling image "awx-repo.mydomain.com/awx-repo"
  Warning  Failed     19s (x2 over 20s)  kubelet            Error: ImagePullBackOff
  Normal   Pulling    5s (x2 over 20s)   kubelet            Pulling image "awx-repo.mydomain.com/awx-repo"
  Warning  Failed     5s (x2 over 20s)   kubelet            Failed to pull image "awx-repo.mydomain.com/awx-repo": rpc error: code = Unknown desc = failed to pull and unpack image "awx-repo.mydomain.com/awx-repo:latest": failed to resolve reference "awx-repo.mydomain.com/awx-repo:latest": failed to do request: Head "https://awx-repo.mydomain.com/v2/awx-repo/manifests/latest": x509: certificate signed by unknown authority
  Warning  Failed     5s (x2 over 20s)   kubelet            Error: ErrImagePull

```

My question are :

* How can I double check the pod is using credentials I set from AWX GUI ?
* As credentials works from the server, is there a "dedicated format" we have to set from AWX GUI ?
* Does the certificate x509 warning may impact the pull action ? (even if my repo seems well configured with a wildcard certificate"

# Answer
After days and tests here's the solution :

Kubernetes worker and master got the wrong certificate.

In my case, i have to :

* create a .pem certificate chain composed of the intermediate + wildcard certificate
* convert the .pem chain into a .pem fileas my tool (Nutanix Kubernetes Engine) need a .pem format.

Finally, workers and master got certificate updated into /etc/containerd/certs.d/my\_repo\_folder.

This makes AWX working and successfully pull image from my private registry.
> 0 votes

---
Tags: kubernetes, docker-registry, nexus, awx
---


# Question
Title: Why Minikube has different kernel version in its pods/containers than host OS?
I am running Minikube on a Linux kernel 6.1. I added a uname -r command in an image to check which kernel version the container "sees". And its 5.10.57.

I expected the version to be 6.1 as well. Why is it different?

Minikube is using the kvm2 driver.

# Answer
I found out that Minikube uses an ISO to boot under a hypervisor and thus needs a kernel. I thought Minikube itself runs on the host OS. https://minikube.sigs.k8s.io/docs/contrib/building/iso/ https://github.com/kubernetes/minikube/issues/10501
> 2 votes

---
Tags: kubernetes, linux, minikube
---


# Question
Title: Unable to connect to the server: tls: failed to verify certificate: x509: certificate signed by unknown authority
I am facing issue when trying to import k8s cluster in Rancher.

here are the logs of when trying to run the commands:

Commnad:

```
curl --insecure -sfL https://172.31.91.168/v3/import/zlxz57dj7nhpdzx4cw7lvfdnj9zt7n2tpk4df5v7frq2tfjf52j49j_c-m-jgw4wzm5.yaml | kubectl apply -f -

```

Logs

```
clusterrole.rbac.authorization.k8s.io/proxy-clusterrole-kubeapiserver unchanged

clusterrolebinding.rbac.authorization.k8s.io/proxy-role-binding-kubernetes-master unchanged
namespace/cattle-system unchanged
serviceaccount/cattle unchanged
clusterrolebinding.rbac.authorization.k8s.io/cattle-admin-binding unchanged
secret/cattle-credentials-f144527 unchanged
clusterrole.rbac.authorization.k8s.io/cattle-admin unchanged
deployment.apps/cattle-cluster-agent configured
service/cattle-cluster-agent unchanged

```

Second Command:

```
kubectl apply -f https://172.31.91.168/v3/import/zlxz57dj7nhpdzx4cw7lvfdnj9zt7n2tpk4df5v7frq2tfjf52j49j_c-m-jgw4wzm5.yaml

```

Logs:

```
Unable to connect to the server: tls: failed to verify certificate: x509: certificate signed by unknown authority

```

Unable to identify the issue. As rancher is running on separate e2 instance under Docker and K8s cluster is running separate machine and worker node as well.

For reference sharing the snap shot:

I check out almost the number of given answer here related to SSL but not able to find and relevant answer.

# Answer
Create file daemon.json in /etc/docker/ inside put this: { "insecure-registries" : \["docker-registry-machine-name:443","docker-registry-machine-name:80","0.0.0.0/0"\] } and save it.

After that with jenkins user run this command: git config --global http.sslVerify "false"

Then restart Jenkins and should work.
> 0 votes

---
Tags: kubernetes, linux, ssl, rancher
---


# Question
Title: Using docker as a substitute for VM for testing
I'm new to docker and I want to start using it in the following scenario.

For example, I need to implement an Asp.Net Core Web API and deploy it to a linux system. The development is done on a Windows machine in Visual Studio. My idea is to create a docker image which mimics the target linux system, develop the service and test it in a docker image and then deploy it to the target system without docker.

In other words, I'm going to use docker for the development process, but the application is not going to be containerized in the production.

Is it a reasonable scenario or am I doing something wrong?

**Update:**

Perhaps I was misleading in using the term "service". I didn't mean linux service. What I really meant was Asp.Net Core web service.

I also considered using WSL, but I would need to install additional software like NGINX to test the application. Usually I use a virtual machine (VMWare) in this case. I prepare the target environment in a virtual machine and test my application there, so I don't have to pollute my host system. But virtual machines have a lot of overhead, so I thought that using docker was going to be less taxing.

# Answer
> 0 votes
Developing a service to test in a docker container is always tricky because you have to have systemd (or whatever) running as your base process to keep the container up. There are ways around it, but I think it's usually better to test the code you're writing as the main process rather than test it as a service.

Just remember, you're not testing systemd.

For that kind of stuff I'd use vagrant rather than docker. Your CI/CD tools probably won't care too much although it'll use more resources.

# Answer
> 0 votes
In addition to Peter comments, I would like to add that you should think even more about your objective than the tool. If your goal is to deploy a system on a linux machine. Docker is not going to show you all the problems you might have using Linux Native.

Without knowing all its requirements I would say that a SubSystem can show you better and prepare you better for this case, or really Vagrant, I'm talking about the subsystem because you probably already have it in your Windows, just activate it.

---
Tags: docker, linux, dotnet
---


# Question
Title: Configurable "Ingress filter" to prevent Grafana Loki for storing log item
I'm currently setting up a stack with `Grafana` and `Grafana Loki` to store, query, and analyze logs from different web applications. Most applications run on .NET using Serilog and the `Serilog.Sinks.Grafana.Loki`-Sink.

From time to time we see errors och warnings in the logs that we know are not relevant or that we want to ignore.

I know there are several options here but none of them are "straightforward":

**1. Filtering them in the reporting web app so that they are never sent to Loki**

This would require a release of the app just to filter out logs. Does not make sense to me.

**2. Filter using LogQL**

In this case the data is still stored in Loki and it would also make my queries complicated - would be really hard to make sure all dashboards uses them same filtering logic.

**3. Filter using loki-configuration** There seems to be some kind of `pre_ingesters`-setting in the Loki configuration but using this would probably require a restart of the Loki-service which is not a great option.

I'm basically looking for something where I (as the user) can create a number of filters (preferably using a web UI) that would be applied before Loki store the log lines, like some kind of proxy-layer.

How can this be solved?

# Answer
> 0 votes
Suggestion (disclaimer: have not tried to do what you are trying to accomplish)...see if Caddy request matchers might help: https://caddyserver.com/docs/caddyfile/matchers

Example of configuring Loki to run with Caddy (in this case, to support TLS): https://github.com/mikequentel/caddy-and-loki-deployment-example

---
Tags: grafana
---


# Question
Title: Key Vault managed stored account encryption permissions denied
Trying to amend an existing storage account to add Customer-Managed-Key Rotation to it, using key vault, as per the docs from Terraform and MS.

Looks like we've done everything right, but we're just getting permissions errors, and not even setting admin seems to resolve it.

Now if we were using access policies, which all the docs do, I suspect it would be fine, but I can't find any roles that explicitly say `Microsoft.KeyVault/vaults/storageaccounts/read` (`They do say Microsoft.KeyVault/vaults/*/read though`).

Given it looks like its' trying to access storage, I also granted my SPN access to that (Even though it already had contrib - but no dice).

The specific url failing is:

https://kmykv.vault.azure.net/storage/mystorageaccount?api-version=7.4:

```
Checking for presence of existing Managed Sctorage Account "mystorageaccoutn" (Key Vault "Key Vault (Subscription: \"a3141da1-06b2-4cae-8032-0000000\"\nResource Group Name: \"myrg\"\nKey Vault Name: \"myvault\")"): keyvault.BaseClient#GetStorageAccount: Failure responding to request: StatusCode=403

-- Original Error: autorest/azure: Service returned an error. Status=403 Code="Forbidden" Message="Caller is not authorized to perform action on resource.\r\nIf role assignments, deny assignments or role definitions were changed recently, please observe propagation time.

\r\nCaller: appid=000000-00-4913-8a44-f6d8112355b0;oid=000000-ed7f-42d0-8f41-0000;iss=https://sts.windows.net/00000-1057-4117-ac28-80cdfe86f8c3/\r\nAction: 'Microsoft.KeyVault/vaults/storageaccounts/read'\r\nResource: '/subscriptions/00000-06b2-4cae-8032-0000/resourcegroups/myrg/providers/microsoft.keyvault/vaults/myvault'\r\nAssignment: (not found)\r\nDecisionReason: 'DeniedWithNoValidRBAC' \r\nVault: myvault;location=uksouth\r\n" InnerError={"code":"ForbiddenByRbac"}

```

Terraform

```
data "azurerm_client_config" "current" {}

data "azuread_service_principal" "keyvaultspn" {
  #This is the SPN for KeyVault defined across AZ - https://learn.microsoft.com/en-us/azure/key-vault/secrets/overview-storage-keys
  # display_name = "Azure Key Vault"
  application_id = "cfa8b339-82a2-471a-a3c9-0fc0be7a4093"
}

resource "azurerm_storage_account" "this" {
  name                              = "mystorageaccount"
  location                          = "uksouth"
  resource_group_name               = "myrg"

  identity {
    type = "SystemAssigned"
  }
}

resource "azurerm_role_assignment" "kvstoragerole" {
  scope                = azurerm_storage_account.this.id
  role_definition_name = "Storage Account Key Operator Service Role"
  principal_id         = data.azuread_service_principal.keyvaultspn.id
}

resource "azurerm_role_assignment" "storagekvrole" {
  scope                = var.key_vault_id
  role_definition_name = "Key Vault Crypto Officer"
  principal_id         = azurerm_storage_account.this.identity[0].principal_id
}

resource "azurerm_key_vault_managed_storage_account" "this" {
  name                         = azurerm_storage_account.this.name
  key_vault_id                 = var.key_vault_id
  storage_account_id           = azurerm_storage_account.this.id
  storage_account_key          = "key1"
  regenerate_key_automatically = true
  regeneration_period          = "P1D"
}

```

# Answer
> 0 votes
Check your Key-vault permission model under your Key Vault -\> Settings -\> Access Configuration on Azure portal.

Azure keyvault mainly allows key vault access using two permission models.

Azure role-based access control Vault access policy

If you are using Azure role-based access control, make sure that you have given 'Key Vault Administrator' role to your service principal from Access Control (IAM) section of your key vault.

Refer: https://learn.microsoft.com/en-us/azure/key-vault/general/rbac-guide?tabs=azure-cli

If you are using Vault access policy, make sure that you have added policy for your service principal with necessary permissions under Access Policies section of your key vault.

You could also use: Set-AzKeyVaultAccessPolicy -VaultName "" -UserPrincipalName "user@domain.com" -PermissionsToSecrets delete,get,list,set

---
Tags: terraform, azure, permissions
---


# Question
Title: Your organization X failed to connect to the X Azure Active Directory
I am trying to connect my Azure DevOps environment to Azure Active Directory. I get the following error:

> Tenant switch failed, on a precondition, with the following message: 1 of 10 total users has multiple active identities with the same UPN. Please either remove the duplicates or change the UPNs to be unique.

There are only 9 users in Azure DevOps. The only thing I can think of is that a user is removed from the organisation and later readded, but I can't find any duplicates in the Users in DevOps. Also, in the target AAD there are no duplicates (I can't imagine you can have duplicate UPN anyway).

I can't really find anything that can help met troubleshoot this issue. Anyone has any ideas to fix this?

# Answer
Maybe you created a cloud account with the same UPN before sync'ing the on premise with azure ad connect? or something else of that nature.

Navigate to graph explorer https://developer.microsoft.com/en-us/graph/graph-explorer log in with a azure ad admin account and type in a query like this

https://graph.microsoft.com/v1.0/users?$filter=startswith(UserPrincipalName,'##UPNHavingIssues##')

There should only be entry, but if there are multiple, then that's where the problem is.
> 1 votes

---
Tags: azure-devops, azure, active-directory
---


# Question
Title: Re-using deployment stages in multiple YAMLs
We have large application we deploy from ADO using classic pipelines. In the process of converting over to yaml. The application is made up of multiple smaller apps which are deployed individually into multiple environments. The environments are being added and removed as projects commence and complete so it is not a fixed list.

The problem is we will need to update 20-30 yamls each time the list changes.

We are using templates to reuse code extensively. The question is - can we have a master list of environments and parameters that can be read into each deployment at deploy time?

Here is a example snippet of the pipeline code we use - calls out to our deployment yaml for each environment/stage:

```
extends:
  template: pipelines/main.yml@templates
    deployStages:
      - template: stages/deploy.yml@templates
        parameters:
          dependsOn: buildContainer
          deploySteps:
            - template: ./app-deploy.yml
              parameters:
                serviceConnection: test
          environment: test
          poolName: 'Azure Pipelines'
          runIfNotMainOrMaster: ${{ eq(parameters.action, 'Deploy') }}
          variableGroup: 'test'
      - template: stages/deploy.yml@templates
        parameters:
          dependsOn: deploy_test
          deploySteps:
            - template: ./app-deploy.yml
              parameters:
                serviceConnection: staging
          environment: stage
          poolName: 'Azure Pipelines'
          variableGroup: 'staging'
      - template: stages/deploy.yml@templates
        parameters:
          dependsOn: deploy_stage
          deploySteps:
            - template: ./app-deploy.yml
              parameters:
                serviceConnection: production
          environment: prd1
          poolName: 'Azure Pipelines'
          variableGroup: 'prd1'

```

etc.

# Answer
Create a variable group for each environment, containing the environment-specific parameter values.

In your parameterized templates, reference the variables from the respective variable group. These templates can be reused across different deployments.

In your individual pipeline YAML files, include the parameterized templates and link them to the appropriate variable group based on the deployment environment.

When you trigger a pipeline for a specific environment, Azure DevOps will automatically inject the values from the corresponding variable group into the templates. This allows you to achieve dynamic configuration without having to manually modify YAML files each time.
> 0 votes

---
Tags: azure, azure-pipelines, yaml
---


# Question
Title: How can I use two different docker container on the production server (after pushing to DockerHub)?
I have the `docker-compose` file with two containers. "nginx" container depends on "demo" (it's python django backend).

```
version: '3.9'

volumes:
  static:

services:
  demo:
    image: findsimilar/demo
    build:
      context: .
    restart: always
    expose:
      - 8080
    volumes:
      - static:/static/
    command: bash -c "
      python manage.py collectstatic --noinput --settings=demo.prod_settings
      && gunicorn demo.wsgi -b 0.0.0.0:8080
      "
  nginx:
    image: findsimilar/demo-nginx
    build: ./nginx
    restart: always
    ports:
      - 80:80
    volumes:
      - static:/etc/nginx/static/
    depends_on:
      - demo

```

I've done `docker compose build` and `docker compose push`. Images have been pushed on DockerHub.

The question is "How can I properly use this images on production server?" I've tried `docker run findsimilar/demo` and `docker run findesimilar/demo-nginx`. The images have been pulled but `findsimilar/demo` haven't been started and `findsimilar/demo-nginx` didn't find a demo:8080 host.

# Answer
In the directory containing the `docker-compose.yml` one can do:

1. Start containers (detached mode): `docker-compose up -d`
2. Stop containers: `docker-compose down --remove-orphans`
> 0 votes

# Answer
Okay, I got the answer.

To use my DockerHub images on production I need special docker-compose file, without build detective. This is the work example in my case:

```
version: '3.9'

volumes:
  static:

services:
  demo:
    image: findsimilar/demo
    restart: always
    expose:
      - 8080
    volumes:
      - static:/static/
    command: bash -c "
      python manage.py collectstatic --noinput --
   settings=demo.prod_settings
  && gunicorn demo.wsgi -b 0.0.0.0:8080
  "
  nginx:
    image: findsimilar/demo-nginx
    restart: always
    ports:
      - 80:80
    volumes:
      - static:/etc/nginx/static/
    depends_on:
      - demo

```

The file is similar with docker-compose in the question but without `build` and `context`

I did `docker compose pull` and `docker compose up -d` on my server and it worked.
> 0 votes

---
Tags: docker, docker-compose, dockerhub
---


title: docker image push does not work

text:
# Question
Title: docker image push does not work
I have created a new docker image with a

```
# docker build -t my-phpapache:7.4 .

```

Now, when I try to make

```
# docker image push

```

I'm logged in to a Docker hub in a browser. I see the errors:

```
$ docker image push --all-tags docker.io/library/my-phpapache
The push refers to repository [docker.io/library/my-phpapache]
f04ed43187a6: Preparing
0ff1b9aaef0c: Preparing
c18803e039cd: Preparing
a735fdc00d49: Preparing
de7c912f2726: Preparing
e835c99ddfc4: Waiting
1f55d9e78afa: Waiting
5aefa6797d83: Waiting
70af74272d2e: Waiting
58e3131f3b01: Waiting
a1bae98a9430: Waiting
752cff7a1101: Waiting
be3607e92e69: Waiting
2ffebc0bdeea: Waiting
4c94b016478b: Waiting
a9019c838a13: Waiting
015643a98838: Waiting
f2c64a370cec: Waiting
d5b8874e6c41: Waiting
9eb82f04c782: Waiting
denied: requested access to the resource is denied

```

# Answer
To push, you need write access to the destination repository. With Docker Hub, those repositories must be under your userid, or in an organization where you have write access. The default path you see added, `docker.io/library`, is used by official images, which makes pulling official images easier. But since you aren't the author of docker official images, that won't work for pushing.

That means you need to:

```
docker build -t your-user/your-image:v1 .
docker push your-user/your-image:v1

```

Where `your-user` is the same user you used when running `docker login`.
> 1 votes

# Answer
you need to login on your console to your docker registry the push the image to the local or public registry.

```
docker login registry.mydomain.xxx:5000
username: xxxx
password: xxxx
docker push registry.mydomain.xxx:5000

```
> 0 votes

# Answer
Here are the steps that worked for me:

1. docker logout (if already logged in)
2. docker login
3. docker build . -t
4. docker tag /:
5. docker push /:

Before pushing the image we need to tag it with the a name to be displayed on the repository by providing the location to the docker hub repository/registry and the tag name.

Thanks!
> 0 votes

---
Tags: docker
---