title: CICD with dockerized web app

text:
# Question
Title: CICD with dockerized web app
I am having troubles understanding the following DevOps concept.

I have developed a dockerized web app using docker-compose. It is in my local and I can upload it to any cloud service I want and it works fine. The point is that when I try to establish a CI/CD pipeline, specifically CircleCI (I don't know how it works with any other alternative), I found out that I need to use a yml file that specifies a docker image for a given job.

My question is: does this mean that my entire web app will run in this specific image? If so, how can I translate my current working web app that uses docker-compose to a web app that takes advantage of CI/CD?

# Answer
> 0 votes
Yes, you can and you should take advantage of the docker-compose app, see CI and CD are two different things, as you have running stack, so you have one basic version of application and its stack, Now needs to update those whenever you push changes to your app.

We have to push changes to application and build new image, once we have build and push the image to image registry, it cab be local or aws whatever, we have to tell the docker-compose that we have updated the image.

so in CI pipeline you have to add two stages, build and push,

and in CD pipeline, you have to have one deploy stage, and depending upon your server / vm position, you can login to server, and update docker-compose with new image tag, and up it again.

I hope this will give you starting point.

---
Tags: docker, docker-compose, cicd, circleci-2.0
---


# Question
Title: How to enable Google Maps and generate an API key using Terraform?
I've been trying to spin up a project from scratch over the Christmas break to automate the creation of my home network, one of those components is the Ubiquiti Controller which requires a Google Maps API key to plot devices onto maps.

I've been able to automate the creation of the project using Terraform:

```
resource "google_project" "slaterfamily_network" {
  depends_on      = ["data.google_organization.org"]
  name            = "Slater Family Network"
  project_id      = "slaterfamily-network"
  org_id          = "${data.google_organization.org.id}"
  billing_account = "${var.billing_account}"
}

resource "google_project_iam_binding" "account_owner" {
  project = "${google_project.slaterfamily_network.project_id}"

  # It's important to specify ALL members, as this replaces them:
  #   https://twitter.com/devopsreact/status/840233052194320384
  members = [
    "user:richard@slaterfamily.name",
    "serviceAccount:${data.google_service_account.terraform_service_account.email}",
  ]

  role = "roles/owner"
}

```

This works fine for configuring Google Cloud DNS, however, I can't seem to find a way of enabling Google Maps and generating an API key, the following statements enable the Service Usage API, however it fails enabling the Maps API:

```
resource "google_project_service" "service_usage" {
  project = "${var.project_id}"
  service = "serviceusage.googleapis.com"
}

resource "google_project_service" "maps" {
  depends_on = ["google_project_service.service_usage"]
  project = "${var.project_id}"
  service = "maps.googleapis.com"
}

```

With this error:

```
google_project_service.maps: Error enabling service: failed to issue request: googleapi: Error 403: The caller does not have permission, forbidden

```

Which after tracing the HTTP calls gives me the following, formatted as cURL for ease of reproduction:

```
curl -H 'Host: serviceusage.googleapis.com' -H 'content-type: application/json' -H 'authorization: Bearer {{ BEARER_TOKEN }}' -H 'user-agent: google-api-go-client/0.5 Terraform/0.11.7 (+https://www.terraform.io) terraform-provider-google/1.20.0' --data-binary "{}" --compressed 'https://serviceusage.googleapis.com/v1beta1/projects/slaterfamily-network/services/maps.googleapis.com:enable?alt=json&prettyPrint=false'

```

Which results in:

```
{
  "error": {
    "code": 403,
    "message": "The caller does not have permission",
    "errors": [
      {
        "message": "The caller does not have permission",
        "domain": "global",
        "reason": "forbidden"
      }
    ],
    "status": "PERMISSION_DENIED"
  }
}

```

# Answer
> 2 votes
I guess the service ID you're looking for is `maps-backend.googleapis.com` or `static-maps-backend.googleapis.com` if you want to generate static maps.

```
➜  ~ gcloud services list --available |grep maps
maps-android-backend.googleapis.com                                                                  Maps SDK for Android
maps-backend.googleapis.com                                                                          Maps JavaScript API
maps-embed-backend.googleapis.com                                                                    Maps Embed API
maps-ios-backend.googleapis.com                                                                      Maps SDK for iOS
static-maps-backend.googleapis.com                                                                   Maps Static API

```

I've got these result with Google Cloud SDK version 360.0.0, also there is `apikeys.googleapis.com` service available with `alpha` component which makes me think it could be achievable by using gcloud module and `gcloud alpha services api-keys create` command.

# Answer
> 1 votes
I faced the similar problem recently, but fixed it by providing additional roles to the service account. I am not sure what permissions you need, you need to look it up. In my case, I was trying to create an appengine project, So I had to add "Appengine Admin" role to the service account and enable "appengine.googleapis.com" via terraform resource "google\_project\_service"

# Answer
> 0 votes
Thanks @Ahmet! I put up a bounty for this question and Ahmet came through with a great answer. I tried the `gcloud` module for terraform and it worked! However I needed to go a step further as the gcloud command wasn't able to save the output and also couldn't read the file due to permission issues. So I actually took the concept and used the `local-exec` in terraform instead to run a script which does all the gcloud commands I needed.

You can find a list of APIs here: https://developers.google.com/apis-explorer/

Go into the documentation for the one you need and copy the url `*.googleapis.com`, in my example I needed the Geocoding API turned on and an API key for it.

```
# main.tf

module "project_services" {
  source  = "terraform-google-modules/project-factory/google//modules/project_services"
  version = "11.2.1"

  project_id = var.project_id
  activate_apis = [
    "geocoding-backend.googleapis.com",
  ]
  disable_dependent_services  = false
  disable_services_on_destroy = false
}

resource "null_resource" "geocode" {
  provisioner "local-exec" {
    command = "chmod +x ./scripts/script.sh && sh ./scripts/script.sh"
    environment = {
      SERVICE_API = "geocoding-backend.googleapis.com"
      KEY_NAME = "my-key"
      SECRET_NAME = "my-secret"
    }
  }
  depends_on = [
    # e.g. module.secret_manager
  ]
}

```

Then you can create a script like so, with the environment variables being injected in:

```
# ./scripts/script.sh

GCLOUD_LOCATION=$(command -v gcloud)
echo "Using gcloud from $GCLOUD_LOCATION"

gcloud alpha services api-keys create --api-target=service="$SERVICE_API" --display-name="$KEY_NAME" &&
export KEY_NAME=$(gcloud alpha services api-keys list --format="value(name)" --sort-by createTime --limit 1) &&
export API_KEY=$(gcloud alpha services api-keys get-key-string $KEY_NAME --format="value(keyString)") &&
printf $API_KEY | gcloud secrets versions add "$SECRET_NAME" --data-file=-

```

My above script takes the key and inserts it into an existing Secret I created in terraform.

---
Tags: terraform, google-cloud-platform
---


# Question
Title: Configure Azure DevOps Server to send email using Office365
I'm trying to configure my Azure DevOps Server to send emails using my Office 365 account, with no luck. I tried setting the SMTP server as indicated on this page: POP, IMAP, and SMTP settings but it doesn't work and gives me an error if I try to send an email ("Error in processing. The server response was: 5.7.3 STARTTLS is required to send mail").

Is there a way to configure it properly? Or as an alternative, do you know of any SMTP server/proxy that I can install to work around the problem?

# Answer
> 1 votes
To make it work, you need to check the "This server requires a secured connection (SSL)" checkbox and **restart the server**.

---
Tags: azure-devops
---


# Question
Title: Docker Swarm set which service goes to which machine
Docker Swarm set which service goes to which machine.

Is there any way to set which service or service replicas goes to which worker machine ?

# Answer
> 4 votes
In the placement section of the compose file, you can specify constraints. I'd recommend using node labels so you can add more potential nodes (should an existing node fail) without redefining any deployments bound to a node, but you can also specify hostnames. The result looks like:

```
version: "3.9"
services:
  db:
    image: postgres
    deploy:
      placement:
        constraints:
          - "node.hostname==dbhost"

```

---
Tags: docker, docker-compose, docker-swarm
---


# Question
Title: How are ansible playbooks supposed to be imported into a collection?
I'm trying to develop a reusable collection of playbooks that I can run from docker containers. I don't want to share duplicate code between containers and I don't want to have containers that run multiple playbooks so what I'd like do develop is something where I have a

`playbooks/doit.yml`

file and a

`.ansible/collections/playbooks/x/y/playbooks/thing-to-do.yml`

playbook that I can import into my playbook

However, none of the documentation explains how to do that yet.

import playbooks doesn't say anything about collections

https://docs.ansible.com/ansible/latest/modules/import\_playbook\_module.html

and playbooks collection documentation currently says tbd

https://docs.ansible.com/ansible/latest/dev\_guide/developing\_collections.html#playbooks-directory

I'm running ansible 2.9, I suppose I could import the playbook given the full path, but that seems like not what they're wanting me to do.

# Answer
> 0 votes
**tl;dr**

1. playbooks can only have alphanumeric characters and underscores
2. run them with `ansible-playbook namespace.module.playbook_name_without_the_extension`

---

The documentation provided by ansible is getting better:

https://docs.ansible.com/ansible/latest/user\_guide/collections\_using.html#using-a-playbook-from-a-collection

doesn't quite explain why it wasn't working for you and you neglected to mention in your post the name of the playbook you were trying to run.

In experimenting to find the solution to this issue, I installed the power ibmi collection

`ansible-galaxy collection install git+https://github.com/IBM/ansible-for-i.git`

it seemed odd that `ansible-playbook ibm.power_ibmi.ibmi-fix-repo-ptf-group` produced

this error

`ERROR! the playbook: ibm.power_ibmi.ibmi-fix-repo-ptf-group could not be found`

but

`ansible-playbook ibm.power_ibmi.install_fix_repo_dependencies`

did not,

the documentation for the roles directory https://docs.ansible.com/ansible/latest/dev\_guide/developing\_collections\_structure.html#roles-directory spells it out pretty clearly that the roles need to be underscores, not dashes, but the "playbooks" section is still TBD so it wouldn't have been that easy to figure out without trial and error

# Answer
> 1 votes
this feature was recently merged to ansible devel branch. It would be posible using:

```
- import_playbook: testns.testcoll.default_collection_playbook

```

---
Tags: ansible
---


# Question
Title: What are the metrics to consider when doing API performance testing?
I want to establish the benchmark behavior of my API through performance testing.
The purpose is to ensure that the product meet or exceed requirements such as:

* Throughput
* Response time
* Stability
* Resource usage

Some of KPIs I'm aware of:

* Average response time
* Max connexion number
* Success rate

Is there a number of other industry-defined KPIs and what are the most important metrics to consider when stress testing an API ?

# Answer
May I suggest taking a look at JMeter (https://jmeter.apache.org). It's pretty much the OO standard for performance/load testing and comes with a lot of predefined report/KPI output types (https://jmeter.apache.org/usermanual/generating-dashboard.html). We use it extensively as part of our deployment chain for post build/deploy reporting on test environments. During maximum load testing we stream JMeter performance metrics into InfluxDB and visualise them realtime using Grafana https://grafana.com/grafana/dashboards/13644.
> 0 votes

---
Tags: testing, metrics, kpi
---


# Question
Title: How many nodes can I have in single EKS cluster?
I have an EKS cluster and today my subnet had `0` Available IPv4 addresses. Does that mean that there are limits per nodes within 1 cluster in EKS?

# Answer
You should read the official EKS documentation for this. But based upon your question, your subnet having 0 Available IPs means that your cluster depending on how many nodes it has, cannot provide more IPv4 for Pods(for example). It all comes down to the size of the cluster, as each Node size comes with its own IPv4/6 limits. You can check that here.
> 0 votes

---
Tags: amazon-web-services, eks
---


# Question
Title: pypa/gh-action-pypi-publish token value 'does not start with "pypi-"'
I am trying to use the `pypa/gh-action-pypi-publish` action in GitHub to push a Python package to PyPI. I'm in the team that owns microbiomedata/nmdc-schema and successfully uses a GitHub action to deploy the nmdc-schema. I have also read PyPA's advice and tried to follow it precisely.

However, when it comes to authenticating against PyPI, I get this "no matter what I do"

```
Run pypa/gh-action-pypi-publish@v1.2.2
  with:
    user: __token__
    password: ${ secrets.pypi_password }
    packages_dir: dist
    verify_metadata: true
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.8.11/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.8.11/x64/lib
/usr/bin/docker run --name a33c111d135a08e424daf8f55d57f9438a27b_084f0c --label 8a33c1 --workdir /github/workspace --rm -e pythonLocation -e LD_LIBRARY_PATH -e INPUT_USER -e INPUT_PASSWORD -e INPUT_REPOSITORY_URL -e INPUT_PACKAGES_DIR -e INPUT_VERIFY_METADATA -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_OS -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/linkml-template-audit/linkml-template-audit":"/github/workspace" 8a33c1:11d135a08e424daf8f55d57f9438a27b  "__token__" "${ secrets.pypi_password }" "" "dist" "true"
Warning:  It looks like you are trying to use an API token to authenticate in the package index and your token value does not start with "pypi-" as it typically should. This may cause an authentication error. Please verify that you have copied your token properly if such an error occurs.
Checking dist/linkml_template_audit-0.0.1-py3-none-any.whl: PASSED, with warnings
  warning: `long_description_content_type` missing. defaulting to `text/x-rst`.
Checking dist/linkml_template_audit-0.0.1.tar.gz: PASSED, with warnings
  warning: `long_description_content_type` missing. defaulting to `text/x-rst`.
Uploading distributions to https://upload.pypi.org/legacy/
Uploading linkml_template_audit-0.0.1-py3-none-any.whl

  0%|          | 0.00/29.6k [00:00<?, ?B/s]
 27%|██▋       | 8.00k/29.6k [00:00<00:00, 48.7kB/s]
100%|██████████| 29.6k/29.6k [00:00<00:00, 107kB/s]
NOTE: Try --verbose to see response content.
HTTPError: 403 Forbidden from https://upload.pypi.org/legacy/
Invalid or non-existent authentication information. See https://pypi.org/help/#invalid-auth for more information.

```

# Answer
I was only wrapping my PyPI secret token in one pair of curly brackets! Two are required, like

`${{ secrets.nmdc_schema_pypi_token }}`
> 1 votes

# Answer
In my case the issue was that I added secrets as "environment secret" rather than "repository secret", leaving the latter empty.
> 1 votes

# Answer
This happened for me in the following circumstances:

* Forked a project in github
* Added a secret for the fork
* Pushed update to my fork, and created a PR
* The PR action ran in the upstream project
* Build failed, with the "does not start with pypi" error

Instead I created a branch directly in the upstream project (along with the secret token), and when I pushed the branch to the upstream project, it ran as expected and updated TestPyPi.
> 0 votes

---
Tags: python, github-actions
---


# Question
Title: Do GitLab Runner worker nodes need to have network access to GitLab?
GitLab runner has an autoscale feature that allows the Runner Manager to dynamically spawn new worker nodes on demand.

The documentation says that: "GitLab Runner is going to need to talk to your GitLab instance over the network". This is obvious to me since the runner needs to download the source code and push build results and artifacts back to GitLab.

Is this required for the Runner Manager only or also for its worker nodes? Do the nodes communicate with GitLab directly or do they only talk to the Runner Manager, which then propagates their results to GitLab?

In other words: With a GitLab instance in my private corporate network and autoscaling GitLab Runner with AWS EC2, do I need to build a VPN between my corporate network and AWS VPC or will it work out of the box?

# Answer
After performing some experiments: **Yes, worker nodes need access to the GitLab instance.** (At least on GitLab Runner 14.)

The Runner Manager will just spawn the workers and communication happens directly between the workers and the GitLab instances. When the workers cannot access GitLab, you'll probably see an error message like:

```
fatal: unable to access 'https://your.gitlab.instance/awesome/project.git/': Could not resolve host: your.gitlab.instance

```
> 1 votes

---
Tags: amazon-web-services, gitlab, cicd, gitlab-ci-runner, autoscaling
---


title: Docker stack deploy doesn't pull images

text:
# Question
Title: Docker stack deploy doesn't pull images
I'm launching through Ansible (just SSH basically) a docker stack deploy, the issue is also true when launching directly from a `manager`:

```
docker stack deploy --prune --with-registry-auth -c /docker/docker-compose.yml "test-stack"

```

I have my credentials to access this registry are defined in `.docker/config.json`.

The problem is first seen here with a `docker stack ps test-stack`:

```
 ... Shutdown            Rejected 11 minutes ago      "No such image: myregistry.com:12345/myimage:mytag"

```

The workers aren't uploading the images I want. On their logs, I get the following:

```
Jan 31 11:22:24 myhost dockerd[1149]: time="2019-01-31T11:22:24.574302458+01:00" level=warning msg="Error getting v2 registry: Get https://myregistry.com:12345/v2/: http: server gave HTTP response to HTTPS client"
Jan 31 11:22:24 myhost dockerd[1149]: time="2019-01-31T11:22:24.574342640+01:00" level=info msg="Attempting next endpoint for pull after error: Get https://myregistry.com:12345/v2/: http: server gave HTTP response to HTTPS client"
Jan 31 11:22:24 myhost dockerd[1149]: time="2019-01-31T11:22:24.618063948+01:00" level=info msg="Attempting next endpoint for pull after error: unauthorized: authentication required"
Jan 31 11:22:24 myhost dockerd[1149]: time="2019-01-31T11:22:24.618121224+01:00" level=error msg="pulling image failed" error="unauthorized: authentication required" module=node/agent/taskmanager node.id=rarbt9afb16y7j77bfwiacby2 service.id=ej4y7wbwp539u7mmmhfashz93 task.id=c1uy2gnwdcmibp6l2p9rfxcsh
Jan 31 11:22:24 myhost dockerd[1149]: time="2019-01-31T11:22:24.618840324+01:00" level=error msg="fatal task error" error="No such image: myregistry.com:12345/myimage:mytag" module=node/agent/taskmanager node.id=rarbt9afb16y7j77bfwiacby2 service.id=ej4y7wbwp539u7mmmhfashz93 task.id=c1uy2gnwdcmibp6l2p9rfxcsh
Jan 31 11:22:29 myhost dockerd[1149]: time="2019-01-31T11:22:29.548642584+01:00" level=warning msg="failed to deactivate service binding for container test-stack_myservice.1.c1uy2gnwdcmibp6l2p9rfxcsh" error="No such container: test-stack_myservice.1.c1uy2gnwdcmibp6l2p9rfxcsh" module=node/agent node.id=rarbt9afb16y7j77bfwiacby2

```

**The problem** is that, when I do a `docker pull myregistry.com:12345/myimage:mytag` on the workers, it pulls the image successfully and without any problem, but not through an `docker stack deploy` command. Despite the fact that i have the option `--with-registry-auth`.

# Answer
So it turned out that some of the workers didn't have the **correct credentials** to the docker registry. After I applied the correct ones, the command worked normally.

That's weird, since I could do it locally... Anyway, case solved.
> 4 votes

# Answer
@jayooin that's weird, if you use `--with-registry-auth`, you don't normally have to set up the credentials on each worker node as the credentials are passed to the wokers by the manager.

In my case the issue was that I was using a self-signed certificate for accessing the private docker registry. I had configured it correctly on the master node, but not on the workers, so connecting to the registry raised a "x509: certificate signed by unknown authority" error on the workers. Fixing the certificate error on the workers solved the issue, without setting up the registry credentials on them.
> 1 votes

---
Tags: docker, docker-compose, docker-swarm, docker-registry
---


# Question
Title: Azure DevOps issue with GIT, Visual Studio and Azure (Umbraco)
I am completely new to Visual Studio, .net and Azure DevOps

I have a problem with GIT using the GIT bits (rather than CMD), in Visual Studio

I have a .Net application (Umbraco), that has been built and runs fine. I built a CI pipeline in the Azure Deployment blade for my Azure web App. All works fine.

When I go to Azure DevOps I can see this pipeline and it is called myappname -CI

If I add a test file locally to my Repo, file \>new file \> test.txt and save it, it appears as a change in the GIT explorer on the right-hand side. I can then click the "push" blue arrow and off it goes, no errors and appears in my DevOps repo. great! but the problem seems to be that the whole lot syncs, I mean the whole application which then starts off the installation set up of Umbraco. Its odd, its almost as if one small change to the code, fires the DevOps pipeline and pushes the whole application to Azure.

Is this something to do with the CI (continuous Integration), bit of DevOps pipelines?

I am totally sure that all I am sending from VS is one file to my Azure DevOps repo. It then fires and overwrites the whole application. Which of course I do not want to happen, just want to make small dev changes to files, add files and send them to DevOps

Just too add to the confusion, if I FTP in to the application I seem to have a whole different set of files, which suggests its is being built as well

Newbie alert

# Answer
> 0 votes
You need a little more understanding of Git and Continuous Integration (CI) systems. The main goal of a CI system is to automate the building of your project or code when it receives notification that a change has occurred in your git repository. But there are nuances that can be added so a specific change to the code repository so it doesn't start the CI process.

First, understand the .gitignore file. It is used to prevent specific files to be checked into your git repository.

Second, the build pipeline can be triggered for all or specific branches. So if you check in code into the `develop` branch and you only want the CI process to start when there are changes to the `master` branch, then you can configure the build pipeline to monitor the `master` branch only.

Third, you can filter the DIRECTORY within the git repository to monitor. So let's say that you have two folders: `/docs` and `/src`. You can have the CI build pipeline only monitor the `/src` directory and ignore everything else. So check-ins to the /docs will not set off the build pipeline.

Lastly, with Azure DevOps you can ignore the complete commit check-in IF you use `***NO_CI***` at the end of your commit message. So even if you have none of the other filters in place mentioned above, you can prevent the CI build pipeline from starting if you include the `***NO_CI***` string in your commit message.

---
Tags: continuous-integration, git, azure-devops, azure, azure-pipelines
---


# Question
Title: How to prompt when win_service service exists?
I'm trying to figure out the steps needed to get when or other conditionals to react when service is present on hosts targeted on my task.

With this I get report that service is on one of the nodes but how could I trigger a shutdown for the service, reboot for *only* this server and wait until service is again running.

### Task to run

```
  - name: Check Service by Display name | NameOfMyService
    win_service:
      name: name_of_my_service
    register: service
    ignore_unreachable: yes

  - name: Print status of Windows Service | NameOfMyService
    debug:
      var: service

```

### Output:

```
ok: [win_host] => {
    "service": {
        "can_pause_and_continue": false,
        "changed": false,
        "depended_by": [],
        "dependencies": [
            "Afd",
            "Tcpip"
        ],
        "description": "Apache/2.4.46 (Win64)",
        "desktop_interact": false,
        "display_name": "Name_of_the_service",
        "exists": true,
        "failed": false,
        "name": "Name_of_the_service",
        "path": "\"C:\\Path\\to\\service\Apache.exe\" -k runservice",
        "start_mode": "auto",
        "state": "running",
        "username": "LocalSystem"
    }
}

```

There is '"exists": true,' but how could I trigger notifier or something like that from the output.

All help appreciated :)

# Answer
> 1 votes
Use conditonals.

```
  - name: Check Service by Display name | NameOfMyService
    win_service:
      name: name_of_my_service
    register: service
    ignore_unreachable: yes

  - name: Print status of Windows Service | NameOfMyService
    debug:
      var: service

  - name: Frobnicate the service
    frobnicate:
      service: name_of_my_service
    when: service.exists | default(false)

```

---
Tags: ansible, windows
---


# Question
Title: How to get a list scaled down pods in OpenShift?
Besides making a list of pods and list of deployments and then comparing, is there a way in the cli and/or OCP console to list the scaled down pods?

# Answer
Running the `oc get hpa` command provides a table showing min/max and current replicas.

```
oc get hpa hpa-resource-metrics-memory

NAME                          REFERENCE                       TARGETS         MINPODS   MAXPODS   REPLICAS   AGE
hpa-resource-metrics-memory   ReplicationController/example   2441216/500Mi   1         10        1          20m

```

Running a `describe` on specific HPAs or `oc get event` allows you to see the downscaling events.

```
oc describe hpa hpa-resource-metrics-memory

Name:                        hpa-resource-metrics-memory
Namespace:                   default
Labels:                      <none>
Annotations:                 <none>
CreationTimestamp:           Wed, 04 Mar 2020 16:31:37 +0530
Reference:                   ReplicationController/example
Metrics:                     ( current / target )
  resource memory on pods:   2441216 / 500Mi
Min replicas:                1
Max replicas:                10
ReplicationController pods:  1 current / 1 desired
Conditions:
  Type            Status  Reason              Message
  ----            ------  ------              -------
  AbleToScale     True    ReadyForNewScale    recommended size matches current size
  ScalingActive   True    ValidMetricFound    the HPA was able to successfully calculate a replica count from memory resource
  ScalingLimited  False   DesiredWithinRange  the desired count is within the acceptable range
Events:
  Type     Reason                   Age                 From                       Message
  ----     ------                   ----                ----                       -------
  Normal   SuccessfulRescale        6m34s               horizontal-pod-autoscaler  New size: 1; reason: All metrics below target

```
> 3 votes

---
Tags: openshift
---


# Question
Title: Using SSH to open interactive session in ansible
Is there a semi-easy way to use Ansible to open an interactive session?

I've spent a good amount of time making use of keys and other credentials stores to make a stellar ansible inventory file. Problem is, when I want to get on the servers that I can easily manage, I've gotta go out and find those creds and take the time to log into them.

It's almost easier for me just to run adhoc commands with ansible instead of logging in via ssh. Is there a way to pause ansible with a running interactive session so I can get a command prompt? I noticed that sometimes it lets me accept host keys, but other times it bombs out, (I think this happens when running ssh proxy), so I know things can be done interactively.

# Answer
> 4 votes
You could try `ansinble-console`

It mostly feels like you just have a shell open, unless you use and ansible task name as your command, in which case prefixing it with `shell <whatever you wanted to run>` should work.

# Answer
> 3 votes
I recently made myself a small script that seems to do the work. It certainly can be done better but maybe you can get some inspiration:

```
#!/bin/bash

set -e

if [ -z "$1" ]
 then
    echo "No host patter supplied. Please provide a host pattern that would match only one host form the inventory."
    echo "Usage: ansible-ssh <host_pattern>"
    echo "Example: ./ansible-ssh services[0]"
    exit 1
fi

USER=sysadm
HOST_PATTERN=$1
HOST=`ansible -m shell -a "echo {{ groups.$HOST_PATTERN }}" localhost  | sed 1d`

if grep -q "\[.*\]" <<< "$HOST"; then
  echo "Host pattern $HOST_PATTERN matches more than one host: $HOST"
  exit 1
fi

echo "Connecting to $USER@$HOST..."
ssh $USER@$HOST

```

Just edit your default USER (in my case sysadm) and execute in the following way:

```
./ansible-ssh services[0]

```

Where `services` is your inventory entry holding one or more IPs.

# Answer
> 2 votes
Here is my solution, which processes variables that Ansible uses to connect hosts, specifically ansible\_ssh\_common\_args, and generates an SSH configuration file and wrapper shell scripts that they use. It consists of a single playbook and a Python script.

Detailed explanation: https://dmrub.github.io/ansible/ssh/2020/10/17/interactive-ssh-with-ansible.html

Source code on github:

https://github.com/dmrub/ansible-ssh-scripts-creator

---
Tags: ansible, ssh
---


# Question
Title: DNS resolution fails when container is started via docker-compose
I recently decided to try out windows containers and i am using my private minecraft server for that purpose (as a hobby project).

I have an image prepared, based on Windows Server Core 2022.

When i start the container manually, it works perfectly:

```
docker run -it -p 25565:25565 --mount type=bind,source="C:\MinecraftDocker\MinecraftServer",target=C:/MinecraftServer minecraft:winserver2022

```

(i am mounting the minecraft server folder to allow a persistent world and server config)

However, i usually use the following docker-compose file to simplify this process:

```
services:
  mc-server:
    build:
      context: .
      dockerfile: Dockerfile_server2022
    image: minecraft:winserver2022
    platform: windows
    container_name: MinecraftServer
    restart: unless-stopped
    ports:
      - "25565:25565"
    volumes:
      - type: bind
        source: C:/MinecraftDocker/MinecraftServer
        target: C:/MinecraftServer

```

Doing this, the container and server process inside it start up normally, but connecting to it leads to some error message claiming the minecraft authentication servers were down (they are nof course not). A nearly identical compose file work perfectly fine with linux containers (on the same host system).

The weird part is that i built that image that i use to start the contianer manually with the above mentioned compose file. It seems to me that, when starting the service via docker-compose, something is blocking communication from the container to mojangs servers, but not incoming connections from the clients (i can see the failed connection attempt and name of the connecting account in the servers console output). However, i can not for the life of me, figure out what. I have tried removing everything except the bare minimum (context, ports and volumes) from the compose file, but to no avail.

Are there any relevant differences in how docker-compose configures networking and firewalls on a container vs what docker run does? Specifically in interaction with windows server core?

**EDIT: DNS is at fault** I have done some further digging and found the URLs for the API-Servers the Minecraft server application talks to when verifying client identities. That led me on to discover that any DNS resolution from the container fails when it is started via docker-compose, but not when started via docker run. It can ping the relevant IP's just fine either way. Question is, why?

**EDIT2: Workaround** Turns out the docker container attempts to get DNS responses from the gateway configured in the network compose generates, and for some reason does only gets "server failed" responses. Conversely, docker run just grabs the hosts DNS config which is why that works.

When i override the docker-compose created container's primary DNS server by adding this into the service's config, it is suddenly able to resolve host names just fine:

```
dns: <DNS server IP>

```

While i would really like to find out why the container doesn't get a response from the network gateway, this is at least a usable workaround. Any more insights or ideas as to the root cause, or why this issue does not arise for linux containers launched via compose, would be appreciated, but my immediate problem is at least solved

# Answer
> 3 votes
# Workaround

While have not found the root cause for why this only affects the specific combination of windows containers and docker-compose, i want to add the workaround mentioned in the second edit for other peoples benefit:

I solved my problem by overriding the container's primary DNS server. After adding this into the service's config, it is suddenly able to resolve host names just fine:

```
dns: <DNS server IP>

```

Turns out the docker container attempts to get DNS responses from the gateway configured in the network that compose generates, and for some reason only gets "server failed" responses. Conversely, docker run just grabs the hosts DNS config which is why that works. Why only my windows containers are affected, i do not know.

Further reading:

DNS in the compose file reference

A guide on networking in compose

# Answer
> 0 votes
Maybe try this syntax, with quotes for the volume:

```
services:
  mc-server:
    build:
      context: .
      dockerfile: Dockerfile_server2022
    image: minecraft:winserver2022
    platform: windows
    container_name: MinecraftServer
    restart: unless-stopped
    ports:
      - "25565:25565"
    volumes:
      - "C:/MinecraftDocker/MinecraftServer:C:/MinecraftServer"

```

---
Tags: docker, docker-compose, dockerfile, windows
---


# Question
Title: How to exit a jenkins job if a script executing gets access denied?
I have written this script that performs db backup

```
#!/bin/bash

#For taking backup
DIR=/media/storage/backup/db_backup/
DATESTAMP=$(date +%d-%m-%y-%H-%M)
DB_USER=backup
DB_PORT=$2
DB_PASS='secret'
HOST=$1

if [[ `id -u` != 0 ]]; then
    echo "Must be root to run script"
    exit
fi

if test -z "$HOST"
then
  echo "HOST not passed."
  exit 1
fi

if test -z "$DB_PORT"
then
  echo "PORT not passed."
  exit 1
fi

# remove backups older than $DAYS_KEEP
DAYS_KEEP=7
find ${DIR}* -mtime +$DAYS_KEEP -exec rm -f {} \; 2> /dev/null
# create backups securely
umask 006
# list MySQL databases and dump each
DB_LIST=`mysql -h $HOST -u $DB_USER -p"$DB_PASS" -e'show databases;'`
DB_LIST=${DB_LIST##Database}
for DB in $DB_LIST;
do
 if [[ "${DB}" == "performance_schema" ]]
 then
  echo "Skipping database ${DB}."
  continue
 fi
 FILENAME=${DIR}${DB}-${DATESTAMP}.msql.${DB_PORT}.gz
 echo "Initiating backup of $DB for ${HOST} on port ${DB_PORT}"
 mysqldump -h $HOST -P $DB_PORT -u $DB_USER -p"$DB_PASS" $DB --single-transaction | gzip > $FILENAME
done

```

the script above is called from jenkins job.

How can I exit jenkins job if I get this error:

```
ERROR 1045 (28000): Access denied for user 'backup'@'172.118.1.21' (using password: YES)

```

# Answer
> 1 votes
The jenkins job will pass or fail based on the exit code of the script. You need to make sure you exit with a non-zero value when `mysqldump` gives you that error.

---
Tags: jenkins, shell-script
---


# Question
Title: Proper technologies to create a horizontably scalable index search
I am sorry for the newbye question. Im trying to figure out the best architecture / set of technologies to implement some very specific system:

1. I have several databases indexes (used for searching data elements) that are broken down into small pieces. Each datababase index is actually a serialized data structure of mine and can be composed by N index file pieces. All pieces of all databases are in a s3 bucket and can be properly identified (pieces of one same database can be found by filename pattern).
2. For searching into one specific database, we need to load all the N file pieces from s3 into RAM memory for fast search. But all pieces might not fit one machine RAM, so theoretically, each database can / will be distributed over several machines.
3. Once a query is requested for a specific database, we need to send the query to all machines that contains any piece of the database (some might contain more than one piece, no problem), and then the machines will returns if it was found or not.
4. Some database might get to be unused for a while and will get unloaded from machines (the database index file pieces will be unloaded from RAM).
5. As any database index piece is not loaded when a query arrives, it must be loaded into one of the service machines that stores the pieces in memory.

What I think it need to be done:

IDEA

1. A master node will contain a map of loaded/unloaded databases and which pieces are loaded in which machine.
2. A request will arrive to master for a specific database, and it will check if all pieces for that database are loaded, if not, it will request loading sending a command to one of the available machines.
3. More machines are put available as soon as memory of the whole set of service machines get low (auto scaling)
4. After all pieces are loaded in some machines, the master updates the map of pieces and send the query to all of the corresponding machines that contains the database pieces.
5. The master will also have timestamps or a LRU for the pieces so it can unload after some time not being used.

So for this solution we would have a master and service slaves. Slaves contains api to load/unload a piece and also for listing which piece it has loaded.

Drawbacks from this solution:

This solution involves a master node that will be a bottleneck and a context that reflects the service nodes and its stored in master. If master fails, how this context can be restored? Not so trivial. Possible fixes would be having lots of masters and many more service slaves. Also if a master dies it can recover the state by quering the slaves.

Can anyone give me any hint on the technologies I could use? I think amazon ECS can help.

Thanks in advance.

# Answer
> 0 votes
You might want to have a look at this video:

> "Why We Built Our Own Distributed Column Store" by Sam Stokes

It illustrates how you could build a highly scalable columnar data store using SSDs rather than loading everything into RAM. They got really high performance out of this solution (try the Honeycomb demo to see it).

It might fit your use case?

---
Tags: kubernetes, amazon-ecs, distributed-systems
---


# Question
Title: cap_add and device options not working in docker stack file
I am trying to start docker container on the rockpix node with cap\_add and device options but docker stack errors saying

```
$ docker stack deploy -c docker-stack.yml vpnproxy
services.vpn_proxy.deploy Additional property cap_add is not allowed
$ docker stack deploy -c docker-stack.yml vpnproxy
services.vpn_proxy.deploy Additional property device is not allowed

```

Below is my stack file.

```
version: "3.9"
services:

  vpn_proxy:
    image: ciasto/alpine-myvpn-proxy:v1
    volumes:
      - /etc/openvpn:/etc/openvpn
      - /etc/haproxy:/etc/haproxy
    networks:
      - vpn_proxy_frontend
    deploy:
      replicas: 1
      update_config:
        parallelism: 2
        delay: 10s
      restart_policy:
        condition: on-failure
      cap_add: NET_ADMIN
      device: /dev/net/tun
      placement:
        constraints:
          - "node.hostname==rockpix"

networks:
  vpn_proxy_frontend:

```

What am I doing wrong ?

# Answer
You're not doing anything wrong, `cap_add` and `devices` are ignored when deploying a stack in Swarm mode. See the Notes section of the following links:

https://docs.docker.com/compose/compose-file/compose-file-v3/#cap\_add-cap\_drop https://docs.docker.com/compose/compose-file/compose-file-v3/#devices
> 1 votes

---
Tags: docker, docker-swarm
---


title: Copy and unarchive Ansible Module

text:
# Question
Title: Copy and unarchive Ansible Module
I tried below ansible tasks with copy and unarchive modules and facing error

```
- name: find and archive
  hosts: localhost
  tasks:
    - name: find the file
      copy:
        src: "{{ item }}"
        dest: "/home/ec2-user/bb"
        remote_src: yes
      with_fileglob:
        - "/home/ec2-user/aa/Data*"
      register: result
    - name: show the result file path
      debug:
        msg: "{{ item.path }}"
      loop: "{{ result.files }}"
    - name: extract file
      unarchive:
        src: "{{ item.path }}"
        dest: /home/ec2-user/cc/
      loop: "{{ result.files }}"

```

the error is:

```
fatal: [localhost]: FAILED! => {"msg": "'dict object' has no attribute 'files'"}

```

# Answer
You are registering the output of the `copy` module in the `result` variable. When the copy module runs in a loop, it returns a `results` key that contains each item of the loop.

The copy module does not have a `files` return value. The return values are listed here. What you want to use is the `dest` value. The `dest` value is where you copied the file using the `copy` module, that will become your `src` value for the `unarchive` module.

You can adjust your playbook to use those variables like this:

```

- name: find and archive
  hosts: localhost
  tasks:
    - name: find the file
      copy:
        src: "{{ item }}"
        dest: "/home/ec2-user/bb"
        remote_src: yes
      with_fileglob:
        - "/home/ec2-user/aa/Data*"
      register: result
    - name: show the result file path
      debug:
        msg: "{{ item.dest }}"
      loop: "{{ result.results }}"
    - name: extract file
      unarchive:
        src: "{{ item.dest }}"
        dest: /home/ec2-user/cc/
      loop: "{{ result.results }}"

```
> 1 votes

---
Tags: ansible
---


# Question
Title: Error when renaming docker-compose service
I'm new to docker-compose but my intuition is that it should not matter at all how a service is named, like a variable in a programming language. Is that correct and is there some obscure bug or is there some magic in the background that I don't understand?

The following example from the bitnami phppgadmin repository only works if the service is named exactly "postgresql":

```
version: '2'
services:
  postgresql:
    image: docker.io/bitnami/postgresql:11
    environment:
      - POSTGRESQL_PASSWORD=bitnami
    volumes:
      - 'postgresql_data:/bitnami'
  phppgadmin:
    image: docker.io/bitnami/phppgadmin:7
    ports:
      - '80:8080'
      - '443:8443'
    depends_on:
      - postgresql
volumes:
  postgresql_data:
    driver: local

```

As soon as I rename the service to anything else, it breaks with a "login failed" in the phppgadmin interface with no further information neither on the page nor in the docker compose logs:

```
version: '2'
services:
  foo:
    image: docker.io/bitnami/postgresql:11
    environment:
      - POSTGRESQL_PASSWORD=bitnami
    volumes:
      - 'postgresql_data:/bitnami'
  phppgadmin:
    image: docker.io/bitnami/phppgadmin:7
    ports:
      - '80:8080'
      - '443:8443'
    depends_on:
      - foo
volumes:
  postgresql_data:
    driver: local

```

I am using `docker-compose down` and then `docker-compose up` to make sure there is no leftover container from a previous run.

# Answer
The service name, in this case `postresql`, represents the name of the host for other Docker containers to address. By switching the service name to `foo` you now have a container that can be addressed with that hostname.

by default `phppgadmin` looks for Postgresql to be running on a host named `postsgresql`, you can change this by passing the `DATABASE_HOST` environment variable to the `phppgadmin` container.

```
phppgadmin:
  image: docker.io/bitnami/phppgadmin:7
  ports:
    - '80:8080'
    - '443:8443'
  environment:
    - DATABASE_HOST=foo
  depends_on:
    - foo

```
> 1 votes

---
Tags: docker-compose
---


# Question
Title: Checkout shared library with tag with legacySCm
So trying to checkout shared library by tag. This works perfectly with moderSCM, but for reasons we need to use other way (legacySCM);

```
def lib = library identifier: 'jenkins-shared-library@41',
          retriever: legacySCM([$class: 'GitSCM',
          userRemoteConfigs: [[credentialsId: 'account',
         url: 'https://website.com.repo.git']]])

```

Doing that results to:

```
00:00:01.518  Loading library jenkins-shared-library@41
00:00:01.521  Selected Git installation does not exist. Using Default

```

However it does not checkout the tag "41". It just checkouts master. Why is the case?

# Answer
I think it should work and I don't know why it isn't.

It is unusual for someone to load a shared library dynamically using the `library` step instead of using the `@library` annotation. Since you are specifying a `retriever`, why don't you pass the tag to it?

You can find out all the options for `GitSCM` by going to the snippet generator and trying the `library` step.

This is what I came up with:

```
library changelog: false, identifier: 'jenkins-shared-library', retriever: legacySCM([$class: 'GitSCM', branches: [[name: '41']], extensions: [], userRemoteConfigs: [[credentialsId: 'account', url: 'https://website.com.repo.git']]])

```

but I personally like to structure my code a bit more

```
final Object lib = library(
    [
        identifier: 'jenkins-shared-library',
        changelog: false,
        retriever: legacySCM(
            [
                $class: 'GitSCM',
                branches: [
                    [
                        name: '41'
                    ]
                ],
                extensions: [],
                userRemoteConfigs: [
                    [
                        credentialsId: 'account',
                        url: 'https://website.com.repo.git'
                    ]
                ]
            ]
        )
    ]
)

```

I hope that specifying the tag via the GitSCM works for you.

I think you should drop by https://community.jenkins.io/ and post about what you are trying to do in your pipeline. When you load a library dynamically because you lose one of the main benefits which is that shared libraries arent subjected to script security, at least when they are loaded via the `@library` annotation.
> 0 votes

---
Tags: jenkins, jenkins-pipeline
---


# Question
Title: How to ensure data protection on remote server?
We have sensitive data from a customer and need to train a Neural Network for our customers.

However we have no severs to train the Neural Networks, so we want to rent GPU servers. There are many cloud services with affordable prices and we can get root access to them.

The problem is, that we aren't guaranteed that our data is safe on those servers, which is very important for our customer. We want to ensure that the server's hosting party has no also root access and could not theoretically copy our code and data.

Is there a way to run our Python code safe ?

We could make a binary of our Python program but it could be decompiled in data and code, right?

We could also install an virtual machine OS inside, but is that safe for our data ?

Is there a docker image for an encrypted OS , or does something like this exists ?

Is there a solution for our problem out there except of buying immense hardware ?

Thanks a lot for bringing in your experience and helping us!

# Answer
> 1 votes
First of all, all data you can compile can be decompiled if you really want its a question about time, so you are correct here.

I did not know where in the world you are, but I will recommend AWS, Azure, Google Cloud and some of the big players out there, its not cheap but here you are sure about the provider did not have your root access key.

If you lose your root key or SSH key you its will require you can create a snapshot of the VM disk and restore it for a new VM.

Both AWS, Azure and Google have a very high level of security and use a vary advance ACL for users so its your responsibility to have the security ofc.

I'm located in EU and working for a company where GDPR is very important, so I will recommend one of the big players, and read the data protection privacy. If needed contact a lawyer to review the company based on your use case.

If its still not securer enough for the data you working with, I will highly recommend locally hardware and here the customer shut pay for that hardware fee you spent based on your contract.

that's my recommend

---
Tags: docker, linux, python, servers, encryption
---


# Question
Title: How to get the classic Jenkins build status icons back?
From my POV the classic Jenkins build status icons were, and still are, a masterpiece of icon art:

All the others weren't, and still aren't, bad either. (How is the saying? "If it ain't broke, don't break it!"? :)

While I appreciate that the new ones are SVGs, this  looks more like a shuriken that killed my build, rather than a sun and this cloud  looks boringly simplified compared to the classic one, looks like being drawn in a 101 MS Paint class.

How can I get the classic icons back into my Jenkins instance? They're still all there in the `jenkins.war`.

# Answer
That icon change did not go over well. JENKINS-65124 was raised as a result.

The change did two things:

* It overhauled the how the images are consumed such that they are served up as SVG now. The new ones are now served up from: `${JENKINS_HOME}/war/images/build-status/weather-sprite.svg`, along with the build-status icons from `.../build-status-sprite.svg`
* It replaced the "Classic icons" from the Tango Pproject with new flat icons from jenkins-core-theme. The Tango icons were not incorporated into the weather-sprite.svg.

I was not a fan of the Classic icons (too convoluted for 48px or smaller) or new set (see Jenkins issue), so I took a stab at an alternate set, available attached to the issue as weather-jenkins.svg; just rename to `weather-sprite.svg` and drop into the exploded war location.

Your solution to get the Classic icons back would be to incorporate the SVG for the five Tango icons into `weather-sprite.svg` in place of the existing definitions. Unfortunately, the straight replace does not render properly, resulting in a loss of the gradient shading (which makes up most of the SVG definitions). I don't know enough about SVG to sort it out.

btw: cross-posting is usually frowned upon, but also reinforces my meta concern that no one knows where to post Jenkins issues w/o proper guidance.
> 4 votes

---
Tags: jenkins
---


# Question
Title: Scraping different Prometheus metrics at different rates
I am running a Tendermint network. I am using Prometheus to track various metrics, and to track node health. For performance metrics such as `tendermint_consensus_total_txs`, I want updates every second. For health metrics such as `go_gc_duration_seconds_sum`, I only want updates every minute.

How can I configure Prometheus to scrape metric A every second and metric B every minute? Or alternatively how can I configure it to only keep every 60th sample of metric B?

Frequently updating performance metrics is important for business reasons, but I need to avoid ballooning my storage costs by storing lots of samples for metrics I don't need that frequently.

# Answer
You can define a different `scrape_interval` in the scrape config of two different jobs: a job for *slow* polling and one for *fast* polling.

If your metrics are on different endpoints you can simply adjust which endpoint should be scraped more frequently than the other.

If your metrics are on the same endpoint, you can define the same endpoint in both job and use the metric\_relabel\_configs of your target to adapt the metrics you want to ingest:

On the *fast* job, keep only the metrics you want to keep

```
metric_relabel_configs:
- source_labels: [__name__]
  regex: go_gc_duration_seconds_sum
  action: keep

```

If the duplication of metrics between the jobs is an issue, you can drop them in the *slow job*:

```
metric_relabel_configs:
- source_labels: [__name__]
  regex: go_gc_duration_seconds_sum
  action: drop

```

And optionally rename the job label in the *fast* job to match the *slow* job (exercise left to the reader).
> 2 votes

---
Tags: prometheus
---


# Question
Title: Cant ssh into running container containing portainer through docker exec
I cant connect to my running container with portainer. I tried already executing `docker exec -it <CONTAINER-ID> /bin/bash`. Also changed the shell path many times (/bin/sh, /usr/bin/sh, bash). Every time I get message:

> OCI runtime exec failed: exec failed: container\_linux.go:344: starting container process caused "exec: "/bin/bash": stat /bin/bash: no such file or directory": unknown

Container was created with command:

```
docker run -d -p 8000:8000 -p 9000:9000 --name=portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer-ce

```

Can you tell me why I can't connect to this container only? Portainer web interface is working fine... I thought each container had access to shell, and basic tools such as `ls`

# Answer
> 2 votes
Containers may or may not include a shell, depending on how the container was built. Looking at docker hub it looks like the base image may be based on a "scratch" image, meaning that there may not be anything in the container other than the code needed to run the portainer service. The image layers here also suggest this is the case.

If you want a shell, you may want to use the `portainer/portainer-ca:alpine` tag. That should give you a container with a full alpine os (and the shell you are looking for). As another note, you may want to try using `/bin/sh` instead as not all images provide a bash shell.

# Answer
> 1 votes
While it is true that you can log in to most containers via /bin/bash or /bin/sh it isn't a general requirement to ship a working shell. The official portainer docker container does not have a shell for security reasons according to staff.

There are several things you can do. One would be to build your own docker container image with a Dockerfile and include a shell. If this is just for a one-off situation, you can also copy a shell (busybox looks like a good candidate) via `docker cp $(which busybox) portainer:/` followed by `docker exec -it portainer /busybox sh`. This assumes that your running portainer container is indeed called "portainer".

# Answer
> 0 votes
> I thought each container had access to shell, and basic tools such as ls

Only containers that have included a shell has support for a shell.

---
Tags: docker
---


title: how often Jenkins should be upgraded

text:
# Question
Title: how often Jenkins should be upgraded
Currently we have the version 2.200 installed and we want to upgrade it. I want to know what is the best way to do it. Thank you

# Answer
You should be using the LTS (point - ie; 2.204.6) \- for Production. Read the upgrade guides.

Read my prior advice.

In upgrading, I'd suggest take the latest release of each LTS in sequence (2.204.6 / 2.222.4 / 2.235.5, etc). You could chance skipping one if no significant changes.

Best to not fall more than 2 LTS behind. use a backup instance to upgrade and validate every bump.
> 2 votes

---
Tags: jenkins, jenkins-plugins
---


# Question
Title: Does `npm audit` add any value when using `dependabot`
## Context

There are multiple ways to scan projects for vulnerabilities.

Dependabot can be configured to check repositories for issues, and automatically submits pull requests to resolve.

NPM Audit will scan the packages used in an NPM solution for known vulnerabilities.

We're trying to work out whether, if Dependabot is enabled, there's any added value to using `NPM Audit` in our pipelines. I'm asking this solely from the perspective of what's detected; not how the tools work (i.e. whether they can cause a pipeline to block/fail).

## The actual question

Do both tools base their decisions on some common known-issue database, or is it common to see each tool detect different sets of problems?

# Answer
> 8 votes
Dependabot and `npm audit` both poll the Node Security Working Group database for Node-based projects. However, Dependabot has the added ability to check dependencies in numerous other types of projects as well.

Also, each report Dependabot generates includes useful info and links directly to a GitHub Advisory Database listing (e.g., CVE-2017-16021) that itself has multiple links to other listings such as NIST. On the other hand, `npm audit`'s reporting is limited to a single link to an npm advisory listing with far less info (e.g., #100).

Overall, Dependabot covers all the bases — and more — that `npm audit` does. No need to use both unless you need a customized audit flow in your package scripts.

---

## Update

As of 7 October 2021, both methods provide info from the GitHub Advisory Database. From the links above, you can see that former npmjs.org listings now redirect to GitHub. The other points I made still stand after this change.

# Answer
> 3 votes
As per October 2021:

**GitHub Advisory Database now powers npm audit** : "This means that every version of the npm CLI that supports security audits is now talking directly to the GitHub Advisory Database" - The GitHub Advisory Database is what powers **Dependabot** and now also *npm audit*

https://github.blog/2021-10-07-github-advisory-database-now-powers-npm-audit/

---
Tags: security, node.js, dependency-management, npm
---


# Question
Title: How to mount EC2 intances to AWS EFS via Terraform
I'm trying to Terraform a few EC2 instances, and would like to automate the provisioning steps that involve mounting their filesystems to EFS. The commands to run this manually are as follows:

```
$  sudo mkdir /var/www/html/efs-mount-point
$  sudo mount -t nfs -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport ${module.efs_mount.file_system_dns_name}:/ /var/www/html/efs-mount-point

```

As you can see from my string interpolation, I have already attempted to set things up in a Terraform `provisioner "remote-exec"` block when the EC2 instances get created.

This works for the most part, as the directory gets created - but I'm running into errors with the actual mounting step. I believe that Terraform may be trying to mount before everything has established proper network connections. I'll include the full error below.

**Is this the right approach?** I can't help but feel like I could set up the EC2 AMI to be already mounted or have some startup mount script instead of relying on Terraform provisioners (as they are noted as a last-resort approach in the docs)

*Networking Error for context:*

```
│ Error: remote-exec provisioner error
│
│   with aws_instance.example-instance-with-efs[0],
│   on main.tf line 72, in resource "aws_instance" "example-instance-with-efs":
│   72:   provisioner "remote-exec" {
│
│ error executing "/tmp/terraform_1760279708.sh": Process exited with status 32

```

# Answer
> 0 votes
A better approach I came to discover was using the `user_data` of a launch configuration. While it still suffers some of the same issues that provisioners introduce, there's a lot less room for error in terms of permissions and timing as seen in other questions.

For Example:

```
resource "aws_launch_configuration" "sample" {
  image_id          = "ami-04afe279c8bff9ed8"
  instance_type = "t2.micro"
  security_groups = [
    aws_security_group.efs.id,
  ]

  user_data = <<-EOF
              #!/bin/bash
              mount -t nfs -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport ${module.efs_mount.file_system_dns_name}:/ /your/mount/point/
              EOF
  lifecycle {
    create_before_destroy = true
  }
}

```

I should note, that this also implies my additional use of an Auto Scaling Group - which may or may not fit into everyone's requirements.

---
Tags: amazon-web-services, terraform, amazon-ec2, amazon-efs
---


# Question
Title: Gitlab pipeline + Jump host + execute HELM
What is worked:

Steps:

* directly go to server via SSH (behind the JH)
* via SSH to another server
* pull from Artifactory for testing only
* executing HELM chart to pull from artifactory and deploy on k8s WORKS

What I want todo:

Steps:

* run CI/CD
* I already pushed image to artifactory (works)
* execute HELM Charts that already worked similar on remote host, but from pipeline GitLab

What I am confused

* I need bypass JH from GitLab (i can check a host)
* Running helm chart from pipeline to execute on remote host to deploy k8s

What is here best practice if I am on via JH?

* use SSH then execute
* bastion host?

# Answer
Recommended way is to have an agent on the target host that would do the deployment.

Possible options include Argo or Flux. Generally, ssh'ing from CICD pipeline into target host especially with Bastion or Jump Host infrastructure is insecure.

Note that it is recommended to push the helm chart itself to the registry as well and treat it same way as container image artifacts.

For an end-to-end example of how such agent pattern would work consider my detailed tutorial here - https://worklifenotes.com/2021/05/22/helm-cd-with-reliza-hub-tutorial/

Note, it's using simplified agent that we wrote. This is fine for some cases, but it's also trivial to reproduce similar approach with either Argo or Flux.
> 0 votes

---
Tags: kubernetes, gitlab, cicd, helm
---


# Question
Title: Unable to configure cert-manager. Wrong status code '403', expected '200'
I', trying to configure a cert-manager in a Baremetal server. I followed this link & this one

the `k describe challenge my-domain.com-xqwh4-4005106243-2059835058`

get

```
Status:
  Presented:   true
  Processing:  true
  Reason:      Waiting for HTTP-01 challenge propagation: wrong status code '403', expected '200'
  State:       pending
Events:
  Type    Reason     Age    From          Message
  ----    ------     ----   ----          -------
  Normal  Started    2m27s  cert-manager  Challenge scheduled for processing
  Normal  Presented  2m26s  cert-manager  Presented challenge using HTTP-01 challenge mechanism

```

the `k describe order my-domain.com-xqwh4-4005106243` has `State: pending`

the `k describe certificate my-domain.com`

```
Status:
  Conditions:
    Last Transition Time:        2021-10-11T01:30:50Z
    Message:                     Issuing certificate as Secret does not exist
    Observed Generation:         1
    Reason:                      DoesNotExist
    Status:                      False
    Type:                        Ready
    Last Transition Time:        2021-10-11T01:30:50Z
    Message:                     Issuing certificate as Secret does not exist
    Observed Generation:         1
    Reason:                      DoesNotExist
    Status:                      True
    Type:                        Issuing

```

If a call directly the http://my-domain.com/.well-known/acme-challenge/xxx-xxx-xx-NSMHzTtTCneahED5Ns7HpTfABow it works well

Any idea how to troubleshoot it? I tried all the things from https://cert-manager.io/docs/faq/troubleshooting/

the cert ingress

```
kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
  name: cm-acme-http-solver-plvqx
  generateName: cm-acme-http-solver-
  namespace: default
  uid: 9a9f6829-3b89-4990-9038-f45285a4ae92
  resourceVersion: '3155628'
  generation: 1
  creationTimestamp: '2021-10-11T01:30:52Z'
  labels:
    acme.cert-manager.io/http-domain: '1019414173'
    acme.cert-manager.io/http-token: '902303313'
    acme.cert-manager.io/http01-solver: 'true'
  annotations:
    nginx.ingress.kubernetes.io/whitelist-source-range: 0.0.0.0/0,::/0
  ownerReferences:
    - apiVersion: acme.cert-manager.io/v1
      kind: Challenge
      name: my-domain.com-xqwh4-4005106243-2059835058
      uid: 03bd05a0-d39d-4377-ad2a-43963b4ea9b9
      controller: true
      blockOwnerDeletion: true
  managedFields:
    - manager: controller
      operation: Update
      apiVersion: networking.k8s.io/v1
      time: '2021-10-11T01:30:52Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:metadata:
          f:annotations:
            .: {}
            f:nginx.ingress.kubernetes.io/whitelist-source-range: {}
          f:generateName: {}
          f:labels:
            .: {}
            f:acme.cert-manager.io/http-domain: {}
            f:acme.cert-manager.io/http-token: {}
            f:acme.cert-manager.io/http01-solver: {}
          f:ownerReferences:
            .: {}
            k:{"uid":"03bd05a0-d39d-4377-ad2a-43963b4ea9b9"}: {}
        f:spec:
          f:ingressClassName: {}
          f:rules: {}
    - manager: nginx-ingress-controller
      operation: Update
      apiVersion: networking.k8s.io/v1
      time: '2021-10-11T01:31:13Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:status:
          f:loadBalancer:
            f:ingress: {}
      subresource: status
spec:
  ingressClassName: nginx
  rules:
    - host: my-domain.com
      http:
        paths:
          - path: >-
              /.well-known/acme-challenge/mIHD-FglhGgu-QV-NSMHzTtTCneahED5Ns7HpTfABow
            pathType: ImplementationSpecific
            backend:
              service:
                name: cm-acme-http-solver-22qx2
                port:
                  number: 8089
status:
  loadBalancer:
    ingress:
      - ip: 137.100.90.100

```

# Answer
I have less reputation to add this in comment, are you sure your domain.com is resolving to said IP, ,and any .htacess or any rule preventing it, sometimes it take time to resolve and it route to old domain.com
> 0 votes

# Answer
Thanks for all the help. the Issue was solved with a workaround https://github.com/jetstack/cert-manager/issues/4003#issuecomment-904420841

basically the cert manager pod was unable to access the service pod. Seems a dns or something blocking the http call, redirecting to https and the cert-manager fails

So to avoid go-out and go-in again

```
k -n cert-manager edit deployment cert-manager

```

and add in `spec`

```
spec:
  ...
  hostAliases:
    - hostnames:
      - something-do.com
      ip: 138.178.91.111 # server IP

```

this will add that line to the `/etc/hosts` so will call directly to your service
> 0 votes

---
Tags: kubernetes, cert-manager, letsencrypt
---


title: evenly distribute pods over nodes

text:
# Question
Title: evenly distribute pods over nodes
I've got an on-premise cluster with 1 master and 3 workers. Now there's a deployment with 3 replica's and they MUST be evenly distributed over all the nodes.

Now we can use "topologySpreadConstraints" on kubernetes/hostName label for that but it doesn't always work.

```
apiVersion: apps/v1
kind: Deployment
metadata:
 name: revproxy
 labels:
  app: proxy
spec:
 replicas: 3
 selector:
   matchLabels:
     role: proxy
 template:
   metadata:
     labels:
       role: proxy
   spec:
     topologySpreadConstraints:
     - maxSkew: 1
       topologyKey: kubernetes.io/hostname
       whenUnsatisfiable: ScheduleAnyway
       labelSelector:
         matchLabels:
           app: revproxy

```

There's many times 2 pods running on the same node and 1 pod on another and zero pods on the 3rd. But when manually deleting a second pod then it ALWAYS gets scheduled on the empty node.

How can I use topologySpreadConstraints to have pods evenly distributed over the nodes?

# Answer
Found the answer.

When the deployment has a yaml with the following labels.

```
kind: Deployment
metadata:
 name: revproxy
 labels:
  app: proxy
spec:
 selector:
   matchLabels:
     role: proxy
 template:
   metadata:
     labels:
       role: proxy

```

The matchLabels in topologySpreadConstraints with matchLabels 'revproxy' is wrong.

```
topologySpreadConstraints:
     - maxSkew: 1
       topologyKey: kubernetes.io/hostname
       whenUnsatisfiable: ScheduleAnyway
       labelSelector:
         matchLabels:
           app: revproxy

```

When pointing to just 'proxy' in the labelSelector the pods get distributed over the 3 nodes evenly.

```
     matchLabels:
       app: proxy

```

Am still confused about the 3x the same labels in a description, changing any of these breaks the deployment, but at least it works.
> 1 votes

---
Tags: kubernetes, azure-kubernetes-service
---


# Question
Title: Is there a way to run the Jenkinsfile from command line?
I don't want to use Jenkins UI, instead I would like to use the command line, from command line I want to execute Jenkinsfile to the Jenkins server.

Is this possible?

Let's say I have a Jenkinsfile, If I run the Jenkinsfile, It shoud execute the Jenkinsfile from Jenkins server. Assume that the job not yet configured through Jenkins UI.

# Answer
> 3 votes
I've done this before. Essentially you can use the Jenkins CLI to import/export jobs, which will allow you to create and/or update a job with the definition you want to run. Be it JenkinsFile or some groovy script or whatever. Check this blog for details.

Once you (re)define the job, you will simply remotely execute it using the same Jenkins CLI. You can check the progress of the job from CLI and even read the logs to see progress. I cannot share the code, as the copyright is owned by one of my previous employers, but it is easy to recreate.

# Answer
> 2 votes
I assume you would like to run locally to test logic? If so there is a unit testing framework for Jenkins declarative / scripted pipeline. I have not personally used it but this would enable me to test stubbed logic locally.

Alternatively, as you're no ta fan of GUIs you could have a look at Jenkins X which has a very powerful CLI and is more gitops driven.

# Answer
> 0 votes
you can try using that tool, it allows to run scripts from the command line and some other thinkgs:

https://github.com/jenkinsci/jenkinsfile-runner

---
Tags: jenkins-pipeline, jenkins-plugins, jenkinsfile, jenkins2
---


# Question
Title: What is the maximum expiration duration for Azure SAS token created to connect to the Azure IoT Hub?
I want to send messages to the **Azure IoT Hub** using the **MQTT** protocol. I followed this video and managed to connect my Linux-based device (a **Raspberry Pi**) to the IoT Hub.

As instructed in the aforementioned tutorial, I used the **Azure IoT Explorer** to generate a connection string with SAS token.

However I noticed that the user is prompted to define an expiration for the SAS token, given in minutes (second box from the bottom in the image below):

I would like to have my device send messages to Azure over extended periods of time (*i.e.* several weeks). How can I specify such a long duration?

I tried passing 20,000 minutes (~ 1 month) as an argument. I was able to generate a seemingly valid **SAS token connection string**, but then again, if I pass an aberrant value (*e.g.* 0 or -1) I am still able to generate a token. Although my token is currently working, I am concerned that it will expire after a default "maximum expiration duration".

So what is the maximum duration that can be passed in the **Expiration (minutes)** field?

**EDIT:** according to this SO post, the maximum expiration date for the SAS token is 365 days:

> You can specify the expiration date for a SAS token, I think the maximum is 365 days, but forever is not a possibility.

# Answer
Technically there is no limit according to various documentation I've looked at.

There may be some limitations imposed by the UI you are using (typically 1 year as you've seen), but if you generate the token with the CLI or programmatically there should be no upper limit within reason (a large enough number could exceed the data capacity of various things, but a hundred years is reported to work fine).

Another limit is one imposed by security best practices, but I don't believe that's within the scope of this question.
> 0 votes

---
Tags: azure, authentication
---


# Question
Title: Why do I need a Nat Gateway for ECS containers using AWSVPC?
So here is a scenario:

I have a route table that routes all traffic to the internet gateway, and I have a subnet associated with that route table, which makes it a public subnet. Now I create another route table and I route all its traffic to a Nat gateway that I created I created in the public subnet abovesubnet. Now I create another subnet in this private route table, and associate that with my ECS containers. So in effect, this is how the traffic is being routed:

But what is wrong with doing this:

Why doesn't associating it with the internet gateway directly work? The first one seems like the same thing but with extra steps. I would really like to know why exactly that is necessary, and what happens "under the hood" in both cases, and what causes the second way to not work.

# Answer
While this is completely possible, it would mean that your instance would be accessible from the internet. Public subnet means that your instances are accessible directly from the outside. Private subnet are not accessible from the outside as instances do not have a publicly routable IP.

In short, security
> 3 votes

# Answer
When you use awsvpc for your ECS containers, those containers can't connect to a internet gateway directly since awsvpc can't provide public IP addresses to the ENIs of the containers despite them sitting in a public subnet (https://docs.aws.amazon.com/AmazonECS/latest/bestpracticesguide/networking-outbound.html).

Thus you need to use a NAT gateway to translate from private IP address to a public IP address.
> 1 votes

---
Tags: amazon-web-services, amazon-ecs, aws-vpc
---


title: Does Prometheus log scrape-duration?

text:
# Question
Title: Does Prometheus log scrape-duration?
I'd like to graph *scrape duration* for a particular target.

The Prometheus web interface has a Status of Targets page, which tabulates the (time elapsed since) "last scrape" and corresponding "scrape duration". Does Prometheus itself publish metrics such as scrape count and cumulative duration, or are scrape durations logged someplace else?

# Answer
Yes, you can use the scrape\_duration\_seconds metric.

Then you can run queries in the Prometheus query box like:

```
scrape_duration_seconds{instance="localhost:9100", job="node"}[5m]

```

or use Grafana.

To count the (incremented) number of scrapes served by the exporter, I believe what you want is the promhttp\_metric\_handler\_requests\_total metric. The metric is returned multiple times along with resulting HTTP status code: 200, 500, 503, possibly others. Successful scrapes have code 200. Other status codes will typically show a count of zero.

If you are only interested in counting successful scrapes, you could do something like this:

```
promhttp_metric_handler_requests_total{instance="localhost:9100", job="node", code="200"}

```
> 3 votes

---
Tags: prometheus
---


# Question
Title: Ansible Playbook always interactively asks for Become password, even though no become user is specified?
I have a simple ansible playbook that copies a script and executes a script on an EC2 instance.

```
- name: Playbook Deploy jar
  hosts: all
  remote_user: ubuntu
  tasks:
    - name: Copy Provisioning Script
      copy:
        src: ./prod-setup.sh
        dest: .
        mode: '0755'

    - name: Execute the Provisioning script
      command: sh ./prod-setup.sh

```

The .pem file used to connect to the host is specified in the 'Hosts' file. There is no become user but everytime I run it, I am interactively queried for a become password.

```
$superuser: ansible-playbook -i hosts -K playbook.yml
BECOME Password:

```

No Become password is set because there is no become user so literally pressing 'Enter' is sufficient to continue execution but this means I can't deploy this playbook as part of a script.

I checked ansible.cfg and none of the settings have been overridden. There are no other config files. What else could be causing this? Is there a way to circumvent this?

# Answer
You are explicitly asking to be prompted for the privilege escalation password by running `-K` which according to the docs is:

> -K, --ask-become-pass ask for privilege escalation password

Try running the `ansible-playbook` command without the `-K`
> 2 votes

---
Tags: ansible
---


# Question
Title: Describe the machine and service infrastructure with a diagram
We have a provider that manages a Baremetal server where there are many VPSs inside and each VPS manages some services.

We would like to have the infrastructure described in terms of machines (servers, disks, etc.), services and communications, so as to facilitate a possible change of supplier in understanding the system.

Are there any standards in this regard?

# Answer
> 1 votes
Adding on my comment, I can only share my personal experience and what worked with our clients in the past.

1. You have to have a Diagram with an at least high overview on how the services will work: Example Client\>Internet\>Load Balancer\>EC2\>Web Server\>Database and etc. There are many programs that you can use for that, example draw.io or Lucidchart(I prefer Lucidchart for the UI and it has a free option).
2. Some price ranges and estimations, the simplest way I used was make an estimation in Azure Calculator and export it in CSV. Make some highlights that you think are important to know and any additional calculations based on your services(for example what they pay you for XX of time work for 1 year).
3. Make a few estimations. What we did as an estimation thats for the best price, but does not ensure the best performance(basically based on the app and load the Minimum Requirements). And an estimation for best performance, where the price is high but you know thats going to hold a lot of load.

These are 3 major components IMO. We also used a detailed list of the services we provided and "plan" type which you can use. For example plan 1 includes 1 service, plan 2 includes 5 and etc(basically like a subscription).

I am explaining it like Im the one providing you a service and letting you know what you will get. In your specific case, you can ask all the above =\> Prices for each component and Server size, disk size and etc. Where all the connection goes, the list of VM/Servers and their specifications. What is used for what and so on.

In a few words, the best practices is to know what you are paying for and how it is being used. In what details you want that is fully up to you. And you as a client should have the full right to do that. If you wanted from me for example to see how much traffic you get, I would give you that with no problems.

---
Tags: virtualization, infrastructure
---


# Question
Title: [Multiple projects, one Azure subscription], Terraform - feasibility of distributed configuration
I know terraform a bit, but in the past I worked just with one terraform repository serving all infrastructure (with modules per provider/function).

```
infrastructure
   |- modules
   |   |- gcp
   |       |- main.tf
   |       |- variables.tf
   |   |- vsphere
   |       |- main.tf
   |       |- variables.tf

```

---

Now I have following situation:

* one Azure subscription
* Need for multiple separated infrastructure "buckets" - per 2 different projects, and I would like to separate test/dev (which should be more ephemeral) from production servers (long life).
* Need to hand over the infra per project in the future

Is it convenient and feasible to have:

* a bit of terraform code in each project repository, to keep it together
* many tfstates on same Azure subscription (one state per each infra bucket) that are isolated on Azure side by (resource groups?)

Any advice Guys?

# Answer
> 0 votes
You may want to use Terraform Cloud this will allow you to publish your modules from the project repos to a registry and you can then consume these modules from another repo containing the terraform that you use for the IaC deployments you then need to work out what the pipeline triggers for this repo will be.

Terraform cloud will also give you remote storage of the tfstate files and you will need a state per environment that you deploy. Azure DevOps and GitHub (and probably all other CI/CD providers) let you define environments and associate variables with them so you can use these to define which state file and resource group to target along with any other environment specific values you may need to use.

---
Tags: terraform, azure, terraform-module
---


# Question
Title: What are minimum permissions required for an IAM user to provision EC2 instances?
I want to provision an EC2 instance with Ansible. So for that purpose I want to create a separate IAM user to keep things clear.

I can assign the full EC2 access to the user. But I don't want any other part of my account be affected if something bad happens.

What roles do I need to attach to the IAM user to be able to provision instances?

# Answer
**quick answer**: - The user will need the permissions to execute the ec2 run-instances command.

Explanation on how to do this:

First create a policy (not Role) that authorizes the EC2 RunInstances action (find it in the table in the middle of the link). The EC2 RunInstances action requires access to many resources and you need to define which resources to maintain least privileges.

Review this AWS document explaining RunInstances and resource permissions needed. It also includes some example policies.

Note: AWS doesn't allow attaching roles directly to users. Instead, the user would to assume the role where you have added your new policy to that role, or the user would be added to an iam user group that had your new policy added to the permission policies.

======

To double check this works, I followed these steps:

1. Created a simple iam user with cli access: userX
2. Created a new iam policy using the visual editor that can access RunInstances action with any resource (not recommended for you in production, but ok for testing) : EC2RunInstancesAllResourcesPolicy
3. Created an iam user group: RunInstanceAdminGroup
4. Added EC2RunInstancesAllResourcesPolicy to the RunInstanceAdminGroup user group
5. Added userX to the RunInstanceAdminGroup
6. ran the ec2 run-instances command from the cli using userX credentials and it succeeded!
> 1 votes

---
Tags: amazon-web-services, ansible, amazon-ec2, aws-iam, provisioning
---


# Question
Title: Kubernetes readinessProbe: do health-checks run forever?
I have some doubts about the HTTP readinessProbe:

1. Suppose that the probe fails several times (failureThreshold) and a pod is marked as NOT READY. Will k8s keep trying the readinessProbe forever and eventually mark the pod as Ready if it begins to respond correctly? Or a pod that is marked as Not Ready is left there forever without retrying?
2. In case the Not Ready pod is left there forever, will the K8s Deployment try to start new pods? Or I will be left with less pods available?
3. If a pod passes the readinessProbe, will k8s keep checking if it is ready and eventually (if it begins to fail) mark it as not ready in the future? Is that possible? Or a pod that is marked once as ready is ready forever?

# Answer
1 - yes, containers are left there forever. Think of readiness probe as a breaker, if the app gets overwhelmed it cuts the traffic to the pod until it's ready to get traffic again.

2 - no, by itself, readiness probes only stop/start traffic to a pod. To signal kubernetes that the pod needs to be restarted use liveness probes.

3 - yes, as per answer 1. Readiness probes are a way of letting kubernetes know that the pod can handle traffic or if they need a break.

**probe: startup probes - waits for the app to start**

use case: apps that need some time before start serving traffic, for example, building assets on start, pre-fetching assets, building caches, etc.

**liveness probes - checks if the app is healthy**

use case: check if the app is running, if the app has exited due to an exception and needs to be restarted, or if the app is running but can't serve requests

**readiness probes - checks if the app can handle traffic**

use case: apps that can arbitrarily stop receiving traffic, e.g. maintenance mode, data intensive apps that can degrade in performance and need a way to

You should use liveness and readiness probes always, use startup probes when your app takes more time than usual to start.

Example:

* python app with postgres database:
* liveness probe: get /health status 200
* readiness probe: get /health status 200, timeout 1s

the /health endpoint queries the database using a simple query (select id, email from user limit 1) and returns a json

if the pod loses connectivity to the database, the liveness probe will fail and the pod will be restarted. if the database is being overwhelmed by requests, the query will take more than 1s, the probe will mark the pod as not ready, preventing that additional requests reach the database and preventing it from progressing from overwhelmed to crashed. once the database is able to catch up and starts returning queries in less than 1s, readiness probe will mark the pod as healthy again and kubernetes will resume serving requests to the pod.
> 2 votes

---
Tags: kubernetes
---


# Question
Title: Kubernetes: when does a deployment kill its pods?
I know that a pod may die for various reasons. For example in case of a node failure the pod is lost.

However I can't find an answer to this question: **are there any cases where the pod is alive and it gets killed on purpose by the deployment that created it?**

The only thing I can think about is scaling (e.g. scale down). Are there any other cases?

Basically I would like to understand if a bare pod has usually a longer live compared to a deployment pod or if the probability to die is the same.

# Answer
A bare pod and is exactly the same as a pod created by a deployment, the difference is that the deployment automates several tasks that you would have to execute manually.

deployments, daemonsets, statefulsets, jobs and cronjobs are all pod managers, they all create/delete pods under the hood.

Let's say you have a service sending requests to a pod running nginx containers that serve your site and you want to update the new container image that has the new files.

steps:

1. create a new pod with the updated image
2. portforward to it and check if the pod is up and nginx is responding to requests
3. add the labels that the service will look for
4. delete the old pod

sounds good? that's exactly what the deployment will do if you just update the image directly on it.

You can have some resilience built into pods with pod lifecycle, which will attempt to restart the containers, but there's a limit to it, and managing changes and scaling pods manually is awful.
> 1 votes

---
Tags: kubernetes
---


# Question
Title: Set up Mongo Cluster across 2 data centers with writing enabled on both
We're planning to expand our software to the South East Asia region. A self-hosted Mongo Cluster is fully set up at China's AWS Data Center. How can we set up the MongoDB replicate set to AWS Singapore and allow writing on both regions and rely on MongoDB for asynchronously sync up data behind the scenes?

# Answer
> 2 votes
**At this point, it is not possible to have that exactly.**

1. You can enable async replication from one MongoDB instance to several read replicas.
2. You can have two writable instances, and write to the same collections but you can't write to the same documents on both servers. This is achievable by using sharding. But each server will have half of the data.

Since you are in aws, you can implement your own data synchronizer using serverless. It's not a great solution, but it gets the work done.

Let's call the mongdb running on China as mongo-cn and the MongoDB that will run on Singapore mongo-si

* Step 1: enable replication set on mongo-cn, this will not replicate data anywhere, but will create the oplog which will allow you to use change streams
* Step 2: create a fargate container that will have one job and one job only, read the change stream and publish all writes to sqs-si
* Step 3: create a lambda trigger on sqs-si that runs for every sqs message, this lambda will get the sqs message and write it to mongo-si
* Step 4: stop mongo-cn, create an ami from mongo-cn, clear the sqs-si and start the mongo-cn again
* Step 5: copy the mongo-cn snapshot to Singapore and start it as mongo-si
* At this point, mongo-cn is being replicated to mongo-si, now you need to create a second subscriber
* Step 6: create a second fargate subscribing to all changes in the changestrem from mongo-si and publishing those changes to sqs-cn
* Step 7: create a lambta trigger on sqs-cn that runs for every sqs message, this lambda will get the message and write it to mongo-cn

Now you have two writeable mongo db instances being asynchronously replicated across regions, you have buffered writes in sqs and some control/logging by using cloudwatch and lambda triggers.

You will also have zero consistency if the two regions decide to write to the same document at the same time.

This is not a good solution, but some scenarios might require it.

# Answer
> 0 votes
I'm assuming you are using Amazon DocumentDB for the MongoDB cluster.

As of today you can't have active/active configuration and asynchronous replication for DocumentDB. The closest feature for DocumentDB are global clusters which allow global reads, but not global writes.

Moreover, keep in mind that AWS China is entirely separated from the AWS Global Network. Even if there were a feature for active/active configuration, you probably would not be able to set it up between AWS China and other AWS regions.

---
Tags: amazon-web-services
---


# Question
Title: how to increase "Atom feed for all" default value
Inside any Jenkins job, I have the button "Atom feed for all" at the bottom of the page. It gives an XML with details about the last 10 builds. I need to increase it somehow to get more than 10. Is it possible? Do I require admin user for that?

# Answer
The behavior is hardcoded; you cannot change or configure it in vanilla Jenkins (you could conceivably configure this behavior by writing your own plugin). Specifically, the behavior is: as long as there are at least 10 completed builds in your job, the feed should always contain at least 10 builds. There will only be more than 10 builds in the feed if there have been more than 10 completed builds in the past 7 days.

From the source code, the rssAll page fetches all new builds using the newBuilds() method:

```
    public void doRssAll(StaplerRequest req, StaplerResponse rsp)
            throws IOException, ServletException {
        RSS.rss(req, rsp, "Jenkins:" + getDisplayName() + " (all builds)", getUrl(), getBuilds().newBuilds());
    }

```

and the newBuilds() method returns all completed builds from the past 7 days, or the 10 most recent builds if there are fewer than 10 builds in the past 7 days:

```
    /**
     * Reduce the size of the list by only leaving relatively new ones.
     * This also removes on-going builds, as RSS cannot be used to publish information
     * if it changes.
     * <em>Warning:</em> this method mutates the original list and then returns it.
     */
    public RunList<R> newBuilds() {
        GregorianCalendar cal = new GregorianCalendar();
        cal.add(Calendar.DAY_OF_YEAR, -7);
        final long t = cal.getTimeInMillis();

        // can't publish on-going builds
        return filter((Predicate<R>) r -> !r.isBuilding())
        // put at least 10 builds, but otherwise ignore old builds
        .limit(new CountingPredicate<R>() {
            @Override
            public boolean apply(int index, R r) {
                return index < 10 || r.getTimeInMillis() >= t;
            }
        });
    }

```
> 1 votes

---
Tags: jenkins
---


# Question
Title: Deliver docker container on the server with no internet
I have a project which is written in Python. After I push it's going to be pulled by another team and after that they are going to deliver the project to windows server machine wich has no internet. It is going to be host on the windows server machine which is only accessable via local intranet. Is it possible to run all the installations on docker container during the CI/CD process and deliver that 'ready' container (not just an image) to the host? I'm using GitHub if that is important

# Answer
> 5 votes
you can use the docker save command to save an image to a tar archive in your CI/CD pipeline:

```
docker save image-name > image-name.tar

```

Then you can send that tar to your team. They can install that image to the air gapped server with docker load

```
docker load --input image-name.tar

```

---
Tags: docker, cicd
---


# Question
Title: Execute python file existing inside image with Jenkins
I have an Image A, which has a python file "main.py". I want to know how I can use Jenkins to run that file from a different pipeline. I want to use Jenkins to go inside that image and then run that file which is inside Image A.

I tried using .withRun but I don't think I'm doing this right.

```
docker.image(ImageA).withRun() {
    sh "main.py ${parameters to pass}"
}

```

When I run it like this, it executes the local main.py that I have and executes that inside the container which isn't what I want. I want to execute the main.py that is already inside the container and pass in parameters.

Any ideas?

# Answer
I actually figured out the solution. In the ImageA pipeline, I added the 'main.py' to the artifacts and then in ImageA, I cloned 'main.py' and executed it.

ImageA:

```
always {
        archiveArtifacts artifacts: 'main.py', onlyIfSuccessful: true
    }

```

Current pipeline:

```
sh "curl {url to imageA artifact}/main.py -o main.py"
docker.image(ImageA).inside {

    sh "python main.py "

}

```
> 0 votes

---
Tags: docker, jenkins, jenkins-pipeline, groovy
---


# Question
Title: Can one run a docker container with systemd from a jenkins pipeline?
I have been unsuccessful running a docker container with systemd from a jenkins pipeline without getting the following error message:

```
ERROR: The container started but didn't run the expected command. Please double check your ENTRYPOINT does execute the command passed as docker run argument, as required by official docker images (see https://github.com/docker-library/official-images#consistency for entrypoint consistency requirements).
Alternatively you can force image entrypoint to be disabled by adding option `--entrypoint=''`.

```

Pipeline:

```
pipeline {
    agent {
        docker {
            image "centos/systemd"
            args "--entrypoint='/usr/sbin/init' -u root -v /sys/fs/cgroup:/sys/fs/cgroup --privileged"
        }
    }
    options {
        ansiColor('xterm')
        skipDefaultCheckout(true)
        timestamps ()
    }
    stages {
        stage('Test systemctl') {
            steps {
                sh ' systemctl' }
        }
        }
    }

```

It seems that jenkins is stealing any option of a `CMD` arg and putting in `cat`. Which seems foolish because `/usr/sbin/init` needs to be the first thing run. Ideally as the `CMD`.

# Answer
> 2 votes
> It seems that jenkins is stealing any option of a CMD arg and putting in cat

Indeed I think this *is* what Jenkins is doing. It comes down to how they have designed their docker support. The container is started with a program that does nothing, so that they can run various commands inside it, via `docker exec` mechanisms. This is good for most use cases, because the command itself can fail without taking out the whole container with it.

For your systemd user case, maybe you should consider to use the Jenkins sidecar container support: https://www.jenkins.io/doc/book/pipeline/docker/#running-sidecar-containers.

---
Tags: docker, jenkins, jenkins-pipeline
---