# Question
Title: Azure ARM Triggers (e.g. Resource Created, Resource Removed, etc)
I am trying to find a way to **trigger events on creation/removal of specific Azure Resources**.

My goal is to add and/or remove these resources information to an App Gateway.

Can that even be done? My alternative would be to use a scheduled solution but I would love to avoid that.

Thanks in advance,

# Answer
So you can use something like Azure Event Grid where you can listen to events from different types of Azure Resources and then that event could be listened to by a Webhook trigger which then can trigger your pipeline, see here:

https://docs.microsoft.com/en-us/azure/event-grid/overview
> 1 votes

---
Tags: azure
---


# Question
Title: Does a slot swap cause an application to restart when using deployment slot settings?
My web application in Azure has two deployment slots - staging and production.

The purpose of the application is to draw a shape. The version currently deployed to the production slot draws a square. The version deployed to the staging slot draws a circle.

Both versions of the application have an application setting called Colour which controls the colour of the shape. Importantly, this is a deployment slot setting - see the example capture below:

The value of this setting is blue in the staging slot and green in the production slot.

When I view the application in the staging slot, a blue circle is drawn. When I view the application in the production slot, a green square is drawn.

When I perform a slot swap from staging into production and view the application in the production slot, a green circle is drawn.

I am struggling to get my head around this. A big advantage of swapping deployment slots is that I am able to swap my pre-warmed staging code into the production slot and prevent users from suffering from downtime or cold-start performance issues. However, by swapping the staging application into the production slot which has different values in the deployment slot setting, haven't I made changes to the application configuration which will trigger a restart? If I was to modify web.config manually or update the deployment slot setting in Azure, the application in the slot would restart. How is slot swapping getting around this?

# Answer
if both are running 'warmed up' and in 'Always On' mode then the swap would not cause a restart of any kind. But go to this URL and read the point #1 and if any of these things are done then it will restart.

https://docs.microsoft.com/en-us/azure/app-service/deploy-staging-slots#what-happens-during-a-swap
> 0 votes

---
Tags: azure
---


# Question
Title: CI/CD ARM Templates - Should the cloud infrastructure and resources be checked on each deployment?
Given a company adopts infrastructure as code deployment model using ARM templates.

Is it a good practice to attempt to re-deploy the ARM templates alongside the application itself ?

If yes what deployment mode (incremental/substitution) should be used and why?

Thank you!

# Answer
if you are doing ARM then follow the replace ideology if you can manage to have things stateless but if not then maybe go via the Terraform route than ARM IMHO
> 1 votes

---
Tags: azure-devops, azure, infrastructure-as-code
---


# Question
Title: How many iops do I actually get in a General Purpose SQL Database in Azure?
I'm running a serverless SQL database in azure, taking a load from some data transformed by Databricks.

Every time I try to write to the database the, IOPS percentage shows up as 100% which makes it clear that this is my bottleneck.

In the database configuration tab, I can see that for general purpose I get 500-20000 IOPS provisioned. But how do I actually get and can I control this in any way?

# Answer
> 1 votes
Follow this article

But in serverless mode you will have to look at limits for the vCores your DB is using at that second, so if vCores are 4 then you will have limit of 1280 and Throughput of '5-7 ms (write) and 5-10 ms (read)'

---
Tags: azure, sql
---


# Question
Title: How to create a "silent" MySQL docker container?
How can I create a `MySQL` docker container as below in the background:

```
root:~#
root:~# docker run -e MYSQL_ROOT_PASSWORD=pass --name sql-db -p 3306:3306 mysql &
[1] 137878
root:~# 2020-09-01 08:25:52+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.21-1debian10 started.
2020-09-01 08:25:52+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2020-09-01 08:25:52+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.21-1debian10 started.
2020-09-01 08:25:52+00:00 [Note] [Entrypoint]: Initializing database files
2020-09-01T08:25:52.978020Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.21) initializing of server in progress as process 42
2020-09-01T08:25:52.986993Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.

root:~#
root:~# 2020-09-01T08:26:03.346159Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2020-09-01T08:26:29.360246Z 6 [Warning] [MY-010453] [Server] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.

root:~#
root:~# 2020-09-01 08:27:35+00:00 [Note] [Entrypoint]: Database files initialized
2020-09-01 08:27:35+00:00 [Note] [Entrypoint]: Starting temporary server
2020-09-01T08:27:35.640682Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.21) starting as process 89
2020-09-01T08:27:35.782142Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2020-09-01T08:27:37.207138Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2020-09-01T08:27:37.562152Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: /var/run/mysqld/mysqlx.sock
2020-09-01T08:27:38.036662Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2020-09-01T08:27:38.036911Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
2020-09-01T08:27:38.071799Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
2020-09-01T08:27:38.105387Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.21'  socket: '/var/run/mysqld/mysqld.sock'  port: 0  MySQL Community Server - GPL.
2020-09-01 08:27:38+00:00 [Note] [Entrypoint]: Temporary server started.
Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.

2020-09-01 08:27:51+00:00 [Note] [Entrypoint]: Stopping temporary server
2020-09-01T08:27:51.591046Z 10 [System] [MY-013172] [Server] Received SHUTDOWN from user root. Shutting down mysqld (Version: 8.0.21).
2020-09-01T08:28:09.444982Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.21)  MySQL Community Server - GPL.
2020-09-01 08:28:09+00:00 [Note] [Entrypoint]: Temporary server stopped

2020-09-01 08:28:09+00:00 [Note] [Entrypoint]: MySQL init process done. Ready for start up.

2020-09-01T08:28:10.181619Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.21) starting as process 1
2020-09-01T08:28:10.297757Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2020-09-01T08:28:11.863493Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2020-09-01T08:28:12.702126Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060, socket: /var/run/mysqld/mysqlx.sock
2020-09-01T08:28:14.231915Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2020-09-01T08:28:14.232156Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
2020-09-01T08:28:14.470976Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
2020-09-01T08:28:14.498996Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.21'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  MySQL Community Server - GPL.

root:~#
root:~#
root:~# docker container ls
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                               NAMES
8a63d8b9d9c2        mysql               "docker-entrypoint.s…"   2 minutes ago       Up 2 minutes        0.0.0.0:3306->3306/tcp, 33060/tcp   sql-db
root:~#
root:~# docker exec -it 8a63d8b9d9c2 bash
root@8a63d8b9d9c2:/#
root@8a63d8b9d9c2:/#

```

so that it's a bit less spammy?

(Yes, I could just switch to another terminal, using something like `tmux`, or route output to `/dev/null` of course.)

# Answer
You can run the container in detached mode by passing `-d` or `--detach` with the run command: https://docs.docker.com/engine/reference/run/#detached-vs-foreground
> 3 votes

---
Tags: docker, containers, bash, images, mysql
---


title: How to run multiple windows in cmd

text:
# Question
Title: How to run multiple windows in cmd
Wondering if possible to run multiple windows in command script. I have a nose.js file that automates an online process and wish to run it once in cmd so it has multiple windows doing the same thing for efficiency purposes. Please help

# Answer
node.js runs via a command line (no windows involved). If you want to run more than one copy of the same program you can simply start it more that once.

Take this simple one: https://nodejs.org/en/docs/guides/getting-started-guide/

Since it runs on a dedicated port (3000) if you start it more that once it will fail.

Here is something I found on google that details all the ways to do this: https://itnext.io/4-solutions-to-run-multiple-node-js-or-npm-commands-simultaneously-9edaa6215a93
> 1 votes

---
Tags: github
---


# Question
Title: git reflog is showing plain text password used
We are using Jenkins Freestyle Project to push the changes on the remote server. We are executing shell script on remote host using ssh for it. To pull the changes on remote host, we are using origin url with git username and git password. The credentials should not be visible in plain text in the url that's why we have stored them in variables using 'secret text(s) or file(s)' option of 'Build Environment'.

At the Jenkin's end it is working as it is expected. The git credentials are not visible to the users who are using Jenkins for other projects but we are facing issue on the remote server where project was deployed. The remote server is showing git credentials in plain text. Any user with ssh access of the remote server is able to run the git reflog command in the project directory.

Port 22 cannot be opened on the server where gitlab is deployed so we cannot use ssh keys method to create the build in Jenkins. We can use only http method to pull the changes.

Is there any way so we could implement to avoid showing the git credentials in plain text in the project directory.

# Answer
You could use the git credentials.helper feature https://git-scm.com/docs/gitcredentials
> 1 votes

---
Tags: jenkins, git, credentials
---


title: Force role session name in AWS IAM

text:
# Question
Title: Force role session name in AWS IAM
I want to enforce users to use their IAM username as the role session name when assuming a role in AWS. I've tried the following condition in IAM policies:

```
{
  "Effect": "Allow",
  "Principal": {
    "AWS": "arn:aws:iam::<redacted>:root"
  },
  "Action": "sts:AssumeRole",
  "Condition": {
    "StringLike": {
      "sts:RoleSessionName": "${aws:username}"
    }
  }
}

```

While this works fine when assuming a role as a user it does not work when assuming a role from a role with administrator privileges (`Allow * on *`). The only way to block this would be an explicit `Deny` when a role tries to assume a given role and does not have session name set up. Any ideas how to write a policy doing this? A simple `Deny` like below does not work because `aws:username` is not present when `Assumed Role` is the principal (see docs).

```
{
  "Effect": "Deny",
  "Principal": {
    "AWS": "arn:aws:iam::<redacted>:root"
  },
  "Action": "sts:AssumeRole",
  "Condition": {
    "StringNotLike": {
      "sts:RoleSessionName": "${aws:username}"
    }
  }
}

```

# Answer
I have not tried it but I think if you change your deny to a list of elements it will work.

```
{
  "Effect": "Deny",
  "Principal": {
    "AWS": "arn:aws:iam::<redacted>:root"
  },
  "Action": "sts:AssumeRole",
  "Condition": {
    "StringNotLike": {
      "sts:RoleSessionName": [
        "${aws:username}",
        "AnotherAllowedName"
      ]
    }
  }
}

```

This will allow an IAM user to assume the role if they use their username as the session name and it will also allow other Principals to assume the role if they use "AnotherAllowedName" as the session name but all other session names would be declined. I found this AWS blog post helpful when researching this answer
> 2 votes

---
Tags: amazon-web-services, aws-iam
---


# Question
Title: how to send a password to MySQL with docker?
Just getting started, want to run the **downloaded** version of `mysql`:

```
root:~#
root:~# docker images
REPOSITORY           TAG                 IMAGE ID            CREATED             SIZE
basex/basexhttp      latest              3339bf6bc898        4 days ago          226MB
twilio/twilio-java   latest              611ab7ece1cf        12 days ago         993MB
mysql                latest              0d64f46acfd1        3 weeks ago         544MB
jetty                latest              5f997007f18d        4 weeks ago         522MB
hello-world          latest              bf756fb1ae65        8 months ago        13.3kB
root:~#
root:~# docker run hello-world

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

root:~#
root:~# docker run mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=password
2020-09-01 06:53:03+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.21-1debian10 started.
2020-09-01 06:53:03+00:00 [ERROR] [Entrypoint]: mysqld failed while attempting to check config
    command was: mysqld -p 3306:3306 -e MYSQL_ROOT_PASSWORD=password --verbose --help
    Enter password: mysqld: Can not perform keyring migration : Invalid --keyring-migration-source option.
2020-09-01T06:53:03.366882Z 0 [ERROR] [MY-011084] [Server] Keyring migration failed.
2020-09-01T06:53:03.368621Z 0 [ERROR] [MY-010119] [Server] Aborting
root:~#

```

Without re-downloading the image.

# Answer
> 4 votes
As the documentation clearly states `docker run` command should be invoked as follows:

```
docker run [OPTIONS] IMAGE[:TAG|@DIGEST] [COMMAND] [ARG...]

```

so when you run

```
docker run mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=password

```

Docker assumes `-p 3306:3306 -e MYSQL_ROOT_PASSWORD=password` is a command you want to run inside the container, not a list of options for `docker run` command.

The proper `docker run` invocation with the parameters you specified is:

```
docker run -p 3306:3306 -e MYSQL_ROOT_PASSWORD=password mysql

```

---
Tags: docker, containers, images, mysql, passwords
---


# Question
Title: What is the difference between Jenkin's `Replay` and `Rebuild` options?
If you want to re-run a Jenkins job, you can click the `Rebuild` button which sometimes appears as the `Replay` button.

**What is the difference `Rebuild`ing a job versus `Replay`ing it?**

**Why is it that some jobs display `Replay` while others display `Rebuild`?**

# Answer
> 20 votes
Rebuild, using the Rebuilder plugin allows a completed job to be re-executed while allowing you to change the parameters, assuming your job is parameterized.

Replay does something similar, except it doesn't show you parameters. Instead, it shows the pipeline code directly. This is helpful when wanting to test changes iteratively as you can click Replay, change one or two lines of code, then let it execute. The job will run using the same parameters as it did the first time it ran/completed.

---
Tags: jenkins, jenkins-pipeline
---


# Question
Title: convert declarative pipeline to scripted pipeline
I already used the following declarative script as a part of my cleanup script, now I should convert it to the scripted pipeline, how I can convert the post to the scripted pipeline?

# Answer
> 2 votes
The scripted equivalent to the declarative

```
steps {
  // pipeline steps here
post {
  failure {
    // failure handler here
  }
  success {
    // success handler here
  }
  always {
    // always handler here
  }
}

```

is the native Groovy `try { } catch { }`:

```
try {
  // pipeline steps here

  // success handler here
} catch(Exception e) {
  // failure handler here
} finally {
  // always handler here
}

```

I'm not sure off the top of my head how to add an abort handler, but I think you might be able to do it by inspecting the `currentBuild.result` variable in the failure handler block.

---
Tags: jenkins
---


# Question
Title: Can we use Ansible for monitoring configuration?
I understand how Ansible works and what it is used for.

However, I was wondering if there is a way or feature (in open-source or enterprise edition) in Ansible using which, we can make sure that the configuration of a VM instance or server is always in sync with Ansible?

***For example:***

I used Ansible to install a few things (like MySQL server, spin up some docker containers etc.) on a Google Compute Instance. Now after some time I manually SSH into this VM and stop the docker container created by Ansible. Can Ansible be notified of this in order to create the container again?

So if Ansible sets configuration of a VM in ***state A*** and another user manually changes the state to ***state B***, is there a way that Ansible can be notified of this change and runs the playbooks again to set the configuration back to ***state A***?

# Answer
The simple answer is "yes" you *can* use Ansible for monitoring configuration, but you will have to do some extra work. As Vasily stated in their answer, Ansible does not have a built in triggering mechanism, so you need something to trigger the convergence of state. This can be done in several ways, the easiest being probably `ansible-pull`. As the docs state, `ansible-pull`

> ... inverts the default push architecture of ansible into a pull architecture, which has near-limitless scaling potential. The setup playbook can be tuned to change the cron frequency, logging locations, and parameters to ansible-pull. This is useful both for extreme scale-out **as well as periodic remediation.**

*emphasis added*.

Ansible is idempotent, so it *does* know about the state of target machines, contrary to what Vasily's answer states - it just doesn't persist it in the same way as Terraform does.

Your real problem does not sound like the question you are posing ("Can Ansible be used to ensure the state of a machine?"), but rather **"how can I detect configuration drift and trigger a remediation?"**

If you think of this as a monitoring problem, you can reframe the question as "how should I monitor my services in production?". Monitoring implies alarms on changes in state, which trigger actions to remediate. In your case, you would define an alarm on the health check of the service, when that is triggered, execute the playbook which converges to the desired state.

Since you have already invested time in writing the Ansible playbook, it would make sense to re-use it. The simplest would be to use that same playbook to invoke a cron which runs `ansible-pull` every 5 minutes or so, depending on your particular case.
> 2 votes

# Answer
Ansible is not state manager like Terraform but just automation tool. I think the only way to keep **state A** configuration is run ansible in cron. I saw such use case with chef automation tool.
> 0 votes

---
Tags: ansible, configuration-management, configuration, google-compute-engine, ansible-inventory
---


# Question
Title: Can't access startfile http://localhost/
Having created a docker container as:

```
docker run -d --name=jetty  jetty

```

How do I now get a hello world page as?

```
Looking up localhost:8080
Making HTTP connection to localhost:8080
Alert!: Unable to connect to remote host.

lynx: Can't access startfile http://localhost:8080/

```

and:

```
Looking up localhost
Making HTTP connection to localhost
Alert!: Unable to connect to remote host.

lynx: Can't access startfile http://localhost/

```

containers:

```
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
5379d3f41d20        jetty               "/docker-entrypoint.…"   4 seconds ago       Up 2 seconds        8080/tcp            jetty

```

# Answer
created with:

```
docker run -d -p 80:8080 -p 443:8443 jetty

```

gives a start page as:

```
Error 404 - Not Found.

   No context on this server matched or handled this request.

   Contexts known to this server are:

   Context Path Display Name Status LifeCycle
     __________________________________________________________________

   [1]icon  [2]Powered by Eclipse Jetty:// Server
     __________________________________________________________________

References

   1. http://eclipse.org/jetty
   2. http://eclipse.org/jetty

```
> 1 votes

---
Tags: docker, containers, images, servers, http
---


# Question
Title: How to gain root access to an existing docker container?
Able to get a response from the docker hub `basex` container:

```
nicholas $
nicholas $ lynx http://localhost:8984/ --dump
No function found that matches the request.
nicholas $

```

Ran the container as:

```
root $
root $ docker run -ti     -d     --name basexhttp     --publish 1984:1984     --publish 8984:8984     --volume "$(pwd)/basex/data":/srv/basex/data     basex/basexhttp:latest
71cbb84cf53143d00e6009d1ea5caa9de96c094c5d6aefc11cad95188b976776
root $
root $ docker container list
CONTAINER ID        IMAGE                    COMMAND                  CREATED             STATUS              PORTS                                                      NAMES
71cbb84cf531        basex/basexhttp:latest   "/usr/local/bin/mvn-…"   26 seconds ago      Up 23 seconds       0.0.0.0:1984->1984/tcp, 0.0.0.0:8984->8984/tcp, 8985/tcp   basexhttp
root $
root $ docker run -ti basex/basexhttp:latest bash
bash-4.4$
bash-4.4$ whoami
basex
bash-4.4$
bash-4.4$ pwd
/srv
bash-4.4$
bash-4.4$ sudo su -
bash: sudo: command not found
bash-4.4$
bash-4.4$ su -
su: must be suid to work properly
bash-4.4$

```

but how do I gain root access?

# Answer
Solution is to use `exec` as:

```
root@doge:~#
root@doge:~# docker run -ti     -d     --name basexhttp     --publish 1984:1984     --publish 8984:8984     --volume "$(pwd)/basex/data":/srv/basex/data     basex/basexhttp:latest
5699af1dc04aa9c78066222e4b0b8e5621aa2ab9dd2bd46bcd87c6a0377c1960
root@doge:~#
root@doge:~# docker container list
CONTAINER ID        IMAGE                    COMMAND                  CREATED             STATUS              PORTS                                                      NAMES
5699af1dc04a        basex/basexhttp:latest   "/usr/local/bin/mvn-…"   33 seconds ago      Up 30 seconds       0.0.0.0:1984->1984/tcp, 0.0.0.0:8984->8984/tcp, 8985/tcp   basexhttp
root@doge:~#
root@doge:~# docker exec -ti --user root basexhttp bash
bash-4.4#
bash-4.4# whoami
root
bash-4.4#
bash-4.4# su basex
~ $
~ $ whoami
basex
~ $
~ $ exit
bash-4.4# exit
exit
root@doge:~#
root@doge:~# docker container list
CONTAINER ID        IMAGE                    COMMAND                  CREATED              STATUS              PORTS                                                      NAMES
5699af1dc04a        basex/basexhttp:latest   "/usr/local/bin/mvn-…"   About a minute ago   Up About a minute   0.0.0.0:1984->1984/tcp, 0.0.0.0:8984->8984/tcp, 8985/tcp   basexhttp
root@doge:~#
root@doge:~# docker ps -a
CONTAINER ID        IMAGE                    COMMAND                  CREATED              STATUS              PORTS                                                      NAMES
5699af1dc04a        basex/basexhttp:latest   "/usr/local/bin/mvn-…"   About a minute ago   Up About a minute   0.0.0.0:1984->1984/tcp, 0.0.0.0:8984->8984/tcp, 8985/tcp   basexhttp
root@doge:~#

```
> 1 votes

---
Tags: docker, containers, databases, bash, images
---


# Question
Title: docker: Error response from daemon: Conflict. The container name "/basexhttp" is already in use by container
Am I even correctly following the documentation on creating and running a docker container below?

```
[root@ ~]#
[root@ ~]# docker run -d \
>     --name basexhttp \
>     --publish 1984:1984 \
>     --publish 8984:8984 \
>     --volume "$HOME/basex/data":/srv/basex/data \
>     --volume "$HOME/basex/repo":/srv/basex/repo \
>     --volume "$HOME/basex/webapp":/srv/basex/webapp \
>     basex/basexhttp:latest
4b827c082bebe45b8be5b7c956f1ada92407181aeec82dc5f71a63f2c5af537e
[root@ ~]#
[root@ ~]# docker run -ti \
>     --name basexhttp \
>     --publish 1984:1984 \
>     --publish 8984:8984 \
>     --volume "$(pwd)/basex/data":/srv/basex/data \
>     basex/basexhttp:latest
docker: Error response from daemon: Conflict. The container name "/basexhttp" is already in use by container "4b827c082bebe45b8be5b7c956f1ada92407181aeec82dc5f71a63f2c5af537e". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.
[root@ ~]#
[root@ ~]#
[root@ ~]# docker container list
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
[root@ ~]#
[root@ ~]# docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
[root@ ~]#
[root@ ~]# docker ps -a
CONTAINER ID        IMAGE                    COMMAND                  CREATED              STATUS                          PORTS               NAMES
4b827c082beb        basex/basexhttp:latest   "/usr/local/bin/mvn-…"   About a minute ago   Exited (1) About a minute ago                       basexhttp
[root@ ~]#

```

referencing:

https://hub.docker.com/r/basex/basexhttp/

https://docs.basex.org/index.php?title=Docker&mobileaction=toggle\_view\_desktop

where I've copied/pasted what I **think** are the relevant commands.

# Answer
> 5 votes
You're attempting to create two different containers, each with the same name. Each time you execute `docker run` it will create a container, and you've named the containers the same with `--name basexhttp`. These names must be unique, even among stopped containers. To delete an unneeded container, you can stop and remove it with:

```
docker container stop basexhttp
docker container rm basexhttp

```

(Note that `docker stop` and `docker rm` also work, but it's easier to read and understand the more verbose syntax)

---
Tags: docker, containers, databases, bash, images
---


# Question
Title: cannot get bash prompt: Error response from daemon: Container foo is not running
A basic `jetty` container:

```
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                                  NAMES
7c10b0ca16bd        jetty               "/docker-entrypoint.…"   21 seconds ago      Up 18 seconds       8080/tcp, 0.0.0.0:80->800/tcp, 0.0.0.0:443->8443/tcp   jetty01

```

can easily get a `bash` terminal as:

```
docker exec -ti jetty01 bash

```

created as so:

```
docker run -d \
    --name jetty01 \
    -d \
    -p 80:800 \
    -p 443:8443 \
    jetty

```

but, this approach fails with the `basex` from docker hub:

```
docker exec -ti basexhttp bash
Error response from daemon: Container 52b5337d7e6a22964310afb0be5f79ace38993173218fae8d9dcd858e4681142 is not running

```

created as:

```
docker run -d \
    --name basexhttp \
    --publish 1984:1984 \
    --publish 8984:8984 \
    --volume "$HOME/basex/data":/srv/basex/data \
    --volume "$HOME/basex/repo":/srv/basex/repo \
    --volume "$HOME/basex/webapp":/srv/basex/webapp \
    basex/basexhttp:latest

```

How do I get a root bash terminal?

How do I start the `basex` container? I tried:

```
docker start 52b5337d7e6a22964310afb0be5f79ace38993173218fae8d9dcd858e4681142

```

still nothing listed. But, `docker ps -a` shows:

```
CONTAINER ID        IMAGE                    COMMAND                  CREATED             STATUS                     PORTS               NAMES
52b5337d7e6a        basex/basexhttp:latest   "/usr/local/bin/mvn-…"   10 minutes ago      Exited (1) 2 minutes ago                       basexhttp

```

although it doesn't show in `docker ps` because it's not running.

# Answer
> 2 votes
Whoever created `basex/basexhttp:latest` does not seem to know Docker very well. They copied files necessary to a directory that was later configured as a volume, **which means it can only be bound to an anonymous volume, otherwise files inside will be removed**. They left source code inside the image, they used a single stage build with `maven:3-jdk-8-alpine` as base image, which means the final image has many unnecesary things installed. They left the entrypoint from the base image intact, which I guess is why they needed to create an empty `/srv/.m2` directory and set it as Maven config dir for the image to start. It switches user from `root` to `basex`, which causes permission problems with bind mounts. Generally it's a mess.

I see two ways to successfully run this image:

First:

```
docker run -d \
    --name basexhttp \
    --publish 1984:1984 \
    --publish 8984:8984 \
    --entrypoint "" \
    basex/basexhttp:latest /usr/local/bin/basexhttp

```

This uses only anonymous volumes and clears that unnecessary entrypoint.

You can stop it with `docker stop basexhttp` and then remove it with `docker rm -v basexhttp`. That `-v` is important - it removes the volumes - unless you memorize their very long IDs and which was mounted which you won't be able to reuse them anyway, so you may as well remove them from your system.

Second:

```
mkdir -p $HOME/basex/data
mkdir -p $HOME/basex/repo
mkdir -p $HOME/basex/webapp
chmod a+w basex
sudo chown -R 1984:1984 $HOME/basex
docker run -d \
    --name basexhttp \
    --publish 1984:1984 \
    --publish 8984:8984 \
    --volume "$HOME/basex/data":/srv/basex/data \
    --volume "$HOME/basex/repo":/srv/basex/repo \
    --volume "$HOME/basex/webapp":/srv/basex/webapp \
    --entrypoint "" \
    basex/basexhttp:latest bash -c "cp -r /usr/src/basex/basex-api/src/main/webapp/WEB-INF /srv/basex/webapp && exec /usr/local/bin/basexhttp"

```

This creates the necessary directories and sets their permissions so that both your user on the host and user.

You can stop it as in the previous example and remove with `docker rm basexhttp`. This time `-v` is not necessary, as there are no anonymous volumes.

You can verify that the contaienr is running correctly by executing:

```
docker exec -ti basexhttp basexclient

```

Username and password are both admin.

---
Tags: docker, containers, databases, bash, images
---


# Question
Title: how to start a docker container which dies immediately after startup?
On the understanding, from `#docker` on IRC, that the `basex` container from docker hub died after starting up, how do I now make it run something which doesn't immediately exit?

I'm only trying to get a `bash` prompt at this point:

```
docker run -ti     --link basexhttp:basexhttp     basex/basexhttp:latest basexclient -nbasexhttp

```

results in:

```
docker: Error response from daemon: Cannot link to a non running container: /basexhttp AS /charming_villani/basexhttp.

```

per the `basex` official doc's.

In trying to add docker logs I've tried:

```
sudo journalctl -fu docker.service

```

but will have to look into that a bit as there was no output.

---

trying a network as suggested:

```
root $
root $ docker network create -d bridge basexhttpnetwork
74c94db9195cf2b671123d016a9082f4aeacbbcef6ed91e4252744afba45d529
root $
root $ docker run -ti --name basexhttp --network=basexhttpnetwork basex/basexhttp:latest basexclient -nbasexhttp
docker: Error response from daemon: Conflict. The container name "/basexhttp" is already in use by container "2473f440e93855f967482aed405c30796a779d3592bd66a00fe48c2131b43a5e". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.
root $

```

# Answer
> 3 votes
The proper way to run this image, according to the README, is:

```
docker run -d \
    --name basexhttp \
    --publish 1984:1984 \
    --publish 8984:8984 \
    --volume "$HOME/basex/data":/srv/basex/data \
    --volume "$HOME/basex/repo":/srv/basex/repo \
    --volume "$HOME/basex/webapp":/srv/basex/webapp \
    basex/basexhttp:latest

```

No weird linking to itself is necessary.

If you wan to run a shell in an already running container (started as above) use:

```
docker exec -it basexhttp sh

```

or you can try bash

```
docker exec -it basexhttp bash

```

but the container is Alpine-based, so bash might not be installed.

If you want to run a new container and execute shell inside run

```
docker run -it --rm basexhttp sh
# or bash if it's installed
docker run -it --rm basexhttp bash

```

The above will remove the container once you exit bash (but the 3 volumes it creates will remain, you need to find and remove those by yourself).

Edit:

It seems README is wrong about how to start the container. See my answer to this question for more details.

# Answer
> 1 votes
Try something along these lines:

**First, create a network for the container:**

`docker network create -d bridge basexhttpnetwork`

**Then, run the container in that network:**

```
docker run -ti --name basexhttp --network=basexhttpnetwork basex/basexhttp:latest basexclient -nbasexhttp

```

# Answer
> 1 votes
First of all, `--link` is deprecated.

Secondly, you don't need the daemon logs, instead you need to take a look at what the container's logs are saying

```
docker logs <container name/id>

```

Once you have that, you'll need to work your way up in fixing whatever the issue there might be. This isn't `docker` related.

The first error message states that you're trying to link to a nonexistent container, and the second error pops up because even though the container isn't running (or failed), it's still there, and you can't use the same name for two containers. You need to remove the previous one using `docker rm <container name/id>`.

---

> how do I now make it run something which doesn't immediately exit?

You do that using the `--entrypoint` flag, like the following

```
docker run -ti --entrypoint bash nginx:latest

```

This won't start the nginx daemon, instead you'll be put into a bash prompt.

Make sure you're using the `-ti` flag if you want to use something like bash/ash, i.e. an interactive tty.

# Answer
> 0 votes
After random stumbling, this works:

```
root $
root $ docker run -ti -d \
>     --name basexhttp \
>     --publish 1984:1984 \
>     --publish 8984:8984 \
>     --volume "$(pwd)/basex/data":/srv/basex/data \
>     basex/basexhttp:latest
0cbd60aa6261c7e43646f675765e2dbf38fbfd57f641ca87dc6f6b8cd4fd36e0
root $
root $ docker exec -ti \
> --user root \
> basexhttp \
> bash
bash-4.4#
bash-4.4# whoami
root
bash-4.4#
bash-4.4# pwd
/srv
bash-4.4#
bash-4.4# ls
basex
bash-4.4#
bash-4.4# exit
exit
root $

```

I'll go through the answers more thoroughly, but thanks to all.

---
Tags: docker, containers
---


# Question
Title: Minecraft docker-container refuses connection
So I have this Dockerfile:

```
FROM openjdk:8u212-jre-alpine

LABEL maintainer "cowboy_patrick"

RUN apk add --no-cache -U \
  openssl \
  imagemagick \
  lsof \
  su-exec \
  shadow \
  bash \
  curl iputils wget \
  git \
  jq \
  python \
  tzdata \
  rsync \
  nano \
  sudo \
  knock \
  ttf-dejavu

EXPOSE 25565 25565

COPY minecraft /root/minecraft
WORKDIR /root/minecraft

RUN chmod +x /root/minecraft/start_minecraft.sh

CMD /root/minecraft/start_minecraft.sh

```

This just copies an existing world into an alpine with java and executes some prewritten scripts, I had written for running it on a vps (doing automatic backups etc.)

When I execute it with

> docker run --network host -p 25565:25565 --name mcs mcserver

I can see the following in the container-dashboard:

For good measures you can see my console-output here but it's pretty standard. Just a minecraft-server that boots up.

The only thing I don't understand is why the connection gets refused when I try to join localhost:25565:

Does anyone have an idea on how I can allow this connection? And in extent allow connections from outside my machine (assuming that port-formwarding and everything works).

# Answer
I’m glad my comment helped. I read the article OP mentioned in his answer and reread his question and I think the issue isn’t using `-p` instead of `—publish` as the two work the same, but specifying `--network host`. In Linux’s version of Docker daemon this prevents port publishing from working and since both Docker for Windows and Docker Toolbox use some sort of virtualization to run that version of Docker for Linux containers, I can only assume it has the same effect. The difference being that host in question is not Windows, but whatever Linux is running on top, so the container is completely unreachable.
> 1 votes

# Answer
The comment from Konrad Botor got me the idea that there are known issues on just the windows docker version. So I googled for it and in fact there are some differences! Using -P or --publish 25565:25565 worked for me. Read more here
> 0 votes

---
Tags: docker, dockerfile, networking
---


# Question
Title: docker run vs docker-compose - different results?
I have just started exploring Docker and am having trouble with docker commit/-compose.

I have copied a file into /root/.guacamole/lib (in the container) and then created an image using docker commit.

An `ls` of the relevant path in the container using

```
docker run -it --name="name" image bash

```

confirms that the file exists, but when the following docker-compose file sets up the container, the file is missing, even though it reports using the same image.

Any idea why that should be?

Thanks

```
$ cat docker-compose.yml
version: "2"
services:
  guacd:
    image: "guacamole/guacd"
    container_name: guacd
    hostname: guacd
    restart: always
    volumes:
      - "/data/shared/guacamole/guacd/data:/data"
      - "/data/shared/guacamole/guacd/conf:/conf:ro"
    expose:
      - "4822"
    ports:
      - "4822:4822"
    network_mode: bridge

  guac2:
    image: "guac2"
    container_name: guac2
    hostname: guac2
    restart: always
    volumes:
      - "/data/shared/guacamole/guacamole/guac-home:/data"
      - "/data/shared/guacamole/guacamole/conf:/conf:ro"
    expose:
      - "8080"
    ports:
      - "8084:8080"
    network_mode: bridge
    environment:
      - "GUACD_HOSTNAME=pc.lan"
      - "GUACD_PORT=4822"
      - "MYSQL_HOSTNAME=pc.lan"
      - "MYSQL_PORT=3306"
      - "MYSQL_DATABASE=guacamole_db"
      - "MYSQL_USER=guacamole_user"
      - "MYSQL_PASSWORD=password"
      - "GUACAMOLE_HOME=/data"

```

Here is the terminal text:

```
user@pc:~$ docker run -it --name="test" guac2 bash
root@b6dcd7dda985:/opt/guacamole# cd /root/.g*/lib
root@b6dcd7dda985:~/.guacamole/lib# ls
mariadb-java-client-2.6.2.jar  mysql-connector-java-5.1.46-bin.jar  mysql-connector-java-5.1.46.jar
***[NB 3 files listed above]***
root@b6dcd7dda985:~/.guacamole/lib# exit
exit
user@pc:~$ docker stop test
test
user@pc:~$ docker rm test
test
user@pc:~$ docker ps -a
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
ea92c2db77f6        guac2               "/opt/guacamole/bin/…"   51 minutes ago      Up 51 minutes       0.0.0.0:8084->8080/tcp   guac2
52aea98f7e2c        guacamole/guacd     "/bin/sh -c '/usr/lo…"   About an hour ago   Up 57 minutes       0.0.0.0:4822->4822/tcp   guacd
user@pc:~$ docker stop guacd
guacd
user@pc:~$ docker stop guac2
guac2
user@pc:~$ docker rm guacd
guacd
user@pc:~$ docker rm guac2
guac2
user@pc:~$ docker-compose up -d
Creating guac2 ... done
Creating guacd ... done
user@pc:~$ docker exec -it guac2 bash
root@guac2:/opt/guacamole# cd /root/.g*/lib
root@guac2:~/.guacamole/lib# ls
mysql-connector-java-5.1.46-bin.jar  mysql-connector-java-5.1.46.jar
***[NB only 2 files listed above]***
root@guac2:~/.guacamole/lib# exit

```

# Answer
I'm a bit new to Docker as well, but are you 100% certain that you performed a Docker push and your commit of your docker image that you're building with composer? And you have matching versions on the remote and your local machine?

I'm also not seeing where you are mounting the '~/.guacamole/lib' path in your docker-compose.yml under `volumes:`
> 1 votes

---
Tags: docker, docker-compose
---


# Question
Title: DevOps vs SRE vs Production Support Engineers
DevOps primarily focuses on Delivery Speed and SRE focuses on Reliability in production but where does Production Support Engineers fit who also focuses on production monitoring, alerting, performance, user experience, incident management, RCA and work on code defects and understand business functionality?

* In a SRE world, would production support engineers be more aligned to SRE or merged with SRE?

# Answer
The best answer to DevOps vs SRE is here What's the Difference Between DevOps and SRE ? From the above series, you will understand **Class SRE implements DevOps** and both are working on a similar ground and it's pretty align goal for both of them.

As a production support, you will be doing the task which you have mentioned in the question, apart, you need deep visibility, alert and notifications, latency, metrics, tracking, security, etc. lots of things to take care. And, it basically providing the service to the end users who is utilizing your application or system. Also, I see production support team have written the dockerfile and bash scripts to automate many tasks. So, whenever user faced any issue that first comes to Production Support team and it might went to other IT teams if it's not easily resolve. But, if you look at SRE, their sole purpose is to keep system in reliable state every time, maintain resiliency, etc. In some organization, the DevOps does this part too.

From my perspective, all the three terms are same as their goal align and just differ in functionality. These functionality differs from organization to organizations. But, DevOps are intended to bring culture and shift in paradigm so that everyone will work collaboratively. Hope this helps.
> 5 votes

# Answer
I don't want to nit pick, but I respectfully disagree with the idea that DevOps is about speed while SRE is about reliability. I understand this is an easy thing to think -- especially because SRE has "Reliability" in its name -- but it a'int so. :-)

SRE is all about velocity.

We think that over the long run you'll go much much faster if you recognize when the wheels are starting to go wobbly (SLOs and Error Budgets) and slow down in those cases to make things better. That, in our experience, is much faster than having to stop for a goodly long time because we let a bunch of reliability related technical debt pile up.

SRE can be thought of as a specific instantiation of DevOps with a more uniformly enforced set of constraints. (Hence the **Class SRE implements DevOps** formulation above.)
> 4 votes

# Answer
This is a good question and it is worth bearing in mind that that Ops can mean different things in different organizations. There is a) what you might call SysOps which is to do with the reliable running of the systems and which SRE effectively replaces in a DevOps context and b) what one might call Application Support which deals with manual intervention of a platform where there are gaps in the functionality or errors at the application level.

A Production Support role might cover either or both of these roles whether in a first or second line capacity. SRE is not really there to take on the Application Support function although they may (along with Development) be involved in engineering improvements to the platform to avoid the need for that manual intervention going forwards and through driving effective PIRs.

In your example the Production Support function seems to already cover much of what an SRE would also be responsible for. Depending on your Production Support Engineer skill set an SRE might approach the solution differently. In your case I would suggest you bring people with SRE skill set (monitoring, automation, DevOps working methods eg effective collaboration, PIR process as well as general SysOps skills) to work with the existing Product Support engineers. Whether you rebrand them as SREs or continue to have 2 roles would depend on their currrent skill set and potential but this could be a good career progression pipeline to recognise.
> 2 votes

---
Tags: culture, sre, organization
---


# Question
Title: az aks browse doesn't open browser automatically on WSL and only showing JSON instead of the dashboard
`az aks browse` works fine on my Mac Mini, but I'm trying to work with it in WSL2 Ubuntu 18.04 with Azure CLI on a Windows 10 machine.

```
$ az version
This command is in preview. It may be changed/removed in a future release.
{
  "azure-cli": "2.1.0",
  "azure-cli-command-modules-nspkg": "2.0.3",
  "azure-cli-core": "2.1.0",
  "azure-cli-nspkg": "3.0.4",
  "azure-cli-telemetry": "1.0.4",
  "extensions": {}
}

```

Kubernetes 1.14.8 which shouldn't make a difference given it works fine in macOS. Appears to be an issue with WSL2.

The first issue is running:

`az aks browse -g <groupname> -n <clustername>`

Just brings up the following in `az cli`:

```
Proxy running on http://127.0.0.1:8001/
Press CTRL+C to close the tunnel..

```

It doesn't automatically open the browser from WSL2 like it does in macOS and Linux. So you have to `CTRL + Click` the `http://127.0.0.1:8001` which will open the browser.

The second issue is it only brings up a JSON response with a list of API endpoints that looks like the following:

```
paths
0   "/api"
1   "/api/v1"
2   "/apis"
3   "/apis/"
4   "/apis/acme.cert-manager.io"
5   "/apis/acme.cert-manager.io/v1alpha2"
6   "/apis/admissionregistration.k8s.io"
7   "/apis/admissionregistration.k8s.io/v1beta1"
8   "/apis/apiextensions.k8s.io"
9   "/apis/apiextensions.k8s.io/v1beta1"
10  "/apis/apiregistration.k8s.io"
11  "/apis/apiregistration.k8s.io/v1"
12  "/apis/apiregistration.k8s.io/v1beta1"
13  "/apis/apps"
14  "/apis/apps/v1"
....

```

And doesn't do anything after that.

Followed all of the necessary steps for `az login`, `az aks get-credentials`, and `RBAC`.

What gives?

# Answer
> 1 votes
I would expect it to work like `az login` and open up a new tab in default browser. But somehow it doesn't work like that. I think the fix must come from azure CLI.

Temporarily, For `az aks browse` you can setup an environment variable on WSL.

```
export BROWSER="/mnt/c/Program Files (x86)/Google/Chrome/Application/chrome.exe"

```

In this case next when I run `az aks browse` it will open up the dashboard in new chrome tab.

You can use this variable to use your favorite browser. I would also add this variable to `.bashrc` or profile to be added on every session.

# Answer
> 0 votes
This is a combination of two things. You need the `BROWSER` variable and there is a bug (as of Sep-2020) where `az-cli` doesn't detect interactive session correctly (https://github.com/microsoft/WSL2-Linux-Kernel/issues/55)

So you need to modify `~/.bashrc`:

```
export BROWSER="/c/Program Files (x86)/Google/Chrome/Application/chrome.exe"

```

end then you can run your browser spawning commands with prefix `DISPLAY=:0` (sets that environment variable for duration of command; in this case az uses it to determine if the session is interactive and a browser should/could be spawned). So for you:

```
DISPLAY=:0 az aks browse -g <groupname> -n <clustername>

```

---
Tags: kubernetes, azure, azure-kubernetes-service
---


# Question
Title: How to handle services when the developers provide incorrectly in k8s
I provided the script to provision a new feature environment inside k8s it will create all the configs required, namespaces, resources and whatever...

in my script, I described this param:

```
parameters {
text(name: 'SERVICE_LIST', defaultValue: '', description: 'Services to deploy, separated by line breaks.')}

```

when the developer is provisioning a new env, he will provide a list of services that he wants to enable on that environment

also in my script, I described the following piece of code which are enabling all services requested(its located inside the stage and step):

```
def services = replaced.split('\\n')
   services.each { SERVICE ->
    sh "echo turning on service ${SERVICE}"
    sh "yq w -i  ${DEPLOY_ENV_FOLDER}/services.yaml '${SERVICE}.enabled' true"}

```

this is an example of a services file:

so, when the developer provides the services that he wants, the script goes to this file, enable the services and apply to k8s

now my question is how to assure that the services provided by the developer are correct and that they exist in the file? otherwise, I need to abort the script. I tried to use `try{} catch{}` in the second piece of code above to define the exception for abort services which are provided by developers incorrectly, do you have any other solution?

# Answer
> 1 votes
@Mkash, don't you know in advance what services you have in your `services.yaml` file?

Maintain a list in groovy, and see whether the input is in that list or not.

Something like:

```
availableServices = ["ew_ws_gateway", "ingress"]

services = ["a", "b", "c", "ew_ws_gateway"]

services.each { svc ->
    if (svc in availableServices) {
        println("Service exist" + svc)
    }
}

```

The other option is to change the parameter from type of `text` to multiple-choice possibly using https://plugins.jenkins.io/extended-choice-parameter/

---
Tags: jenkins, kubernetes, jenkins-pipeline, groovy
---


# Question
Title: trigger jenkins jobs only when a specific branch is updated
I require a setup in jenkins, which enables me to trigger a jenkins job based on a push event to a specific branch in github. tried so many methods neither worked.

# Answer
Install the git plugin, add credentials for the account you want it to use, then set the job to use SCM Polling. It will check on a preset interval for changes to that repo and trigger when changes are detected. If you need instant triggering upon push, you're looking for webhooks, which is significantly more difficult to set up. I find that polling every 2 minutes or so is good enough in most cases. Here's a decent example: https://www.baeldung.com/jenkins-job-schedule
> 2 votes

# Answer
If you're using the declarative DSL, you can use `when`

Suppose the branch you're interested in is `main`, have a stage like:

```
.
.
.
stages {
  stage('Specific Branch') {
    when { branch 'main' }
    steps {
      // only executed when the main branch is changed
    }
  }
}
.
.
.

```

From the Jenkins docs:

> Built-in Conditions

> branch

> Execute the stage when the branch being built matches the branch pattern (ANT style path glob) given, for example: when { branch 'master' }. Note that this only works on a multibranch Pipeline.

> The optional parameter comparator may be added after an attribute to specify how any patterns are evaluated for a match: EQUALS for a simple string comparison, GLOB (the default) for an ANT style path glob (same as for example changeset), or REGEXP for regular expression matching. For example: when { branch pattern: "release-\d+", comparator: "REGEXP"}

While this will only execute changes to the `main` branch, you still need to configure the job to be executed when changes to that branch are made. This can be done with the job dsl:

This slightly modified example is taken from the DSL plugin repo:

```
pipelineJob('job-dsl-plugin') {
  definition {
    cpsScm {
      scm {
        git {
          remote {
            url('https://github.com/your/repo')
          }
          branch('*/main')
        }
      }
      lightweight()
    }
  }
}

```
> 1 votes

---
Tags: jenkins, github
---


# Question
Title: Logging & Monitoring on multiple servers
I lead a small project with 2 groups of servers (servers in a single group are clones with the same microservices, like pods in the k8s), a balance loader, and a DB. Docker & k8s aren't used because of small scale of the project (but there are plans for migration). Instead, our cloud provider allows to set dynamic servers creation/removal depending on the load using filesystem image. This is fine for now, but there is a lack of logging and monitoring, I tried to structure the requirements:

1. A dashboard with server parameters such as CPU, RAM, disk space, network, etc usage for each server. Some dashboards with such technical information are available by our provider, but a provider-independent solution is desired.
2. A dashboard with numbers of responses by HTTP status code, for each *server group* in total and each server separately. We use NginX and it provides a status page, but it works on separate servers only.
3. A logs analysing & digging window, with support of multiline messages (like exception tracebacks).
4. Programming language independence. Our server apps write logs to several different files with a date in filename, probably this will have to be somehow changed.
5. Ability to set email (or even launching a shell script?) notifications with custom triggers.
6. Maybe there are some other important features?

Additional wishes:

7. Logging/monitoring setup should be free to use.
8. Simplicity is preferred. I hope a single solution can be flexible enough to cover all the needs of logging & monitoring.
9. Solution maturity / popularity and availability of docs,examples is important.
10. The solution should persist when migration to containers and orchestration will come.

I imagine the final result to be 1) a server that accumulates all the logs, provides dashboards, and logs digging panel, and 2) a script that is installed on each server, is run by SystemD timer, and periodically sends local logs to the server. But how the script will know which exact log files are presented on the current server install, using some config? Though I can be wrong about this algorithm.

I've heard about LogStash, ElasticSearch, Grafana, Kibana, Zabbix, Loki, Prometheus, etc, tried to google everything out, but found a total mess of recommendations. I'd to see modern explanations of different technologies, stacks, what software can be used together or is analog of another, and a comparison by the mentioned needs. I also hope that the answers will be very helpful to many other developers and beginning project leads for the topic is highly important.

# Answer
Well, I spent hours reading lots of articles, documentation, and finally figured everything out. See the comparative table:

---

### Brief description

**Zabbix** is a full-fledged technology that can deal with both metrics and logs processing, yet can be extended. It also supports automatic hosts detection & network maps building.

**Prometheus** is a metrics monitoring system. It is Kubernetes native, thus it can automatically detect hosts for data extracting. It also has myriads of available adapters: docs: Exporters And Integrations. To visualize the data, Grafana is used. Speaking about data storage, Prometheus has its own local system for that, but there are integrations with other technologies for sharding, see docs: Storage.

**PLG** is a stack for log aggregation. Logs get collected with PromTail, **Loki** is a core processing engine, and Grafana is used as an interface. Loki has no own storage system, so some other solution is required. It stores data as chunks and indexes, which are usually stored separately docs: Storage. As I understood, PLG has no trigger/notification system, but the data extracting part also can parse logs into metrics and export them into Prometheus, so we get alerts. Also, I met a notice that PromTail doesn't support multiline logs, but FluentD can be used instead as it is more flexible.

**ELK / EFK** is a technology stack for logs parsing, indexing, searching, so it can perform complex queries. For data extracting, LogStash was originally used, but FluentD is becoming popular.
> 1 votes

---
Tags: monitoring, microservices, logging
---


# Question
Title: No Such Host: Kubernetes/Docker cannot pull from private k8 registry
I have a private docker registry in k8 in the default namespace with `tls` at `https://docker-registry.default:5000`. By doing local port forwarding to it(at port 5000) and adding `docker-registry.default` to my `/etc/hosts` file, I have been able to pull and push images to it.

However, I'm using a serverless framework(Nuclio) which manages the deploy for me. The error I get is:

```
Normal   Scheduled  7m52s                   default-scheduler                                       Successfully assigned nuclio/helloworld1-96876fcf9-7c87w to gke-your-first-cluster-1-pool-1-fe915942-gnkx
  Normal   Pulling    6m14s (x4 over 7m51s)   kubelet, gke-your-first-cluster-1-pool-1-fe915942-gnkx  Pulling image "docker-registry.default:5000/docker/nuclio/processor-helloworld1:latest"
  Warning  Failed     6m14s (x4 over 7m51s)   kubelet, gke-your-first-cluster-1-pool-1-fe915942-gnkx  Failed to pull image "docker-registry.default:5000/docker/nuclio/processor-helloworld1:latest": rpc error: code = Unknown desc = Error response from daemon: Get https://docker-registry.default:5000/v2/: dial tcp: lookup docker-registry.default on 169.254.169.254:53: no such host
  Warning  Failed     6m14s (x4 over 7m51s)   kubelet, gke-your-first-cluster-1-pool-1-fe915942-gnkx  Error: ErrImagePull
  Normal   BackOff    6m2s (x6 over 7m51s)    kubelet, gke-your-first-cluster-1-pool-1-fe915942-gnkx  Back-off pulling image "docker-registry.default:5000/docker/nuclio/processor-helloworld1:latest"
  Warning  Failed     2m38s (x21 over 7m51s)  kubelet, gke-your-first-cluster-1-pool-1-fe915942-gnkx  Error: ImagePullBackOff

```

It gives a `no such host` error.

A Nuclio developer has confirmed that it's not a nuclio issue since I cannot even do a `docker login` from the Nuclio Admin pod(not the failed pod above) to this internal `docker-registry.default` though I can do a `docker login` to the Docker hub and a `wget` to `docker-registry.default` successfully. The Nuclio Admin pod is Alpine Linux if it helps.

# Answer
It seems your private DNS is misconfigured. Check your host or make sure that `docker-registry.default` is the a valid hostname.

You can run a temporary Ubuntu pod to check if that hostname is resolving correctly from your cluster, e.g.

```
$ kubectl run ubuntu -it --image ubuntu --rm=true -- bash
root@shell:/# apt update && DEBIAN_FRONTEND=noninteractive apt --yes install dnsutils
root@shell:/# dig docker-registry.default
docker-registry.default.    IN  A D0.y0u.h4ve.1P?

```

Additional suggestions:

* If you're using a cloud provider (such as Azure), make sure your Private DNS zone is linked with your cluster's virtual network. Read: Connect privately to an Azure container registry using Azure Private Link
* If your container registry endpoint isn't correct, edit the deployment manifest and fix the image path.
* Test pulling the image from another VM within the same private network.
> 1 votes

---
Tags: docker, kubernetes
---


# Question
Title: ERROR: Unable to read database state: No such file or directory
I was trying to install Photon by pulling the docker image. But when I run `docker build -t photon .` I am getting the below error.

```
 ---> Running in 3076fb42ef8b
ERROR: Unable to read database state: No such file or directory
ERROR: Failed to open apk database: No such file or directory
The command '/bin/sh -c apk add git && git clone https://github.com/s0md3v/Photon.git Photon' returned a non-zero code: 99

```

I observed the same error before when I tried to install some other application. So, I am guessing this is not something specific to Photon. What is causing this error? How to fix this?

# Answer
Please try splitting this line:

`RUN apk add git && git clone https://github.com/s0md3v/Photon.git Photon`

into two:

```
RUN apk add git

RUN git clone https://github.com/s0md3v/Photon.git Photon

```

This will tell you exactly which piece of the command fails. If it is the `apk add git` part maybe try this: `RUN apk update && apk add git`.

Also you may fixing to a specific tag in the `FROM python:3-alpine` as `3-alpine` follows the latest alpine release which may have some unknown/unfixed issues.

See https://hub.docker.com/\_/python/ for available tags.
> 0 votes

# Answer
It may help to try it on another computer or inside a VM to isolate the cause.

I ran into the same issue on one computer (stable Manjaro with snap docker). Interestingly, I was able to build the same Dockerfile without issues on another computer (running KDE Neon with snap docker). I'm not sure what the actual cause of the issue is, but it seems, in my case, to be a problem with docker itself and not the Dockerfile.

Dockerfile:
```
FROM redis:5.0.9-alpine

LABEL version="1.0.4"

ENV SENTINEL_QUORUM=2 \
    SENTINEL_DOWN_AFTER=1000 \
    SENTINEL_FAILOVER=1000 \
    REDIS_MASTER_NAME=redismaster \
    REDIS_MASTER=redis-zero \
    REDIS_SENTINEL_NAME=redis-sentinel

RUN apk --no-cache add drill
[...]

```
Output:
```
ERROR: Unable to read database state: No such file or directory
ERROR: Failed to open apk database: No such file or directory
The command '/bin/sh -c apk --no-cache add drill' returned a non-zero code: 99

```
> 0 votes

# Answer
We had a similar case, where an image doesn't build and had been raising the issue above (while it worked on other machines) we had put a lot of time and efforts to upgrade Docker (on Ubuntu 18.04), clean up all images, volumes ...try many staff to debug, but as crazy as it sounds, after just explicitly restarting the docker service, it has resolved itself. (I wouldn't be able to explain what happened exactly)

So always first try the old "have you(I) turned it on and off again (explicitly)"!

btw before purging images.., at the all beginning have had a different message (it was complaining about access to some Python packages permission).
> 0 votes

# Answer
I was getting this error in the Docker installation that installed using "snap" in Ubuntu 18.04. My problem was resolved when I uninstalled Docker and installed it as in their official documentation.

To uninstall Docker (snap): https://askubuntu.com/a/1155962/1063090
Docker official installation: https://docs.docker.com/engine/install/ubuntu/
> 0 votes

---
Tags: docker, containers, alpine
---


title: AWX cannot find matching host

text:
# Question
Title: AWX cannot find matching host
I have on premise AWX and inventory (instances on AWS). To provision newly launched instance I want to use provisioning callbacks. But when executing this command from the instance

```
$ curl -v -H 'Content-Type:application/json' -d '{"host_config_key": "key1"}' https://my-host.com/api/v2/job_templates/28/callback/

```

an error occurs

```
{"msg":"No matching host could be found!"}

```

What might be wrong with AWX configuration if host is present within the dynamic inventory and `UPDATE ON LAUNCH` flag is enabled?

# Answer
Set `HTTP_X_FORWARDED_FOR` extra header as allowed in settings and you should be good.

https://github.com/ansible/awx/issues/8005
> 0 votes

---
Tags: amazon-web-services, ansible, provisioning, awx
---


# Question
Title: Back-filling prometheus (and related system) metrics?
Projects like Thanos, which are based on prometheus storage formats, seem to say that it is impossible to back-fill data in a prometheus based system.

Can anyone explain why and if there are any possibilities for changing this? Lack of ability to backfill would make it difficult to switch metrics systems without impact (when previous systems can already retain multi years' data).

# Answer
Answering my own question (after a few days of research).

I have yet to test this but https://github.com/Cleafy/promqueen seems to be designed for this purpose:

> PromQueen made possible to record prometheus metrics offline. PromQueen can, \> therefore, backfill the recorded data inside a native prometheus database.
>
> PromQueen is composed of two primary tools:
>
> * promrec tapes the metrics in a specified output file.
> * promplay backfills the prometheus database from scratch.

It doesn't have a lot of contributors but it comes from another POC repo and it was updated in the last 6 months, so I suspect it'll work fine.

In any case, it proves that you can plausibly back-fill prometheus, so it's not impossible.

## Roadmap - Prometheus

You can also find this in the prometheus road-map, so it's clearly slated to be built-in eventually.

https://prometheus.io/docs/introduction/roadmap/

> Backfill time series Backfilling will permit bulk loads of data in the past. This will allow for retroactive rule evaluations, and transferring old data from other monitoring systems.
> 3 votes

# Answer
While the backfill is still in review for Prometheus, you can use https://github.com/sepich/thanos-kit/ `import` command to import custom data in prometheus text format to Thanos.
> 1 votes

---
Tags: prometheus, thanos
---


# Question
Title: Ansible: Run tasks on specific hosts with var/fact from another host
I'm trying to get the output from a command on the first host in group managers and execute it on the other hosts in the same group.

hosts file

```
[managers]
k8s01
k8s02
k8s03

[workers]
worker1
worker2
...

```

main.yaml

```
- name: Create token to join cluster
  shell: kubeadm token create --print-join-command
  run_once: true
  changed_when: false
  when: inventory_hostname == groups['managers'][0]
  register: kubernetes_join_command_result

-name: Set join command for managers
- set_fact:
    kubernetes_join_command_controlplane: "{{ kubernetes_join_command_result.stdout }} --control-plane"
  when: kubernetes_join_command_result.stdout is defined
  delegate_to: "{{ item }}"
  delegate_facts: true
  with_items: "{{ groups['managers'] }}"

- name: Debug var
  debug:
    msg: 'Join Command: {{ kubernetes_join_command_result.stdout }}'

- name: Join managers into cluster
  shell: "{{ kubernetes_join_command_controlplane.stdout }}"
  args:
    warn: no
  when: inventory_hostname != groups['managers'][0] and inventory_hostname in groups['managers']

```

Instead of running the join command in hosts k8s02 and k8s03, I get the output below.

```
TASK [managers: Join managers into cluster] ********************
skipping: [k8s01] => {"changed": false, "skip_reason": "Conditional result was False"}
TASK [manager: another task] ***************************

```

# Answer
> 1 votes
I believe your `hosts` is set to `k8s01` or other group not containing k8s02 and k8s03.
That's why they are not run.

You shall add `with_items: "{{ groups['managers'] }}"` and delegate as you done in the second task `-name: Set join command for managers`.

```
- name: Join managers into cluster
  shell: "{{ kubernetes_join_command_controlplane.stdout }}"
  args:
    warn: no
  when: inventory_hostname != groups['managers'][0] and inventory_hostname in groups['managers']
  delegate_to: "{{ item }}"
  delegate_facts: true
  with_items: "{{ groups['managers'] }}"

```

Alternatively, depending on your design and scope. Do as answer from Botka and set the hosts as:
`hosts: all`
or
`hosts: managers`

# Answer
> 2 votes
The condition works as expected. For example, the inventory and the playbook below

```
shell> cat hosts
[managers]
k8s01
k8s02
k8s03

[workers]
worker1
worker2

shell> cat playbook.yml
- hosts: all
  gather_facts: false
  tasks:
    - debug:
        var: inventory_hostname
      when:
        - inventory_hostname != groups.managers.0
        - inventory_hostname in groups.managers

```

give

```
shell> ansible-playbook -i hosts playbook.yml

PLAY [all] ****

TASK [debug] ****
skipping: [worker1]
ok: [k8s02] => {
    "inventory_hostname": "k8s02"
}
skipping: [k8s01]
skipping: [worker2]
ok: [k8s03] => {
    "inventory_hostname": "k8s03"
}

```

---
Tags: ansible
---


title: Properly shutdown k3s on local machine

text:
# Question
Title: Properly shutdown k3s on local machine
I've just installed k3s on my notebook (Ubuntu). Works fine. Great project. However, i do not need it to run 24/7 so, how do I properly shutdown the k3s cluster. Oddly, I cannot find anything about this in the documentation. Thanks

# Answer
> 18 votes
There is a systemd shutdown task (`/etc/systemd/system/k3s.service`) which stops the server, but not the k8s resources (pods, networks, etc). For this there is a utility script `k3s-killall.sh` which should be at `/usr/local/bin/k3s-killall.sh`

---
Tags: kubernetes
---


# Question
Title: How do I convert "\n" to linebreaks in my docker logs?
I am using `docker-compose up` to start a web server in linux but am finding that the `stdout` is printing "\n" characters instead of creating new lines. This makes stack traces difficult to read;

```
172.19.0.1 - - [12/Sep/2020:02:19:18 +0000] "GET /node/100 HTTP/1.1" 500 337 "http://localhost/node/100/edit" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.102 Safari/537.36"
[Sat Sep 12 02:19:18.426082 2020] [php7:notice] [pid 21] [client 172.19.0.1:58954] ParseError: syntax error, unexpected '::' (T_PAAMAYIM_NEKUDOTAYIM), expecting identifier (T_STRING) in /opt/drupal/web/modules/custom/rsvplist/src/Plugin/Block/RSVPBlock.php on line 37 #0 /opt/drupal/vendor/composer/ClassLoader.php(322): Composer\\Autoload\\includeFile('/opt/drupal/web...')\n#1 [internal function]: Composer\\Autoload\\ClassLoader->loadClass('Drupal\\\\rsvplist...')\n#2 [internal function]: spl_autoload_call('Drupal\\\\rsvplist...')\n#3 /opt/drupal/web/core/lib/Drupal/Component/Plugin/Factory/DefaultFactory.php(96): class_exists('Drupal\\\\rsvplist...')\n#4 /opt/drupal/web/core/lib/Drupal/Core/Plugin/Factory/ContainerFactory.php(17): Drupal\\Component\\Plugin\\Factory\\DefaultFactory::getPluginClass('rsvp_block', Array, 'Drupal\\\\Core\\\\Blo...')\n#5 /opt/drupal/web/core/lib/Drupal/Component/Plugin/PluginManagerBase.php(76): Drupal\\Core\\Plugin\\Factory\\ContainerFactory->createInstance('rsvp_block', Array)\n#6 /opt/drupal/web/core/lib/Drupal/Core/Plugin/DefaultSingleLazyPluginCollection.php(62): Drupal\\Component\\Plugin\\PluginManagerBase->createInstance('rsvp_block', Array)\n#7 /opt/drupal/web/core/modules/block/src/BlockPluginCollection.php(57): Drupal\\Core\\Plugin\\DefaultSingleLazyPluginCollection->initializePlugin('rsvp_block')\n#8 /opt/drupal/web/core/lib/Drupal/Component/Plugin/LazyPluginCollection.php(80): Drupal\\block\\BlockPluginCollection->initializePlugin('rsvp_block')\n#9 /opt/drupal/web/core/modules/block/src/BlockPluginCollection.php(45): Drupal\\Component\\Plugin\\LazyPluginCollection->get('rsvp_block')\n#10 /opt/drupal/web/core/lib/Drupal/Core/Plugin/DefaultSingleLazyPluginCollection.php(83): Drupal\\block\\BlockPluginCollection->get('rsvp_block')\n#11 /opt/drupal/web/core/lib/Drupal/Core/Plugin/DefaultSingleLazyPluginCollection.php(99): Drupal\\Core\\Plugin\\DefaultSingleLazyPluginCollection->setConfiguration(Array)\n#12 /opt/drupal/web/core/lib/Drupal/Core/Plugin/DefaultSingleLazyPluginCollection.php(55): Drupal\\Core\\Plugin\\DefaultSingleLazyPluginCollection->addInstanceId('rsvp_block', Array)\n#13 /opt/drupal/web/core/modules/block/src/BlockPluginCollection.php(34): Drupal\\Core\\Plugin\\DefaultSingleLazyPluginCollection->__construct(Object(Drupal\\Core\\Block\\BlockManager), 'rsvp_block', Array)\n#14 /opt/drupal/web/core/modules/block/src/Entity/Block.php(156): Drupal\\block\\BlockPluginCollection->__construct(Object(Drupal\\Core\\Block\\BlockManager), 'rsvp_block', Array, 'rsvpblock')\n#15 /opt/drupal/web/core/modules/block/src/Entity/Block.php(145): Drupal\\block\\Entity\\Block->getPluginCollection()\n#16 /opt/drupal/web/core/modules/block/src/BlockAccessControlHandler.php(118): Drupal\\block\\Entity\\Block->getPlugin()\n#17 /opt/drupal/web/core/lib/Drupal/Core/Entity/EntityAccessControlHandler.php(105): Drupal\\block\\BlockAccessControlHandler->checkAccess(Object(Drupal\\block\\Entity\\Block), 'view', Object(Drupal\\Core\\Session\\AccountProxy))\n#18 /opt/drupal/web/core/lib/Drupal/Core/Entity/EntityBase.php(314): Drupal\\Core\\Entity\\EntityAccessControlHandler->access(Object(Drupal\\block\\Entity\\Block), 'view', Object(Drupal\\Core\\Session\\AccountProxy), true)\n#19 /opt/drupal/web/core/modules/block/src/BlockRepository.php(56): Drupal\\Core\\Entity\\EntityBase->access('view', NULL, true)\n#20 /opt/drupal/web/core/modules/block/src/Plugin/DisplayVariant/BlockPageVariant.php(137): Drupal\\block\\BlockRepository->getVisibleBlocksPerRegion(Array)\n#21 /opt/drupal/web/core/lib/Drupal/Core/Render/MainContent/HtmlRenderer.php(259): Drupal\\block\\Plugin\\DisplayVariant\\BlockPageVariant->build()\n#22 /opt/drupal/web/core/lib/Drupal/Core/Render/MainContent/HtmlRenderer.php(117): Drupal\\Core\\Render\\MainContent\\HtmlRenderer->prepare(Array, Object(Symfony\\Component\\HttpFoundation\\Request), Object(Drupal\\Core\\Routing\\CurrentRouteMatch))\n#23 /opt/drupal/web/core/lib/Drupal/Core/EventSubscriber/MainContentViewSubscriber.php(90): Drupal\\Core\\Render\\MainContent\\HtmlRenderer->renderResponse(Array, Object(Symfony\\Component\\HttpFoundation\\Request), Object(Drupal\\Core\\Routing\\CurrentRouteMatch))\n#24 /opt/drupal/vendor/symfony/event-dispatcher/LegacyEventDispatcherProxy.php(78): Drupal\\Core\\EventSubscriber\\MainContentViewSubscriber->onViewRenderArray(Object(Symfony\\Component\\HttpKernel\\Event\\ViewEvent), 'kernel.view', Object(Symfony\\Component\\EventDispatcher\\LegacyEventDispatcherProxy))\n#25 /opt/drupal/vendor/symfony/http-kernel/HttpKernel.php(163): Symfony\\Component\\EventDispatcher\\LegacyEventDispatcherProxy->dispatch(Object(Symfony\\Component\\HttpKernel\\Event\\ViewEvent), 'kernel.view')\n#26 /opt/drupal/vendor/symfony/http-kernel/HttpKernel.php(80): Symfony\\Component\\HttpKernel\\HttpKernel->handleRaw(Object(Symfony\\Component\\HttpFoundation\\Request), 1)\n#27 /opt/drupal/web/core/lib/Drupal/Core/StackMiddleware/Session.php(57): Symfony\\Component\\HttpKernel\\HttpKernel->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#28 /opt/drupal/web/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(47): Drupal\\Core\\StackMiddleware\\Session->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#29 /opt/drupal/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(106): Drupal\\Core\\StackMiddleware\\KernelPreHandle->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#30 /opt/drupal/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(85): Drupal\\page_cache\\StackMiddleware\\PageCache->pass(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#31 /opt/drupal/web/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(47): Drupal\\page_cache\\StackMiddleware\\PageCache->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#32 /opt/drupal/web/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(52): Drupal\\Core\\StackMiddleware\\ReverseProxyMiddleware->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#33 /opt/drupal/vendor/stack/builder/src/Stack/StackedHttpKernel.php(23): Drupal\\Core\\StackMiddleware\\NegotiationMiddleware->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#34 /opt/drupal/web/core/lib/Drupal/Core/DrupalKernel.php(705): Stack\\StackedHttpKernel->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#35 /opt/drupal/web/index.php(19): Drupal\\Core\\DrupalKernel->handle(Object(Symfony\\Component\\HttpFoundation\\Request))\n#36 {main}, referer: http://localhost/node/100/edit
172.19.0.1 - - [12/Sep/2020:02:19:18 +0000] "GET /node/100 HTTP/1.1" 500 337 "http://localhost/node/100/edit" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.102 Safari/537.36"

```

I am unsure whether this is a "Docker" or a "Unix" question. Ideally there would be some kind of setting I can use to make `docker run` respect unix line endings, but I'm also open to other solutions.

# Answer
> 2 votes
Try to read logs as follows:

```
docker logs -f <container-name or container-ID> 2>&1

```

---

\[**NOTE**\]:

* `-f` option follows live logs.
* `2>&1` makes output as `stdout` and `stderr` which is useful for grepping and standardizing.

---
Tags: docker, docker-compose, linux
---


# Question
Title: network ansible command error: "msg": "network os centos is not supported"
How do I get the `ansible` config for `rolly` below?

remote:

```
[nsaunders@rolly ~]$
[nsaunders@rolly ~]$ lsb_release -a
LSB Version:    :core-4.1-amd64:core-4.1-noarch
Distributor ID: CentOS
Description:    CentOS Linux release 8.2.2004 (Core)
Release:    8.2.2004
Codename:   Core
[nsaunders@rolly ~]$
[nsaunders@rolly ~]$ ansible --version
ansible 2.9.13
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/nsaunders/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.6/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.6.8 (default, Apr 16 2020, 01:36:27) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]
[nsaunders@rolly ~]$
[nsaunders@rolly ~]$ show config
bash: show: command not found...
[nsaunders@rolly ~]$
[nsaunders@rolly ~]$ ansible show config
usage: ansible [-h] [--version] [-v] [-b] [--become-method BECOME_METHOD]
               [--become-user BECOME_USER] [-K] [-i INVENTORY] [--list-hosts]
               [-l SUBSET] [-P POLL_INTERVAL] [-B SECONDS] [-o] [-t TREE] [-k]
               [--private-key PRIVATE_KEY_FILE] [-u REMOTE_USER]
               [-c CONNECTION] [-T TIMEOUT]
               [--ssh-common-args SSH_COMMON_ARGS]
               [--sftp-extra-args SFTP_EXTRA_ARGS]
               [--scp-extra-args SCP_EXTRA_ARGS]
               [--ssh-extra-args SSH_EXTRA_ARGS] [-C] [--syntax-check] [-D]
               [-e EXTRA_VARS] [--vault-id VAULT_IDS]
               [--ask-vault-pass | --vault-password-file VAULT_PASSWORD_FILES]
               [-f FORKS] [-M MODULE_PATH] [--playbook-dir BASEDIR]
               [-a MODULE_ARGS] [-m MODULE_NAME]
               pattern
ansible: error: unrecognized arguments: config
[nsaunders@rolly ~]$

```

local:

```
nicholas $
nicholas $ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04 LTS
Release:    20.04
Codename:   focal
nicholas $
nicholas $ cat first_playbook.yml
---

- name: Network Getting Started First Playbook
  connection: network_cli
  gather_facts: false
  hosts: all
  tasks:

    - name: Get config for VyOS devices
      vyos_facts:
        gather_subset: all

    - name: Display the config
      debug:
        msg: "The hostname is {{ ansible_net_hostname }} and the OS is {{ ansible_net_version }}"
nicholas $
nicholas $ yamllint first_playbook.yml
first_playbook.yml
  15:81     error    line too long (97 > 80 characters)  (line-length)

nicholas $
nicholas $ ansible all -i rolly.foo, -c network_cli -u nsaunders -k -m centos_facts -e ansible_network_os=centos
SSH password:
rolly.foo | FAILED! => {
    "msg": "network os centos is not supported"
}
nicholas $

```

referencing:

https://www.ansible.com/blog/getting-started-writing-your-first-playbook

# Answer
This error is due to invalid value of `ansible_network_os:`

The playbook you try setup is meant for network devices.
And since centos is not a network device OS the playbook will fail.
Here you can find supported network OS.
Ansible Documentation: ansible\_network\_os values

If you don't have any device running a supported network OS I recommend you to follow the regular user guide.
Ansible User Guide

My suggestion is to setup a easy playbook that will run on the environment you have.
Get the understanding of following concepts.
`Control node`
`Managed nodes`
`Inventory`
`Modules`
`Tasks`
`Playbooks`

Here is a short example.
This will only execute `hostname` do confirm it execute on expected host.
Let's say `nicholas` is your control node and `rolly` is the managed node.

On `nicholas`:
Create a inventory as before with your control node and managed node.
inventory.txt:

```
[local]
localhost ansible_connection=local

[managed_node]
rolly ansible_host=<rolly ip> ansible_user=<rolly user> ansible_ssh_pass=<rolly user password>

```

myplaybook.yml:

```
---
- name: This will get hostname localy on control node
  hosts: localhost
  tasks:
    - name: get hostname
      command: hostname
      register: result

    - name: print hostname
      debug:
          var: result

- name: This will get hostname remote on manged node
  hosts: rolly
  tasks:
    - name: get hostname
      command: hostname
      register: result

    - name: print hostname
      debug:
          var: result

```

And then execute on the control node:
`ansible-playbook -i inventory.txt myplaybook.yml`
> 2 votes

# Answer
```
nicholas $
nicholas $ cat ansible.cfg
[defaults]
inventory = hosts
ask_vault_pass = True

[privilege_escalation]
become_ask_pass = True

nicholas $
nicholas $ sudo ansible-playbook first_playbook.yml --connection=local
BECOME password:
Vault password:
[WARNING]: Unable to parse /home/nicholas/ansible/hosts as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [Network Getting Started First Playbook] **************************************************************************************
skipping: no hosts matched

PLAY RECAP *************************************************************************************************************************

nicholas $

```

As a workaround to using a remote, although it requires some additional configuration apparently.
> 0 votes

---
Tags: ansible, yaml, centos, ssh, ansible-inventory
---


title: How to add timeout to a helm chart?

text:
# Question
Title: How to add timeout to a helm chart?
I have a helm chart which pulls a lot of images from various registries and deploys a lot of pods. It runs a lot of k8s jobs before getting the pods up.

Overall, helm install command takes huge time, so usually my helm install will also have `--timeout 3600`.

Like: `helm install --name` will also have `--timeout 3600`

Can I embed this `--timeout 1800` to the helm chat itself? (rather than providing a command line parameter)

# Answer
> 5 votes
Since the timeout is a Helm option and not a Kubernetes object attribute, it can't be embedded in the chart itself.

The `--timeout` option is the amount of time the Helm utility will wait for Kubernetes commands to complete before marking the release as FAILED.

More Info on Helm CLI options

---
Tags: docker, kubernetes, helm, artifactory, kubernetes-helm
---


# Question
Title: How can I simulate failure of an AWS Availability Zone?
In order to test that our load balancers and Multi-AZ RDS instances are working the way I'd expect them to, I'd like to simulate failure of an AWS Availability Zone.

Short of locating one of the datacenters and setting fire to it (which probably wouldn't be successful anyway), how can I do this? (I'm aware of Netflix' Chaos Gorilla, but it doesn't seem to have been released to the public like Chaos Monkey has, and I'm not sure what techniques it uses to cause its chaos).

# Answer
This (slightly old in cloud years) question on Hacker News provides a few suggestions:

* Remove the security groups from all instances in one zone, which should technically make them all appear 'down'
* Adjust the VPC route tables to black hole the subnets in one availability zone
* Disable an availability zone from the load balancer (docs)

You could also probably adjust the Network ACL attached to the relevant subnets to deny all traffic.

I'm not aware of an existing, available tool that does any of this automatically, though.
> 5 votes

# Answer
This lab from AWS Well-Architected will show you how to implement an AWS AZ failure simulation using Bash, Java, Python, C#, or PowerShell

https://wellarchitectedlabs.com/reliability/300\_labs/300\_testing\_for\_resiliency\_of\_ec2\_rds\_and\_s3/
> 4 votes

---
Tags: amazon-web-services, high-availability, resilience-testing
---


# Question
Title: Azure managed NFS shared storage for Azure VMs
What is a reliable way to implement Azure managed NFS shared for multiple Azure VMs?
The idea is an alternative to AWS EFS for multiple EC2.
I only found the following solutions with disadvantages:

So how do companies get an alternative solution to AWS EFS on Azure which makes sense?

# Answer
> 1 votes
Azure files NFS share documentation was just uploaded: https://docs.microsoft.com/en-us/azure/storage/files/storage-files-how-to-create-nfs-shares

# Answer
> 1 votes
You can use Azure Files

https://docs.microsoft.com/en-us/azure/storage/files/storage-files-introduction

Limits for it can be found from here

https://docs.microsoft.com/en-us/azure/storage/files/storage-files-scale-targets

---
Tags: azure, storage
---


# Question
Title: ansible error: skipping: no hosts matched
I've tried a few variations on running `ansible` locally as a test case:

```
nicholas $
nicholas $ ls
ansible.cfg  ansible.cfg.orig  first_playbook.yml  inventory.txt  playbook.yml
nicholas $
nicholas $ cat ansible.cfg
[defaults]
transport = local

nicholas $
nicholas $ cat playbook.yml
---

- name: Network Getting Started First Playbook
  connection: network_cli
  gather_facts: false
  hosts: all
  tasks:

    - name: Get config for VyOS devices
      vyos_facts:
        gather_subset: all

    - name: Display the config
      debug:
        msg: "The hostname is {{ ansible_net_hostname }} and the OS is {{ ansible_net_version }}"

nicholas $
nicholas $ ansible --version
ansible 2.9.6
  config file = /home/nicholas/ansible/ansible.cfg
  configured module search path = ['/home/nicholas/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.8.2 (default, Jul 16 2020, 14:00:26) [GCC 9.3.0]
nicholas $
nicholas $ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04 LTS
Release:    20.04
Codename:   focal
nicholas $
nicholas $ ansible-playbook --connection=local  playbook.yml
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [Network Getting Started First Playbook] **************************************************************************************
skipping: no hosts matched

PLAY RECAP *************************************************************************************************************************

nicholas $

```

perhaps I need an additional configuration for the localhost loopback?

Although I *can* ping localhost as:

```
nicholas $
nicholas $ ansible localhost -m ping -u root
127.0.0.1 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },
    "changed": false,
    "ping": "pong"
}
nicholas $

```

# Answer
> 6 votes
The answer is in the warning you get:

```
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

```

When you use `hosts: all` in your playbook, `localhost` is not matched.

If you want to run the playbook on localhost, you can do one of the following:

* Change your playbook to `hosts: localhost`.
* Explicitly provide an inventory file that has `localhost`.

You can create an inventory file - `inventory.yml` with the contents:

```
all:
  hosts:
    localhost

```

and then pass this inventory file to ansible explicitly like this:

`$ ansible-playbook --connection=local -i inventory.yml playbook.yml`

You can find more information about inventory in the Ansible documentation.

---
Tags: ansible, yaml, orchestration, ansible-inventory, ubuntu
---


# Question
Title: How can I access port 80 of another container, when port 80 is already exposed?
I have a docker file that runs some integration tests between two containers. It looks a bit like this:

```
version: '3'
services:
    test_client:
        image: client:latest
        depends_on:
          - test_server
        command: bash -c "run_test.sh -p 80"
    test_server:
        image: server:latest

```

`test_server` launches a web server on port 80, but that same port is already exposed within `test_client` (I don't have control over the part of the `client:latest` image where the port is exposed. I tried using the `port: 8080:80` to forward port 80 to 8080 in the server, but that only changes the port exposed on the host, not within the docker environment.

How can I connect `test_client` to `test_server` in such a scenario?

# Answer
You can use a proxy to forward traffic between the two services, I typically do something like this which includes automated SSL with certbot. In the example below I have two services listening on port 80/443 (Sonarqube and Jenkins). You can also see I can reference container hosts by name (https://jenkins:8080)

```
nginx:
container_name: nginx-certbot
restart: unless-stopped
image: staticfloat/nginx-certbot
networks:
  - local
ports:
  - "0.0.0.0:80:80"
  - "0.0.0.0:443:443"
environment:
  CERTBOT_EMAIL: <insert_email>
volumes:
  - ./conf.d:/etc/nginx/user.conf.d:ro
  - letsencrypt:/etc/letsencrypt

```

And this is what my nginx config looks like

```
server {
listen              443 ssl;
server_name jenkins.hauntedmansion.io;
ssl_certificate     /etc/letsencrypt/live/jenkins.hauntedmansion.io/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/jenkins.hauntedmansion.io/privkey.pem;

client_max_body_size 20M;

  location / {
    proxy_set_header   Host $host;
    proxy_set_header   X-Real-IP $remote_addr;
    proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header   X-Forwarded-Proto $scheme;
    proxy_pass         http://jenkins:8080;
    proxy_read_timeout 90;
    proxy_redirect     http://127.0.0.1:8080 https://jenkins.hauntedmansion.io;
  }
}

```

If you want to see the full docker compose example check it out here:

https://gist.github.com/cvega/66e0e5a2815e9b923da2c616f74dea7e
> 1 votes

---
Tags: docker, networking
---


# Question
Title: Cancel a Spot Request vs Terminate Spot Instance
I just started getting familiar with EC2's Spot Requests and Spot Instances (1 day into it!). I understand that I can save some money doing a Spot Request to create a Spot Instance as opposed to creating an on-demand Instance.

What I don't understand is the correct way to "shut things down". If I want to make sure that I'm no longer charged, do I need to to cancel the Spot Request and terminate the Spot Instance?

I noticed that I could solely terminate the Spot Instance: what would this mean in terms of my Spot Request which exists without my instance?

# Answer
Canceling spot requests is good if you have created a 'persistent' spot instance. If you try terminating a spot instance in the EC2 console when it's set as 'persistent', a new spot instance will spin up in its place in a few minutes.

However, if you cancel the spot request this will terminate the current spot instance and also prevent any future rebuilds of this instance. So this means you will not have another instance spin up when the first is terminated. To cancel the spot request, go to the panel on the left 'spot requests' and here you will see all the ones you have created. Search for your instance ID and you will find the associate spot request.

Yes, spot does save a lot of money compared to on-demand but this feature is a bit annoying as I didn't realize it for a while. I was terminating instances but didn't realize new instances were spinning up in their place.

**EDIT** When canceling spot requests now, it gives you the option if you would like to terminate the instance also
> 3 votes

# Answer
You can definitely cancel your spot instance requests via the Management Console and via the CLI. There appears to be a command to terminate the instance also.

See the following AWS docs as a reference:

AWS Docs - Cancel Spot Instance

As for terminating just the spot instance, I'm not sure but I think the, still active, spot request would spin up the instance once the spot price matches the request again.

\** Edit \**

Worth noting that if you terminate the spot instance within an instance hour then you pay for the full hour, even if you only used a few minutes of it.
> 2 votes

---
Tags: amazon-web-services
---


# Question
Title: Kubernetes used to orchestrate Containerized Machines
0

I am using k8s to orchestrate an environment on a local network. My idea is to deploy a number of linux machines each of which should have its own "External" IP.

Right now I found guides to expose services on External IP (e.g. https://kubernetes.io/docs/tutorials/stateless-application/expose-external-ip-address/), but this is tailored to expose services on a precise port.

My need is more targeted to expose an entire machine on a precise static IP and not just a port. This would allow me to easily deploy and reshape any network topology I need. Notice that doing it without exposing the IPs (hence using internal k8s routing/dns) is not an option because this environment should be interfaced with real/bare metal hardware already present in the network.

Here follows a little example of the desired result:

Can anyone relate to this and let me know if this is even possible?

# Answer
You should look at DaemonSets. You can configure it so that the Pods runs on each node of your Kubernetes cluster. Each individual Pod (running your Linux machine) can be exposed to outside network using the IP address of the node on which it is running.

> My need is more targeted to expose an entire machine on a precise static IP and not just a port.

*Before getting into further explanation, I should point out that there is nothing like exposing entire machines.*

Access to any machine is always through a combination of IP address and Port. So you should decide based on which port you would like to access on the container.

**Example:**

Let's say I have a 3 node Kubernetes cluster, I can create a DaemonSet with the application/image of my choice. Below names/details are imaginary, but should give an idea on the mapping.

```
ubuntu-daemonset-pod-xyz    k8s-node-1    192.168.2.1
ubuntu-daemonset-pod-abc    k8s-node-2    192.168.2.2
ubuntu-daemonset-pod-hij    k8s-node-3    192.168.2.3

```

The Pods above can be "exposed" on the IP address of the node on which it is running. That is: `ubuntu-daemonset-pod-xyz` on `192.168.2.1`.
> 1 votes

---
Tags: docker, kubernetes
---


# Question
Title: Ansible warning that host key has changed, even though I've taken multiple steps to prevent this warning
I've followed the recommendations I could find to prevent this from happening. I am using Vagrant, but once the boxes are spun, I'd like to do things with the `ansible` command sometimes.

```
$ grep host_key /etc/ansible/ansible.cfg
host_key_checking = False

$ grep kafka01 ./inventory
kafka01 ansible_host=127.0.0.1 ansible_port=2231 ansible_user='vagrant' ansible_ssh_private_key_file='/home/me/project/project/.vagrant/machines/kafka01/virtualbox/private_key' ansible_ssh_common_args='-o StrictHostKeyChecking=no'

$ export ANSIBLE_HOST_KEY_CHECKING=False
$ ansible kafka01 -m ping -i ./inventory

 [WARNING]: Unhandled error in Python interpreter discovery for host kafka01: Failed to connect to the host via ssh: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@  @    WARNING:
REMOTE HOST IDENTIFICATION HAS CHANGED!     @  @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@  IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!  Someone could be eavesdropping on
you right now (man-in-the-middle attack)!  It is also possible that a host key has just been changed.  The fingerprint for the ECDSA key sent by the remote host is
SHA256:jn5ClU5g566Gsvobf8Xc45DJnIiAllN+RJkg35Nf9KE.  Please contact your system administrator.  Add correct host key in /home/me/.ssh/known_hosts to get rid of this message.  Offending ECDSA key
in /home/me/.ssh/known_hosts:19    remove with:    ssh-keygen -f "/home/me/.ssh/known_hosts" -R "[127.0.0.1]:2231"  ECDSA host key for [127.0.0.1]:2231 has changed and you have requested strict
checking.  Host key verification failed.

kafka01 | UNREACHABLE! => {
.....

```

What's going on here?

Edit: No, I don't want to keep deleting the key. I destroy/build these over and over again. I want to find the correct, long-term solution.

# Answer
> 5 votes
You can add this in inventory:

```
ansible_ssh_common_args="-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null"

```

It's ignoring check host and setting /dev/null as Known host file.

# Answer
> 4 votes
An alternative sintax to `Dilson Rainov`'s answer, more ansible 2.x style (I think).

In your inventory define the host as:

```
vagrant-local:
      ansible_host: 192.168.33.10
      ansible_user: vagrant
      ansible_ssh_private_key_file: .vagrant/machines/default/virtualbox/private_key
      UserKnownHostsFile: /dev/null
      StrictHostKeyChecking: no
      ansible_python_interpreter: /usr/bin/python3

```
* `UserKnownHostsFile` causes the check of the host fingerprint on an always empty file (replacing the check inside `~/.ssh/known_hosts`)
* `StrictHostKeyChecking` allows ssh to connect without the user confirming the detected fingerprint from the vagrant host

---
Tags: ansible, vagrant
---