# Question
Title: How can I access directories inside a docker container from outside (not the reverse!)?
I am trying to setup **plug-and-play** docker images for my development and support team members - however I am running into some problems because I want them to be able to use the editor of their choice!

There are two critical folders that are part of the **build** (`eggs` and `src`) that needs to be exposed to the end users so that they can edit it in any editor of their choice.

All the options to edit container files in host machine talks about mounting a fresh folder from the outside; but like I said, my `src` and `egg` folders are **part of the build**, so I can't let them mount a fresh folder during `docker run`.

I thought of about using volumes, but here are the questions I have on this approach:

1) Do we usually put build stuff that are actually part of the container into volumes? The container has no meaning without these volumes.

2) Can end users take a copy of these volumes and just modify it according to their convenience without the master copy of these volumes? (Like Image vs Container concept?)

All my problems will be solved if I can expose folders inside my container to the outside (not the reverse), but that seems really tricky to achieve!

PS: I haven't research enough on volumes but I thought I could ask this question anyway.

---

Just to complete the workflow, our development/support team usually identifies and fixes these problems in their machine, sends a PR to the Git system which then gets merged etc.

I am trying to use Docker to solve the problem of not re-building the entire web-app in each developers machine running multiple OSes etc.

# Answer
I think you may be using the wrong tool for the job. Docker is not designed to be a version control system, it is designed for containerization. Furthermore, your question makes it sound like the development and support team members are constantly changing the files that are essential to the makeup of the image. This makes using a volume pointless as the Docker image will always be changing.

If I'm understanding what you are trying to achieve (people should be able to modify the `src` and `eggs` folders with whatever editor they want), then you may want to explore adopting a CI/CD workflow:

1. Committed changes to the `src` and `egg` folders trigger a new image to be built.
2. The new image is pushed to a Docker repository.
3. Team members can then pull the updated image.

As for your two specific questions:

> Do we usually put build stuff that are actually part of the container into volumes? The container has no meaning without these volumes.

Not usually. Generally if I have to make changes to a container I would remote into it and then commit the changes.

> Can end users take a copy of these volumes and just modify it according to their convenience without the master copy of these volumes? (Like Image vs Container concept?)

This is sounding like a version control system which Docker is not designed for. You can have a shared volume that users have access to, but the use case is for persisting data outside of containers (e.g. SQL database data).
> 1 votes

---
Tags: docker
---


# Question
Title: Safari test automation using Robot Framework via Jenkins
I spent about one week trying to make my Robot Framework tests work for Safari browser in MacOS, when launched by Jenkins via ssh.

The main issue was that Robot couldn't open a browser when executed in ssh session, reporting "No browser is open".

* MacOS 10.12.6 Sierra (issue was there also with 10.14 Mojave)
* Safari 12.1.2 (issue was there also with Safari 10)
* Robot Framework 3.1.2 (Python 3.6.8 / Python 2.7.16)

My current workaround is:

1. Application "robot\_test" created (in Automator based on shell script type) which is to launch a shell script and save the exit code:

```
/tmp/execute_test.sh > /tmp/execute_test.log 2>&1
echo "export EXITCODE=$?" > /tmp/execute_test.result

```
2. On the Jenkins side, in the shell command section, I am creating an shell script with the contents I need, execute it and wait for result file to appear:

```
export CURRENT_FOLDER=`pwd`
echo "#!/bin/bash
~/.bash_profile
cd $CURRENT_FOLDER
python3 -m robot -parameters robot_script.robot
" > /tmp/execute_test.sh
chmod +x /tmp/execute_test.sh
rm -f /tmp/execute_test.result
MAX_WAIT_TIME=1200
WAIT_TIME=0
open -a robot_test > /tmp/robot_test_app.log 2>&1
while [ ! -f /tmp/execute_test.result ] && [ $WAIT_TIME -le $MAX_WAIT_TIME ]; do
  WAIT_TIME=$((WAIT_TIME+10)); sleep 10
done
cat /tmp/execute_test.log

EXITCODE=1
if [ -f /tmp/execute_test.result ]; then
  chmod +x /tmp/execute_test.result
  source /tmp/execute_test.result
fi
exit $EXITCODE

```

Am I the only one having problems with running robot test from ssh sessions in MacOS? If yes, could you please share you steps to configure the proper setup? If no, then this post might be helpful for someone to have working configuration at least, please share your solutions/workarounds if they are better.

# Answer
Do you run selenium as a separate process?
I had the same issue if you are not running selenium in the background.

(edit) Try java -jar selenium-XX.XX.jar in terminal.

After that run pabot or robot command to run the tests. You can crate app through the Automator and use the same approach you describe to call it from the CI. That is what I end up using.
> 1 votes

---
Tags: jenkins, testing, macos
---


# Question
Title: Is it possible to combine failure stage from groovy with try catch from pipeline?
I have a pipeline with some stages and logic. The only possible debugger is adding try-catch to each stage:

```
stage ("distribution"){
            steps{
                script{
                    try{
    amd_distribution_distribute_bundle
                    }
                   catch (Exception e) {
                    echo e.toString()
                    }
            }
        }
    }

```

however, if one stage fails, I need to stop the pipeline:

```
status = sh script: '''
                set +x
                python3.4 main.py --command create_bundle
                ''',  returnStatus:true
             }
        }
     }

    if (status == 1) {
    error("returning 1 from func")
    }

```

but the try-catch from the pipeline actually cancel it (it proceed to other stages. is there a way to combine them?

# Answer
You may use and update a flag to `false` in the `catch` and then use that flag in the `when` directive of other stage(s) that you want to skip because of any failure in the given stage. Example:

```
boolean flagStageOneSuccess = false

pipeline {
    agent any

    stages {
        stage ('stage-1') {
            steps {
                try {
                    // Do something that might fail

                    flagStageOneSuccess = true

                } catch (Exception e) {
                    // Handle the exception
                    flagStageOneSuccess = false
                }
            }
        }

        stage ('stage-2') {
            when {
                equals expected: true, actual: flagStageOneSuccess
            }
            steps {
                // Do something in stage-2
            }
        }
    }
}

```

It would also benefit you in viewing the status of your stage in `Blue Ocean`. If the stage is skipped, it shows as 'Not built'.
> 4 votes

---
Tags: jenkins-pipeline, groovy
---


# Question
Title: How to connect to postgres container from a node container using docker-compose?
I'm trying to connect from a node app container to a postgresql container using docker-compose but I'm not able to do it. This is my docker-compose.yml file:

```
version: '3.7'
services:
  db:
    image: postgres:12.1
    ports:
      - "5432:5432"
    env_file:
      - "./.env"
    networks:
      - tcp-modbus
    volumes:
      - ./modbustcp.sql:/docker-entrypoint-initdb.d/init.sql
  redis:
    image: redis:5.0.6
    ports:
      - "6379:6379"
    networks:
      - tcp-modbus
  rabbit:
    image: "rabbitmq"
    environment:
      RABBITMQ_ERLANG_COOKIE: "SWQOKODSQALRPCLNMEQG"
      RABBITMQ_DEFAULT_USER: "*"
      RABBITMQ_DEFAULT_PASS: "*"
      RABBITMQ_DEFAULT_VHOST: "/"
    ports:
    - "15672:15672"
    - "5672:5672"
    volumes:
      - "./enabled_plugins:/etc/rabbitmq/enabled_plugins"
    networks:
      - tcp-modbus
  modbus-tcp:
    image: customImage
    ports:
      - "8080:3000"
    links:
      - "db"
    volumes:
      - "./modbus.env:/usr/src/app/.env"
    networks:
      - tcp-modbus
networks:
  tcp-modbus:
    driver: bridge

```

I'm using typeorm to set the connection from the node app and was working fine before dockerizing the app, maybe has something to do with the host name of the container, cause I'm using localhost to connect to the postgresql container. Can I use somehow the container network as a hostname to connect to postgres?

# Answer
docker-compose contains an internal DNS. In the example the service names are also the DNS names. In this case, the DNS of the database is `db`. If one configure this as a hostname in the node app then one should be able to access the database.

Back in the day, the wordpress example of docker-compose helped me understanding this. When I changed `WORDPRESS_DB_HOST: db:3306` to `WORDPRESS_DB_HOST: db2:3306`, the wordpress was not able to access the mysql `db` anymore as depicted in the log. When it was changed back and forth to `db` and to `db2` it was not able to access it and with `db` it was. Then I realized that the service names are actually added to an internal DNS that can be resolved by other services that are defined in the docker-compose.yml.

Recently I realized that running docker-compose up and specifying multiple files using `-f` also ensures that all services get added to the internal DNS. Initially I thought that the resolving was restricted to the same docker-compose.yml, but probably the docker network is responsible. If multiple apps reside in the same network, in this case the default, then they can resolve each other.
> 0 votes

---
Tags: docker, networking, postgresql
---


# Question
Title: Consumption Plan Function App to Firewalled Storage problem
I am trying to connect a (python) function app to a storage resource. The storage has a public container that is accessed by several company internal HTML/CSS pages (`$web`).

Unfortunately, this container *should* contain sensitive information. So in order to prevent leaking data my choice was to configure the firewall to only allow the IP range that is used by the company. So far this seemed to work fine.

However, no I tried to automatically generate files (blobs) using a function app. Since only very few executions are necessary, there is no real justification of using anything but the "Consumption" plan of azure function apps. However, apparently this is not part of the trusted azure services, neither has a fixed IP nor can the consumption plan be set-up in a VNet, and thus it could not pass the storage containers firewall.

I tried adding the function app as a role with contributor rights. I tried adding the outbound IP addresses of the function app (to be found in the resource manager) to the firewall. But both things did not work.

Is there any workaround for this?

# Answer
Consumption Plan Function Apps are not able to be joined to a VNET.

Premium Function Apps however, can be.

If the container has sensitive information then you should not make it public, IP restriction shouldn't be your only protection.

Use another service (such as a function app) to determine a user's identity: then grant them a SAS token for the container.
> 2 votes

---
Tags: azure, firewall
---


# Question
Title: What are good options to implement a devops chatbot for Slack?
I'm considering implementing a chatops platform for my company to help automate code deploys. I've seen some alternatives, which I narrowed down to cog and Errbot. Errbot seems to be actively maintained, while cog hasn't seen a commit to master since July 2018. I liked cog's architecture, which gives me a sense of performance and resiliency; it can easily be deployed in Kubernetes and it is extensible in any language. Also its ACL feature is a big plus.

My previous company's devops team used cog, and it seemed pretty solid to me. However, I'm concerned about deploying it as my current devops team is small and if cog is no longer maintained we cannot afford to maintain it.

Can you share your experiences with any chatops platform you're using, and what are their pros and cons? I'm interested in chatbots that have at least these features:

* Has an ACL built in, similar to cog
* Able to be extended in any language (or at least its extensions can be implemented in Python/JavaScript).

# Answer
> 1 votes
I have found StackStorm to be the most fully-featured ChatOps product.

It satisfies your requirements:

* Authz, RBAC, ACL ✔
* Extensible in Python ✔

See below.

---

StackStorm is marketed as a

> Robust If This, Then That Platform

(but for DevOps)

StackStorm consists of a few components, beyond it's chatops capability:

> * Sensors: Sensors are Python plugins for either inbound or outbound integration that receives or watches for events respectively. When an event from external systems occurs and is processed by a sensor, a StackStorm trigger will be emitted into the system.
> * Triggers Triggers are StackStorm representations of external events. There are generic triggers (e.g. timers, webhooks) and integration triggers (e.g. Sensu alert, JIRA issue updated). A new trigger type can be defined by writing a sensor plugin.
> * Actions Actions are StackStorm outbound integrations. There are generic actions (ssh, REST call), integrations (OpenStack, Docker, Puppet), or custom actions. Actions are either Python plugins, or any scripts, consumed into StackStorm by adding a few lines of metadata. Actions can be invoked directly by user via CLI or API, or used and called as part of rules and workflows.
> * Rules: Rules map triggers to actions (or to workflows), applying matching criteria and mapping trigger payload to action inputs.
> * Workflows Workflows stitch actions together into “uber-actions”, defining the order, transition conditions, and passing the data. Most automations are more than one-step and thus need more than one action. Workflows, just like “atomic” actions, are available in the Action library, and can be invoked manually or triggered by rules. Packs Packs are the units of content deployment. They simplify the management and sharing of StackStorm pluggable content by grouping integrations (triggers and actions) and automations (rules and workflows). A growing number of packs are available on StackStorm Exchange. Users can create their own packs, share them on Github, or submit to the StackStorm Exchange.
> * Audits Audit trail of action executions, manual or automated, is recorded and stored with full details of triggering context and execution results. It is also captured in audit logs for integrating with external logging and analytical tools: LogStash, Splunk, statsd, syslog.

StackStorm's ChatOps is built into it's other components:

> ChatOps leverages two components within StackStorm in order to provide a fluid user experience. These subsystems are the **Action Aliases** and **Notifications** subsystems. You can learn more about each of these individual components in their corresponding sub-sections.

but you can also use your own:

> If you already have a Hubot instance, you’ll need the hubot-stackstorm module installed and the following environment variables set up:

Authentication can be done by StackStorm itself, or delegated to a proxy authentication system:

> The service can be configured with different backends (i.e. PAM, LDAP, etc.) to handle the authentication. If a backend is not specified, an htpasswd-compatible flat file authentication backend is used. To use a different backend, select and install the appropriate python package from the StackStorm community repos and configure st2auth accordingly.

It also supports API keys.

As for RBAC, they claim that it is supported in their "Extreme Workflow Composer"

> Role Based Access Control (RBAC) is only available in Extreme Workflow Composer. For information about Extreme Workflow Composer and the differences between StackStorm and Extreme Workflow Composer, please see stackstorm.com/product.

More at https://docs.stackstorm.com/install/ewc.html

StackStorm has "packs" which are predefined workflows, similar to IFTT or Zapier, which are sets of triggers, rules and actions. You can extend it with your own actions:

> An action is composed of two parts:
>
> * A YAML metadata file which describes the action, and its inputs.
> * A script file which implements the action logic
>
> As noted above, an action script can be written in an arbitrary programming language, as long as it follows these conventions:
>
> * Script should exit with 0 status code on success and non-zero on error (e.g. 1)
> * All log messages should be printed to standard error

---
Tags: chatops
---


title: Using 32 bit for savings

text:
# Question
Title: Using 32 bit for savings
What are the downsides for CI servers to use 32 bit Docker images?

For example, using `FROM i386/python:3.6.9-alpine3.9` instead of `python:3.6.9-alpine3.9` in `Dockerfile`.

`python:3.6.9-alpine3.9` (32 bit, 31.3 MB) is over 4% smaller than `python:3.6.9-alpine3.9` (64 bit, 32.7 MB). The 32 bit images on Dockerhub are usually smaller than the 64 bit images.

When using 32 bit containers, I see no performance difference to the 64 bit containers. Using 32 bit images and containers save me space.

# Answer
> 2 votes
These are really the pitfalls as running a VM or traditional server with 32 bit CPUs. Most notably is memory addressing. 32 bit CPUs can only access up to 4 gigs of memory. This isn't something i would expect most containers have an issue with as they should be single, small, light applications. One thing that you might run into is any binary built targeting specifically 64 bit architecture will not be able to run. This is any application and their DLLs. I don't think any web API will have issues with this, you will need to be aware of console applications. Most dot net applications target `any cpu` allowing the runtime to target the correct underlying architecture. There are applications that do target specific architectures and you will need to be aware of which ones they are.

In today's world, storage space is cheap. The exception to this rule is embedded development (but I don't think that is the case here). You need to ask yourself, is the extra 4% saved space worth the cost of me managing and ensuring that all the applications i'm running work on a 32 bit architecture. Remember 32 bit applications will run on 64 bit architecture, but not the other way around. Unless there is a specific need to only run 32 bit, like testing an emulated embedded environment (which is only 32 bit), I would just go with the 64 bit versions of everything. To me, using 32 bit is solving a problem you don't actually have.

---
Tags: docker, continuous-integration, dockerfile, cicd
---


# Question
Title: Why container can use ifconfig to get the host information?
Today I pull an image from the Internet, and I found that the docker networking mode can be detected in the image's startup script. But docker0 only exists in the host not the container. Why? Dockerfile

```
FROM ubuntu:18.04

RUN apt-get update \
    && apt-get install -y  net-tools   \
    && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

ADD test/ /etc/test/

RUN chmod +x  /etc/test/*.sh

CMD ["/bin/bash", "/etc/test/start.sh"]

```

start.sh

```
#!/bin/sh
set -e

# check for presence of network interface docker0
check_network=$(ifconfig | grep docker0 || true)

# if network interface docker0 is present then we are running in host mode and thus must exit
if [[ ! -z "${check_network}" ]]; then
    echo "[crit] Network type detected as 'Host', this will cause major issues, please stop the container and switch back to 'Bridge' mode" | ts '%Y-%m-%d %H:%M:%.S' && exit 1
fi

```

I run it with `docker run --privileged --net=host` will get

```
[crit] Network type detected as 'Host', this will cause major issues, please stop the container and switch back to 'Bridge' mode

```

# Answer
You are not accessing the hosts network interface information. Rather you are receiving data of the bridge adapter created for your Docker Containers Network.

Please note the results from my web server:

Container networking info:

```
# docker exec -it ghost sh
/var/lib/ghost # ifconfig
eth0      Link encap:Ethernet  HWaddr 02:42:AC:12:00:02
          inet addr:172.18.0.2  Bcast:172.18.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:20 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1536 (1.5 KiB)  TX bytes:0 (0.0 B)

```

Host network info:

```
root@li293-xxx:/data/mydomain.us# ifconfig
br-08d99d9f172e: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    inet 172.18.0.1  netmask 255.255.0.0  broadcast 172.18.255.255
    inet6 fe80::42:57ff:fe74:c49b  prefixlen 64  scopeid 0x20<link>
    ether 02:42:57:74:c4:9b  txqueuelen 0  (Ethernet)
    RX packets 150  bytes 16535 (16.5 KB)
    RX errors 0  dropped 0  overruns 0  frame 0
    TX packets 234  bytes 21425 (21.4 KB)
    TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
    inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
    ether 02:42:82:3d:17:63  txqueuelen 0  (Ethernet)
    RX packets 0  bytes 0 (0.0 B)
    RX errors 0  dropped 0  overruns 0  frame 0
    TX packets 0  bytes 0 (0.0 B)
    TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    inet 66.xxx.xxx.205  netmask 255.255.255.0  broadcast 66.xxx.xxx.255

```

Please review the networking documentation at https://docs.docker.com/v17.09/engine/userguide/networking/
> 1 votes

---
Tags: docker
---


# Question
Title: Internal Server Error while running docker-compose to install Gitlab
I'm installing **Gitlab** on my server using **docker-compose** based on Docker Docs. My SSH port is not the default one (22) and I have changed it to something else like 2228. So the YML file became like this:

```
web:
  image: 'gitlab/gitlab-ce:latest'
  restart: always
  hostname: 'gitlab.mydomain.com'
  environment:
    GITLAB_OMNIBUS_CONFIG: |
      external_url 'https://gitlab.mydomain.com'
      gitlab_rails['gitlab_shell_ssh_port'] = 2228
  ports:
    - '80:80'
    - '443:443'
    - '2228:22'
  volumes:
    - '/srv/gitlab/config:/etc/gitlab'
    - '/srv/gitlab/logs:/var/log/gitlab'
    - '/srv/gitlab/data:/var/opt/gitlab'

```

When I run the `docker-compose up -d`, I receive this result and errors:

```
Starting docker_web_1 ...

ERROR: for docker_web_1  a bytes-like object is required, not 'str'

ERROR: for web  a bytes-like object is required, not 'str'
Traceback (most recent call last):
  File "site-packages/docker/api/client.py", line 261, in _raise_for_status
  File "site-packages/requests/models.py", line 940, in raise_for_status
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.21/containers/9d4483106cf1683f46b5158a9550f9b4375d7c147ecb8b07df533840ad0806b6/start

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "compose/service.py", line 625, in start_container
  File "compose/container.py", line 241, in start
  File "site-packages/docker/utils/decorators.py", line 19, in wrapped
  File "site-packages/docker/api/container.py", line 1095, in start
  File "site-packages/docker/api/client.py", line 263, in _raise_for_status
  File "site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
docker.errors.APIError: 500 Server Error: Internal Server Error ("b'driver failed programming external connectivity on endpoint docker_web_1 (9b32d28cc2f635e085e82a2ef9f4d1fd3afc76f7693e279405ada5f61b96d215): Error starting userland proxy: listen tcp 0.0.0.0:443: bind: address already in use'")

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "bin/docker-compose", line 6, in <module>
  File "compose/cli/main.py", line 72, in main
  File "compose/cli/main.py", line 128, in perform_command
  File "compose/cli/main.py", line 1107, in up
  File "compose/cli/main.py", line 1103, in up
  File "compose/project.py", line 570, in up
  File "compose/parallel.py", line 112, in parallel_execute
  File "compose/parallel.py", line 210, in producer
  File "compose/project.py", line 556, in do
  File "compose/service.py", line 568, in execute_convergence_plan
  File "compose/service.py", line 510, in _execute_convergence_start
  File "compose/parallel.py", line 112, in parallel_execute
  File "compose/parallel.py", line 210, in producer
  File "compose/service.py", line 508, in <lambda>
  File "compose/service.py", line 620, in start_container_if_stopped
  File "compose/service.py", line 627, in start_container
TypeError: a bytes-like object is required, not 'str'
[5738] Failed to execute script docker-compose

```

I'm new in Docker and docker-compose, but I did everything as it is written in the official documentation. May you help me plesae to solve this issue?

# Answer
> 4 votes
In the error message you presented your system is reporting port 443 is already in use. You need to confirm which ports you have in use for other services.

> listen tcp 0.0.0.0:443: bind: address already in use

For what it is worth I recommend you use a nginx reverse proxy. Please reference https://stackoverflow.com/questions/46195852/how-to-run-gitlab-in-docker-container-with-nginx-proxy-over-ssl-with-letsencrypt for further details.

---
Tags: docker, docker-compose, gitlab
---


# Question
Title: How securely to keep google cloud service account key?
How securely to keep google cloud service account key?

I want to ask about how we should securely keep service account key. The scenario looks like:

* Our product is connecting to BigQuery.
* We have project which is used only by developers as playground.
* We need to run integration tests in pipelines and they needs service account to run correctly.
* Developer also should be able to run integration tests locally.

Where should I put the service account key?

I can add it to our repository, but I'm not sure if this solution is ok. (even it is only key with read-only permission to developer version of BigQuery).

The second solution is create a few keys. And every developer should has own. Then every developer have to care about key himself (and we can easy remove key when developer i.e quit job)

# Answer
I would recommend you use their Google Cloud Secrets Engine. Documentation is here: https://www.vaultproject.io/docs/secrets/gcp/index.html

Managing such sensitive secrets by any means other than a Vault service is dangerous.
> 1 votes

# Answer
Add the service account key to you repository is not a good practice when thinking about security.

The second solution may happen and should be a good practice to implement some key rotation process to avoid some dev using a compromised key. Even though, there are some guidance which suggests to avoid the utilization of several keys (Also, I suppose there's a limit in the platform for Service Account key creation)

Take a look at Cloud KMS that may have some suggestion for you on how to manage your keys
> 0 votes

---
Tags: google-cloud-platform, devsecops
---


# Question
Title: How to solve `Error creating config for VM: Invalid JSON` in minishift?
```
minishift start

```

was issued and openshift started:

```
Login to server ...
Creating initial project "myproject" ...
Server Information ...
OpenShift server started.

The server is accessible via web console at:
    https://192.168.42.37:8443/console

You are logged in as:
    User:     developer
    Password: <any value>

To login as administrator:
    oc login -u system:admin

-- Exporting of OpenShift images is occuring in background process with pid 32067.

```

At a certain moment, the laptop ran out of free disk space. Once this issue was solved, an attempt was done to restart the minishift, but all commands return:

```
Error creating config for VM: Invalid JSON

```

**Attempts to solve the issue**

1. It turned out that a minishift process was still running. Killing this process and subsequently run `minishift status` returned the same issue.
2. `sudo systemctl restart libvirtd` did not solve the issue
3. @tm1701 suggested `minishift delete --clear-cache`, but that does not solve the issue either

# Answer
> 0 votes
One option to solve this issue is a suggestion as provided by @AndreasGajdosik, i.e. deleting the `.minishift` directory.

Once `rm -rf ~/.minishift` was issued and `minishift status` was run, the issue has been solved:

```
user@host$ minishift status
Does Not Exist

```

---
Tags: openshift, minishift
---


# Question
Title: How to run a CMD in a docker container that was created using Packer?
So I am creating a docker image with packer. The template defines the provisioners and builders etc. The builder section looks like this:

```
{
    "builders": [
        {
            "type": "docker",
            "image": "ubuntu:latest",
            "export_path": "image.tar",
            "changes": [
                "USER test",
                "WORKDIR /tmp",
                "VOLUME /data",
                "EXPOSE 21 22",
                "CMD sudo myprogram"
            ]
        }
    ]
}

```

When running packer against the template, the output is an `image.tar` file. I can then import it: `docker import image.tar`. And then I start like this `docker run -p 21:21 -it --rm 52c0b0362362 bash`.

I want whenever the image is started that automatically `sudo myprogram` is executed. However it does not seem to work, even tho the template validated successfully. I also tried instead of specifying `CMD sudo myprogram` to set it as entrypoint like so: `ENTRYPOINT sudo myprogram`. Neither worked. How can I make sure whenever my image is started, this command is automatically executed? It must be run as root/with sudo, that's important.

# Answer
> 5 votes
> And then I start like this `docker run -p 21:21 -it --rm 52c0b0362362 bash`

This would not work by design. When you run the image with the command above you are instructing Docker to overwrite the `CMD` instruction from your Dockerfile and therefore telling it to execute `bash` instead of `sudo myprogram`. It doesn't matter what you have set in the `CMD` instruction - it would never be executed.

One way you could go around this would be to use `ENTRYPOINT` as others have suggested. This will make `sudo myprogram` the default command and then `CMD` can act as "additional arguments" to the default command (`sudo myprogram`).

Here's a working example:

```
# build.json
{
    "builders": [
        {
            "type": "docker",
            "image": "ubuntu:latest",
            "commit": true,
            "changes": [
                "ONBUILD RUN apt-get -q update",
                "ONBUILD RUN apt-get install -y -q htop",
                "ENTRYPOINT [\"top\"]"
            ]
        }
    ]
}

# /packer build ./build.json

# run top with delay 3 sec
# docker run --rm -it sha256:hash

# run top with delay 1 sec
# docker run --rm -it sha256:hash -d 1

```

The other solution would be to keep the `CMD [\"sudo myprogram\"]` instruction but to run the image without adding a command at the end: `docker run -it --rm HASH_ID`.

# Answer
> 1 votes
According to @earcam one can run a `docker inspect <id>` to check whether an image contains a CMD. If one builds a docker image using the provided packer snippet in the Q&A, imports it and subsequently runs a `docker inspect 1127c20077ef | grep -i cmd`, the CMD has not been set:

```
"Cmd": null,

```

According to @Greendrake when has to set the context when importing an image by defining `--change 'CMD ["somecommand"]'`

```
docker import --change 'CMD ["bash"]' image.tar

```

and subsequently issue an:

```
docker inspect b604fa244510 | grep -A3 -i cmd
        "Cmd": [
            "bash"
        ],

```

Once the image has been imported and docker run is issued a shell is opened:

```
docker run -it b604fa244510
root@69df4b3f48a2:/#

```

instead of

```
docker run -p 21:21 -it --rm 02163a32f3b7
docker: Error response from daemon: No command specified.
See 'docker run --help'.

```

---
Tags: docker, packer
---


# Question
Title: How to inject credentials to python virtual env in jenkins?
I am trying to **inject my credentials** to a Jenkins Freestyle project.

The project is building a python project in virtual environment with "behave" created with ShiningPanda plugin

can someone help me ? thanks

# Answer
In order to pass masked credentials to your Jenkins job, do the following:

1. In Jenkins, install Credentials Binding plugin.
2. In your Jenkins, go to Credentials and click the global domain.
3. In the left pane, click "Add credential".
4. From the "Kind" dropdown, choose "Username with password".
5. Enter the username and password in the reflective boxes and choose an ID (you will use this ID when you want to access the credentials) and click OK.
6. Then, in your job, under "Build environment", check "Use secret text(s) or file(s)" and it will add a "Binding" section and an "Add" button, click the "Add" button and "username and password separated", choose a variable name to hold the username and another to hold the password and choose the credentials you created earlier from the credentials dropdown.
7. Then, in the "Build" section of the job, you can access the credentials using your chosen username and password variables.

I wrote an article about this subject in the past so feel free to check it for more detailed explanation and screen shots.
> 2 votes

---
Tags: jenkins, python, job-dsl
---


# Question
Title: error validating data: couldn't find type
I have following `yaml` file:

```
---
apiVersion: v1
kind: pod
metadata:
    name: Tesing_for_Image_pull.pod
    spec:
        containers:
        - name: mysql
          image: mysql
          imagePullPolicy: Always
          command: ["echo", "SUCCESS"]

```

When I use `kubectl create -f Tesing_for_Image_pull.yaml` , I get the following error:

```
error: error validating "Tesing_for_Image_pull.yaml": error validating data: couldn't find type: v1.pod; if you choose to ignore these errors, turn validation off with --validate=false

```

How can I solve my problem?

# Answer
kubernetes is picky and case-sensitive:

`kind: Pod`

should do it.

Also, `spec` should not be in `metadata`:

```
apiVersion: v1
kind: Pod
metadata:
  name: Testing
spec:
  containers:
  - name: mysql
    image: mysql
    imagePullPolicy: Always
    command: ["echo", "SUCCESS"]

```
> 1 votes

---
Tags: yaml, kubectl
---


# Question
Title: Which plugin is used to copy file from different environment in jenkins
I want to copy file from different environment in jenkins. Can you please guide me to choose the plugin for copying

# Answer
> 0 votes
You are probably looking for something like the Config File Provider. Its plugin page describes it as a plugin which:

> Adds the ability to provide configuration files (i.e., settings.xml for maven, XML, groovy, custom files, etc.) loaded through the Jenkins UI which will be copied to the job's workspace.

If you need something more like a script (as opposed to the configuration file scenario above), you may be interested in the Managed Script plugin. This allows jobs to consume scripts which are centrally stored and defined by the Jenkins admins:

> Managed scripts are shell scripts which are managed centrally by an administrator and can be referenced as a build step within jobs.

If these do not suit your needs, try looking at the Jenkins plugins list

---
Tags: jenkins-plugins
---


# Question
Title: What does implementing Hashicorp Vault mean in practice?
I have read several Hashicorp Vault tutorials and one of the features that could be used is generating ssh keys for git clone repositories, but that implies that everybody in the company that clones repositories should install vault and an helper like https://github.com/hashicorp/vault-ssh-helper

# Answer
In the hope of resurrecting this question, I will give my opinionated answer.

As I understand it, the use of Vault is part of the drive to give **more autonomy to teams**, by removing fear of leaking or changing secrets. By providing somewhere to store and manage secrets, ops can give dev the means to continue developing at their own pace, following the practices of continuous integration and working in small batches.

With a product like Vault in the pipeline, secrets can be consumed safely in the various phases - build, test, bake, deploy, etc. Implementing Vault in practice means less personal interaction with the service, but more machine interaction with the service, from trusted components in the pipeline.
> 1 votes

---
Tags: culture, hashicorp-vault
---


# Question
Title: Is there one website that contains all relevant DevOps news and gets updated on a daily basis?
I am looking for some DevOps news channel like https://nos.nl/teletekst#101 that shows the headlines of the Dutch news. I am aware of https://tweakers.net/ and github.com, but this does not contain all interesting news on one page.

Basically, I would like to receive updates regarding security, features, bugs about several products, e.g. k8s, docker, docker-compose, ubuntu, cloud, AWS, GCP, Azure, centos.

# Answer
There is a protocol for this: RSS, Rich Site Summary. Technically, there are 3 protocols RSS, RSS2 and Atom, but most readers support all of them, and many servers mix RSS and RSS2 features.

First, choose your RSS reader, from:

Second, choose the sources you care about and subscribe to them - you're the curator and most news websites publish RSS feeds. Many don't even know about it, as CMSs like Wordpress sometimes publish an RSS feed by default. I wouldn't be surprised if it had something to do with SEO... Soon, you'll have an OPML list of your favourites.

From your question:

And a few personal recommendations:

Many readers will have an option to search for a feeds on a website, which simplifies the search.
> 6 votes

# Answer
Probably not 100% comprehensive, but

DevOps.com

might fit the bill?
> 2 votes

# Answer
I've had good success with DZone: https://dzone.com/devops-tutorials-tools-news. This is a nice mix of articles, tutorials, and various publications.

For Docker specific updates/news I follow: https://www.docker.com/blog/

Finally, for specific cloud platforms each one seems to have their own community blogers that post updates. For example:

1. Amazon - https://aws.amazon.com/blogs/devops/
2. Microsoft - https://devblogs.microsoft.com/devops/
3. Google - https://cloud.google.com/blog/products/devops-sre
> 2 votes

# Answer
There is https://www.infoq.com/devops/ which is a good news feed. I get the newsletter and it is a good overview of trends, etc.

I also check https://aws.amazon.com/blogs/devops/ regularly, but that is AWS specific of course.
> 1 votes

# Answer
I realize these cover much broader ground, but I see lots of useful DevOps-related stories on https://slashdot.org/ and Hacker News. I also have created saved searches in Google News.
> 1 votes

# Answer
I'd add https://opensource.com/ (from Redhat) and the O"reilly Newsletter related to Infrastructure and Ops : https://conferences.oreilly.com/infrastructure-ops/io-ca
> 1 votes

# Answer
I'm also a fan of DZone. Another good place is DevOps.com Also, the Sonatype blog and All Day DevOps regularly feature DevOps news, too \[disclosure, I contribute to all of these\]
> 1 votes

---
Tags: docker, kubernetes, docker-compose, cloud, elastic
---


# Question
Title: Azure DevOps Build and Release pipeline artifact name variable
Trying to do something simple. Either access the name of the artifact from my release pipeline via some predefined variable, or create a variable for both my build and release pipeline for the artifact name. I don't want to hard-code it. I would like my build and release pipelines decoupled.

Publish Build Artifacts YAML:

```
- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: 'dist'
    **ArtifactName: $(Storage.ArtifactName)**
    publishLocation: 'Container'
  displayName: 'publish artifact'

```

Output:

```
##[section]Starting: publish artifact
==============================================================================
Task         : Publish build artifacts
Description  : Publish build artifacts to Azure Pipelines or a Windows file share
Version      : 1.158.1
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/publish-build-artifacts
==============================================================================
##[section]Async Command Start: Upload Artifact
Uploading 10 files
File upload succeed.
**Upload '/home/vsts/work/1/s/dist' to file container: '#/3928702/$(Storage.ArtifactName)'**
Associated artifact 10 with build 58
##[section]Async Command End: Upload Artifact
##[section]Finishing: publish artifact

```

# Answer
> 1 votes
I needed to add:

```
variables:
- group: front_end-storage

```

At the top of my pipeline for the pipeline to pick up on the variables and authorize the group.

---
Tags: azure-devops
---


# Question
Title: Is it reliable to use Docker's hash as a mean to determine image's version between different platform
**Situation**

I have a docker image on a Linux box. I push it onto a docker registry (AWS ECR).

Then, I use the image pushed onto that docker registry in a K8s pod.

**Question**

How do I know the docker image used for the K8s pod is the same as the one on the Linux box? Note that the tag is irrelevant. I set it to different things all the time.

# Answer
> 3 votes
Yes, you can use the *repository digest* for this. Note that while this is a hash of the container, it's a hash specific to the repository, and is **NOT** the image id, which is a separate sha256 hash.

Once you push or pull a docker container to/from a registry, it acquires some repository metadata, including the repository digest for that container. You can see that digest by running `docker inspect <container-tag-or-id>` and looking for the `RepoDigests` section of the json (or just pipe that command into `jq '.[].RepoDigests'` if you have it installed.

Now you can compare that with the docker pullable of the kubernetes pod you want to check by running this command:

`kubectl get pod <POD> -o jsonpath='{.status.containerStatuses[*].imageID}'`

If the hashes of both of those values match, then you can be sure they are running the same container.

---
Tags: docker, kubernetes
---


# Question
Title: Can use a private IP address as the puppet certname?
Im very new to puppet but know in /etc/puppetlabs/puppet/puppet.conf I can set cert name to a url form following linux accadamy vidios but can you also set it to a private ip address. Im trying to set up a puppet server and agent on my personal aws account replicateing what i did on my linux accadamy lab servers on ec2s except im trying to get it working with amozon ec2s rather than ubuntu 18.04.

I don't have a url set up for my ec2s as that costs extra can I use the ec2s private ip addresses rather than a url?

# Answer
You can indeed use IP address in the "certname" directive of puppet.conf file.

In addition, you can create a private hosted zone in AWS Route53 that is called for example lab.local and then create an A record for your puppet server such as for example puppetmaster.lab.local and then use this address in the certname directive.

Opening a hosted zone in Route 53 doesn't cost much... around 0.50$ per month for your first 25 zones.

The full Route53 pricing can be found here.
> 1 votes

---
Tags: puppet
---


# Question
Title: Is there a way to clone an ec2 from inside the ec2 terminal?
I want to be able to back up my linux accadamy lab servers but they don't provide a way to to this.

I'm wondering if it's posible to clone my EC2 lab server from inside the EC2 to get round this?

# Answer
> 2 votes
Yes, it can be done.

You can either create an IAM instance role which allows the instance to use ec2 actions such as "create-image" or you can create an IAM user and create for this user access and secret keys and a policy which allows this user to create images in ec2 and then configure the access and secret keys locally on the server using the "aws configure" command.

Whatever method you choose from the above options, the next step would be from within the instance itself and assuming you have aws-cli installed in the instance, first run:

`ec2-metadata -i` to get the instance id.

and then run:

```
aws ec2 create-image --instance-id INSTANCEID --name NameOfImage --no-reboot --region REGION

```

And it will create an image of this instance in the region you provided without rebooting the instance.

The create-image command will create an image of the instance, also called AMI which then you can use to create new instances from.

---
Tags: amazon-ec2, linux
---


title: How to run aws cli with www-data?

text:
# Question
Title: How to run aws cli with www-data?
I can successfully upload a file to my S3 bucket using the AWS cli version 1 with my normal linux user. However, I'm having trouble doing the same with Apache2's www-data.

I have a personal project where I have a web form and after submission, I use shell\_exec to run some shell commands. I've tried the following command but it does not seem to work and I'm not getting any errors so debugging is a bit hard:

```
shell_exec('/home/pi/./local/bin/aws s3 cp /path/to/test.txt s3://s3-bucket-name/');

```

If it makes any difference, I'm doing this on a Raspberry Pi. I'm suspecting it has something to do with the fact that AWS CLI has been installed under the normal user I use and www-data cannot access it.

Any suggestions?

Edit: One solution I found was to use allow www-data to run commands with another user without prompting for password. So this worked then:

```
shell_exec('sudo -u pi /home/pi/.local/bin/aws s3 cp /path/to/test.txt s3://s3-bucket-name/');

```

# Answer
I'd say you can:

1. Using pip, install aws-cli for the www-data user specifically. Example:

pip install awscli --user
2. Copying aws binary to some folder (for example awsonlybin) where it is the only command there and then adding the path of this folder to the www-data user's PATH. Example:

export PATH=$PATH:/usr/local/awsonlybin

In addition to that you should also do one of the following options:

* Create an IAM user and access and secret keys and attach a relevant IAM policy to that user and then configure awscli to use that access key and configure them under the www-data user.
* Create an IAM role and attach a relevant policy to that role and attach the role to that instance.
> 1 votes

---
Tags: amazon-s3
---


# Question
Title: Placing an AWS Lambda in a public subnet
Is it possible to place an aws lambda in a public subnet and thus avoid paying for `NAT`?

The docs state the following

> If your Lambda function needs Internet access, do not attach it to a public subnet or to a private subnet without Internet access. Instead, attach it only to private subnets with Internet access through a NAT instance or an Amazon VPC NAT gateway.

But do not explain why we can not attach the lambda to a public subnet.

# Answer
> 5 votes
No, it's not possible to place an AWS Lambda in a public subnet and thus avoid paying for NAT. Lambda functions cannot have public IPs so they cannot route to the internet without a NAT (gateway or instance).

Put them in a private subnet, ensure the private subnet's default route is a NAT in a public subnet, that the NAT has a public IP, and that the VPC has an IGW.

For more, see this Stack Overflow question.

# Answer
> 2 votes
Provided the *Public Subnet* where you're launching your Lambda is configured with **Auto-assign public IPv4 address: Yes** it should work in public a subnet without NAT.

Whether or not it's a good idea is another question. In general you shouldn't launch any backend services in the public subnets, the public subnets are for things like API Gateways, NAT Gateways, Load Balancers, etc. Not for launching the actual backend systems. That's the best security practice.

However that aside I'm pretty confident that with *auto-assign IPv4 address* enabled Lambdas will work even in the public subnet.

Unless you want the Lambda to connect to any VPC-internal resources (e.g. to your RDS) you may just as well launch it without VPC config (leave those VPC and Subnet settings empty). That will give it access to the internet without having to worry about NAT or public IPs.

Hope that helps

---
Tags: amazon-web-services, aws-lambda, aws-vpc
---


# Question
Title: How well can agile pair programming work in interdisciplinary DevOps contexts?
In agile, pair programming can be performed like the following:

* two specialists of same discipline change the place before the terminal and each has a chance to code and to conceptualize/reflect;
* two specialists of different disciplines work together, but one is rather a customer and cannot code much in the target language; also the coder might not know much about the requirements which then discussed directly and coded and tested and deployed.

How is this relevant/appliable in DevOps context? (facts)

# Answer
Agile fits very well with a DevOps culture and pair programming is no exception.

The following are some examples of "pair programming" principles applied in a DevOps context:

* A pair of individuals working together on a Dockerfile for their service.
* A pairing sits together to work on defining their infrastructure as code.
* A traditional developer and an IT administrator building a CI/CD pipeline together.
* A specialist and tester sit together to define what the testing environment should look like.
* A specialist and a customer sitting together and working on a deployment to the customers infrastructure.
> 2 votes

---
Tags: deployment, culture, agile, software-engineering
---


# Question
Title: Where do I set that agent certificate for Puppet on Amazon Linux?
I am using:

* Puppet server version: 1.2.0
* Amazon Linux 2

I have master and agent set in /etc/puppet/puppet.conf and currently I get back only the certificate for the master on the master. The agent's DNS name under \[agent\] and \[main\] is

certname = doug2c.mylabserver.com

but

`puppet cert list -all` returns:

```
+ "doug1c.mylabserver.com" (SHA256) C2:AE:78:00:8D:EE:40:09:06:26:10:AD:82:47:58:26:C7:86:F5:A1:8A:D8:B1:BE:81:C2:AF:D1:70:F9:04:EF (alt names: "DNS:doug1c.mylabserver.com", "DNS:puppet", "DNS:puppet.mylabserver.com")

```

I tried useing find to locate where it was set to puppet with

```
find / -name doug1c

```

but that didn't work.

# Answer
On the agent I needed to set the server ip set to puppet in /etc/hosts

I had accidentally put it on the agents ip address and then been incorrectly assuming i'd done something wrong on the master rather than the agent.

(server ip) (domian name) puppet
> 1 votes

---
Tags: dns, puppet
---


title: How to redirect tomcat logs TO console?

text:
# Question
Title: How to redirect tomcat logs TO console?
So I have a problem that is the reverse of what everyone else on the internet wants.. I want to take tomcat logs that are being written out to /usr/local/tomcat/logs and have them be output to console instead. (Everyone else is asking the reverse.. how to get things away from console and to file). Reason why is because I'm on a system that captures output to console and directs it to an analytics program, it has to go to this program, and I have no control over how this is designed (ie. I can't change the program or capture process to use files as well as console).

To clarify, I am doing this on a Linux system, specifically Alpine.

# Answer
There is Tomcat configuration file for the logging: `TOMCAT_HOME/conf/logging.properties`. Edit the file and remove all the handlers except the console handler:

```
.handlers = java.util.logging.ConsoleHandler

```

This will direct all logging to the console, **except** for access logs (localhost\_access\_log.XXXX.txt). It is always written to file by the AccessLogValve, configured in 'server.xml'. To redirect that one you need to configure own implementation of the AbstractAccessLogValve... which requires coding.
> 2 votes

# Answer
That doc say next:

> When running Tomcat on unixes, the console output is usually redirected to the file named catalina.out. The name is configurable using an environment variable. (See the startup scripts). Whatever is written to System.err/out will be caught into that file. That may include:
>
> Uncaught exceptions printed by
>
> * java.lang.ThreadGroup.uncaughtException(..)
> * Thread dumps, if you requested them via a system signal
>
> When running as a service on Windows, the console output is also caught and redirected, but the file names are different.
>
> The default logging configuration in Apache Tomcat writes the same messages to the console and to a log file. This is great when using Tomcat for development, but usually is not needed in production.

So if you need to catch the same logs which catch analytic software you need read `catalina.out` file.

Think it's will be helpful to you.
> 1 votes

---
Tags: linux, logging, tomcat
---


# Question
Title: How to resolve the dependencies while installing locally downloaded packages?
I have simple playbook where trying to install the Debian packages downloaded locally in my server. This playbook runs on localhost and install the Debian packages in the same system. But this playbook gives an error **"Dependency is not satisfiable** for some of the packages even tough the dependency package is available in the local repository.

I can download all the dependencies required for the specific package in my local repository using **apt-get install --download-only package\_name**

But in my playbook, i should have a mechanism to install the dependencies first than install the actual package. This task should be dynamic, playbook should resolve the dependencies by itself for any package install.

***When the package has dependency over another package, How the playbook resolve dynamically.***

Some of the options explored:

Using ordered indexed\_items, using gdebi.. Looking for efficient Logic.

**install\_dependencies.yml**

```
---
- hosts: localhost
  vars:
    remote_media_directory: "/home/local_repository"
  become: yes
  tasks:
    - name: find all debian Packages
      find:
         paths:
             - "{{ remote_media_directory }}"
         file_type: file
         recurse: yes
         use_regex: yes
         patterns:
           - '.*deb$'
      register: files_matched_subdirectory

    - name: installation debian packages
      apt:
        deb: "{{ item.path }}"
      with_items: "{{ files_matched_subdirectory.files }}"
      when: ansible_distribution == "Ubuntu"

```

> ansible-playbook install\_dependencies.yml

```
PLAY [localhost] **********************************************************************************************************************************************************************************************

TASK [Gathering Facts] ****************************************************************************************************************************************************************************************
ok: [localhost]

TASK [find all debian Packages] *******************************************************************************************************************************************************************************
ok: [localhost]

TASK [installation debian packages] ***************************************************************************************************************************************************************************
failed: [localhost] (item={u'uid': 0, u'woth': False, u'mtime': 1571850269.5720358, u'inode': 31195175, u'isgid': False, u'size': 46700, u'roth': True, u'isuid': False, u'isreg': True, u'pw_name': u'root', u'gid': 0, u'ischr': False, u'wusr': True, u'xoth': False, u'rusr': True, u'nlink': 1, u'issock': False, u'rgrp': True, u'gr_name': u'root', u'path': u'/home/local_repository/wireshark_3.0.5-1_amd64.deb', u'xusr': False, u'atime': 1571850391.3553286, u'isdir': False, u'ctime': 1571850312.2437878, u'wgrp': False, u'xgrp': False, u'dev': 2051, u'isblk': False, u'isfifo': False, u'mode': u'0644', u'islnk': False}) => {"changed": false, "item": {"atime": 1571850391.3553286, "ctime": 1571850312.2437878, "dev": 2051, "gid": 0, "gr_name": "root", "inode": 31195175, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mode": "0644", "mtime": 1571850269.5720358, "nlink": 1, "path": "/home/local_repository/wireshark_3.0.5-1_amd64.deb", "pw_name": "root", "rgrp": true, "roth": true, "rusr": true, "size": 46700, "uid": 0, "wgrp": false, "woth": false, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}, "msg": "Dependency is not satisfiable: wireshark-qt (= 3.0.5-1)\n"}
failed: [localhost] (item={u'uid': 0, u'woth': False, u'mtime': 1571850269.4560366, u'inode': 31195174, u'isgid': False, u'size': 3605060, u'roth': True, u'isuid': False, u'isreg': True, u'pw_name': u'root', u'gid': 0, u'ischr': False, u'wusr': True, u'xoth': False, u'rusr': True, u'nlink': 1, u'issock': False, u'rgrp': True, u'gr_name': u'root', u'path': u'/home/local_repository/wireshark-qt_3.0.5-1_amd64.deb', u'xusr': False, u'atime': 1571850394.3993108, u'isdir': False, u'ctime': 1571850312.2437878, u'wgrp': False, u'xgrp': False, u'dev': 2051, u'isblk': False, u'isfifo': False, u'mode': u'0644', u'islnk': False}) => {"changed": false, "item": {"atime": 1571850394.3993108, "ctime": 1571850312.2437878, "dev": 2051, "gid": 0, "gr_name": "root", "inode": 31195174, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mode": "0644", "mtime": 1571850269.4560366, "nlink": 1, "path": "/home/local_repository/wireshark-qt_3.0.5-1_amd64.deb", "pw_name": "root", "rgrp": true, "roth": true, "rusr": true, "size": 3605060, "uid": 0, "wgrp": false, "woth": false, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}, "msg": "Dependency is not satisfiable: libc6 (>= 2.29)\n"}
ok: [localhost] => (item={u'uid': 0, u'woth': False, u'mtime': 1571850217.1323407, u'inode': 31195169, u'isgid': False, u'size': 40654, u'roth': True, u'isuid': False, u'isreg': True, u'pw_name': u'root', u'gid': 0, u'ischr': False, u'wusr': True, u'xoth': False, u'rusr': True, u'nlink': 1, u'issock': False, u'rgrp': True, u'gr_name': u'root', u'path': u'/home/local_repository/tree_1.7.0-5_amd64.deb', u'xusr': False, u'atime': 1571850397.4392931, u'isdir': False, u'ctime': 1571850312.2437878, u'wgrp': False, u'xgrp': False, u'dev': 2051, u'isblk': False, u'isfifo': False, u'mode': u'0644', u'islnk': False})
ok: [localhost] => (item={u'uid': 0, u'woth': False, u'mtime': 1571850217.0163412, u'inode': 31195168, u'isgid': False, u'size': 52132, u'roth': True, u'isuid': False, u'isreg': True, u'pw_name': u'root', u'gid': 0, u'ischr': False, u'wusr': True, u'xoth': False, u'rusr': True, u'nlink': 1, u'issock': False, u'rgrp': True, u'gr_name': u'root', u'path': u'/home/local_repository/dstat_0.7.3-1_all.deb', u'xusr': False, u'atime': 1571850399.3192823, u'isdir': False, u'ctime': 1571850312.2437878, u'wgrp': False, u'xgrp': False, u'dev': 2051, u'isblk': False, u'isfifo': False, u'mode': u'0644', u'islnk': False})
        to retry, use: --limit @/home/upgrade/ansible/install_dependencies.retry

PLAY RECAP ****************************************************************************************************************************************************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=1

```

# Answer
Any specific reason that you aren't using your own *real* package repository? If you use Ansible you seem to see the benefit of automating things, so this would be a natural step. And it would enable the appropriate installation order merely based on the dependency graph and the packages available. There are other tools as well for the job, but `reprepo` is easy to use and seems ideal for smaller scale package repositories.

You don't give information about the system *on which* this playbook fails. But the symptoms are rather clear. And none of that has anything to do with Ansible specifically.

* `/home/local_repository/wireshark_3.0.5-1_amd64.deb` could not be installed because it requires *exactly* `wireshark-qt (= 3.0.5-1)` (i.e. exactly that version of the `wireshark-qt` package)
* `/home/local_repository/wireshark-qt_3.0.5-1_amd64.deb` (the dependency named in the previous item) failed because its dependency `libc6 (>= 2.29)` was not available.

In order to figure out what exactly is going on you would have had to give information about the system. However, I can tell you how you can find out more details.

Simply use `apt-cache policy <package-name>` on both the dependencies (`wireshark-qt` and `libc`) to see which versions of the packages are at all available.

If I were to guess your problem is that you attempt to install this on a system which is older than the prerequisites required for the packages you attempt to install. And trying to forcibly install a newer `libc` package is a really bad idea, given its role in the system. It could leave your system unusable.

For example if you were attempting this on the current Debian 10 (i.e. currently Buster==stable), you can see in this table (contents are bound to change, so I am including the screenshot) that alone the `libc6 (>= 2.29)` prerequisite cannot be satisfied:

Alternatively you can look here and navigate between the current releases in the top right (at the time of this writing):

The first step to figure out whether there is a newer package available would be to look at the package search for your current release. To find out what that is, use `lsb_release -a` (to see all information) or `lsb_release -cs` to see the short codename (e.g. `buster`).

Here is a similar overview for Ubuntu:

Looking at the package versions you attempt to install, I'd guess you are attempting to install the `eoan` (19.10) package on an older Ubuntu, am I right?

You *could* attempt to download the sources (`apt-get source <package-name>`) of the two packages in question (`wireshark` and `wireshark-qt`) and attempt to build it yourself. If you're lucky you can build that new Wireshark version against whatever Qt and libc versions are included in your specific release. Arguably Qt (and *its* dependencies) would be more likely to be an issue than libc.

Alas, chances are that it will be easier to upgrade your system to 19.10, wait ~4 months until `focal` aka Focal Fossa (Ubuntu 20.04) gets released or install a VM or container enabling you to run that newer Wireshark version.

But again, none of that has anything to do with Ansible, except that you used Ansible to invoke the installation commands.
> 2 votes

---
Tags: linux
---


title: Database Backup using jenkins

text:
# Question
Title: Database Backup using jenkins
I would like to take MSSQL server db backup. Is there any plugin available for it.

# Answer
I don't know of any current plugins for doing this. That may because a lot of backup functionality is built into MSSQL.

The easiest way to automate a backup via Jenkins may be to use a command line script to use a sql command:

```
Sqlcmd -E -S touch -Q" BACKUP DATABASE [DBName] TO DISK = 'C:\artifact_directory\my_db_backup.BAK'
```

You could also also avoid automating this via Jenkins and just create a scheduled backup directly within SQL Server.
> 2 votes

---
Tags: jenkins, continuous-integration, jenkins-plugins, cicd
---


# Question
Title: Where is Puppet's enviroments directory on Amazon Linux2?
I am using Puppet version 3.8.7 on Amazon Linux2, trying to find the directory containing the Puppet environments.

I tried to find it with the `find` command looking for files named "production" or defaults, following the names used in a tutorial using ubuntu, but there is no production or default file or directory on the Amazon Linux as there is on ubuntu version.

The distribution of Puppet I downloaded was

http://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm

# Answer
> 1 votes
You tried look on `/opt` or `/etc` probably the puppetlabs folder must be in one of these directories. Amazon Linux it's based on CentOS/RedHat family.

---
Tags: amazon-ec2, linux, puppet
---


# Question
Title: How to connect GCP Load balancer to Kubernetes cluster(single node cluster)
I'm creating a single node(name: node1) k8 cluster for demos. I have setup a nginx-ingress controller.

```
service/volted-marmot-nginx-ingress-controller        LoadBalancer   10.100.132.29   <pending>     80:30787/TCP,443:30043/TCP   3h48m

```

I can access it by node1\_ip:30043 for e.g. How do I put a load balancer in front of it? What I want is to have loadbalancer\_ip:443 point to node1\_ip:30043. If that's the wrong way to think of it, please let me know. I can't do it anyway, as the target pool does not have any option to choose port(e.g. 30043) anyway.

If I set the loadbalancer port to 300043 then loadbalancer\_ip:300043 works but I was hoping to hide that.

# Answer
You can use the GCP `HTTP(S)` Load balancer. Since you are running a kubernetes cluster (Assuming you are using GKE), GKE uses Google compute VM instance groups which can be put behind a load balancer as target group.

You can follow the below steps to setup a load balancer and put your kubernetes node behind a load balancer.

1. Login to GCP console and go to `NETWORKING` \> `Network Services` \> `Load balancing`
2. Click on `Create Load Balancer` at the top of the page.
3. Click on `Start Configuration` for `HTTP(s) Load balancing`
4. Select `From Internet to my VMs` if you want to access it from public internet and click `continue`
5. Give a `Name` to your load balancer
6. Click on `Backend Configuration` and Select `Backend services` (from drop down) \> `Create a backend service`
7. Give a name to backend service and select the other parameters appropriately. Select `Instance groups` in `Backend type`.
8. In the `New backend` section, select the kubernetes instance group name from the drop down. After selecting the proper instance group, you can enter the ports which will accept the requests. You can modify other things if required. At the end, click `Create`.
9. Next you can setup `Host and Path rules` if you need any.
10. Next, you have to configure `Frontend configuration` which will be your frontend to accept the requests on the port and send it to the backend target group which we configured in the step 8.
11. Once all the configurations are done, click on `Create` to create your load balancer.

If everything was setup correctly, you should be able to access the contents from the load balancer endpoint.
> 2 votes

---
Tags: kubernetes, load-balance, gcp
---


# Question
Title: How can I find out when Terraform will support newly released AWS services?
For example, AWS announced their new "RDS Proxy" service at re:Invent 2019 and it is now in preview. How can I find out when/if Terraform will support this (for planning purposes)?

https://aws.amazon.com/about-aws/whats-new/2019/12/amazon-rds-proxy-available-in-preview/

# Answer
> 2 votes
Terraform is an open source project that is managed on GitHub. You can follow existing issues or open a new one if your enhancement request isn't found. See their issue list at: https://github.com/hashicorp/terraform/issues

As Martin Atkins points out in the comments, the plugin specific features are each developed and tracked in their respective repositories. Here's the one for AWS: https://github.com/terraform-providers/terraform-provider-aws

# Answer
> 1 votes
Hashicorp is usually quite quick in rolling out support for new AWS services. You can also follow their blog regarding announcements for more such services in the future. Although it does not mention support for RDS Proxy yet, For other services announced during AWS:ReInvent 2019, they announced support during the reinvent week itself on Dec 04th, 2019. https://www.hashicorp.com/blog/hashicorp-re-invent-19-terraform-supports-newly-announced-aws-services

---
Tags: terraform
---


# Question
Title: When to "npm publish" when using git flow
I'm using the following resources as a guide to git flow:

However, none of them explicitly mention at what point you perform the npm publish from. Do you do this after you have bumped the version number but before you finish the release? This would mean the publish is coming from the release branch. Or do you finish the release then perform the publish from the master branch?

Thanks.

# Answer
I'd set up my CI system to do this on every commit to `master`, that is, after the release branch merges.

The first very straightforward reason for this is that the commit to `master` gets tagged with the release, and you want the thing you ultimately publish to match exactly the version of the source code at the tag. While there shouldn't be any merge surprises if git flow is run perfectly, it's still better to publish the content of a tag and not the commit immediately before a merge.

The second reason is that it is a very simple rule that can be automated. If you do any amount of final testing on the release branch, you need to make a manual decision as to whether or not you can release or not; your CI system (probably) can't make that decision just based on the presence of a release branch. On the other hand, since every commit to `master` is merging in a release (or hotfix) branch that you've fully validated, the commit itself is a signal that you're ready to release.
> 4 votes

---
Tags: git, release-management, release, npm
---


# Question
Title: private subnets in an Auto Scaling Group on aws?
I watched a video series wherein the author used 3 private subnets for his Auto Scaling Group, and I also read the premium answer on this forum wherein the author says that you can use both public and private subnets in an Auto Scaling Group:

https://acloud.guru/forums/aws-certified-solutions-architect-associate/discussion/-K\_JalpbCBG\_pUIbNc4k/vpc-private-subnet-and-autoscaling

> "When you create the "Auto Scaling Group" you will specify your VPC and as many of the related "Subnets" that you want to launch instances into - it does not matter if they are private or public it is entirely up to how you have designed your solution."

So I tried it. I use three private subnets in my Auto Scaling Group and the website does not load from the public DNS or the public IP. I notice the IP of the instance belongs to one of the private subnets, 10.0.4.135 (the subnet is 10.0.4.0). Also, the issue is not with Security Groups, because that instance above has a security group allowing all outbound traffic and inbound traffic from 80, 443 and 22.

Now if I used 3 public subnets, then the site will load! So what's the difference? The private subnet uses a NAT Gateway and the public subnet uses an Internet Gateway. It makes sense that traffic CANNOT go to this private subnet since there is no internet gateway to it; that's the whole purpose of having a private subnet.

So then why does the premium member state that an Auto Scaling Group can use both public and private subnets?

# Answer
There is a range of common scenarios when you want to use private subnets to be used in an auto scaling group:

* Your traffic ~~is terminated by~~ reaches your infrastructure on a Elastic Load Balancers and your web server instances are behind the load balancer. This reduces the attack surface of the web servers.

( I think I started to use "terminate" since seeing it being used for SSL/TLS termination, ie the server(s) that hold the secure certificates. But because you could also use a Network Load Balancer at the "edge" of your infrastructure, I see now its not appropriate to use in this context)

* Your auto scaled resources are actually not responding to outside traffic, for example if they are worker resources processing messages from an SQS queue.

There was a similar mention of both "public and private subnets" all over my LinuxAcademy exam preparation, depending on your thinking processes it may be straightforward to just say: "autoscaling can use subnets", but then this may confuse other people.
> 1 votes

---
Tags: amazon-web-services, amazon-ec2, autoscaling
---


# Question
Title: how to prevent Ansible from showing output on screen?
I have this in my playbook which execute a command on remote server and enters its log on local server.

```
 - name: run script
   shell: runuser -l testuser -c "/tmp/test.sh"
   register: myshell_output
 - name: copy output to a local file
   lineinfile:
     dest: /thesaurus/output
     line: "{{ item }}"
     insertafter: EOF
   with_items:
    - "#####################Beginning##########################"
    - "{{ myshell_output.stdout }}"
    - "########################END#############################"
   delegate_to: localhost

```

the output is always shown on screen and troubleshooting gets hard and distracting.

```
TASK [run script] ************
changed: [1.1.1.1]
changed: [2.2.2.2]
TASK [copy output to a local file] ***********
changed: [1.1.1.1 -> localhost] =>
(item=#####################Beginning##########################)
 ok: [2.2.2.2 -> localhost] =>
(item=#####################Beginning##########################)
{
  REALLY LONG AND BIG OUTPUT!
}
changed: [1.1.1.1 -> localhost] =>
(item=########################END#############################)
ok: [2.2.2.2 -> localhost] =>
(item=########################END#############################)

```

is it possible to prevent Ansible from printing these output on screen and just insert output to a file?

# Answer
> 2 votes
This can be done by judicious use of one of the Ansible callback plugins. Set the relevant configuration in your `ansible.cfg` file.

You have a few options...

In order to print *nothing* to the screen, you can use the null callback:

> This callback prevents outputing events to screen

However, this may be overkill and doesn't do what you want, which is to write the output to a file. This can be done with either of the

modules.

I would probably suggest the log\_plays module in your case.

Note: This is going straight from the documentation. I haven't used the tree or log\_plays plugin myself, opting either for the yaml or profile\_tasks one, to make things more readable. It also seems like you want to filter *just a part* of the output, not all of it. You may want to use a combination of filters in order to get the desired effect.

---
Tags: ansible, linux, yaml, centos
---


# Question
Title: Passing parameters from Jenkins to Ansible playbook
How can I pass parameters from Jenkins to my Ansible playbook? Eg. I want to take IP address from users and pass it to the playbook so that it will execute only on those machines.

# Answer
You can use -e switch for that purpose:

```
ansible-playbook -e 'foo=${FOO}'

```

where the FOO can be defined in the Jenkins envirionment variables. If it is defined within the parameters block of your Jenkinsfile, then you need to use `${params.foo}`. Then, the value can be accessed by the Ansible playbook with `{{ foo }}`.
> 2 votes

---
Tags: jenkins, ansible
---


title: Updating software in a docker container

text:
# Question
Title: Updating software in a docker container
I'm new to this community, so feel free to tell me if this is a dup or belongs in another SE.

I am new to docker, and it's a really cool concept, but I'm struggling with the idea of packaging an app that will be updated later. Suppose, for example, that I've packaged an app that uses a postgres database that I included in the same container. When I build a new image going from postgres 11 to postgres 12, I need to migrate the data directory. How is that normally handled with docker?

My example might be bad, because proper design would have the DB in another container, but suppose I want to upgrade that postgres image, does the new postgres image handle the migration automatically? Is there a common way to run a script when the image is upgraded?

# Answer
Migrating data directory between major db version can be problematic.

I can suggest to create and restore db dump. You can upgrade postgresql db container as follow:

1. Backup database
2. Shutdown postgres 11 container
3. Run new postgres 12 container
4. Restore backup

or single line command:

```
docker exec postgres-11-container pg_dumpall -U postgres | docker exec -i postgres-12-container psql -U postgres

```
> 4 votes

---
Tags: docker
---


# Question
Title: Sonarqube stage in jenkins pipeline fails with 403 error
have a pipeline job which keeps failing on Sonar Quality Gate stage with the below error:

> \[Bitbucket\] Build result notified org.sonarqube.ws.client.HttpException: Error 403 on http://illinXXXX:XXXXX/api/qualitygates/project\_status?analysisId=XXXXXXXX at org.sonarqube.ws.client.BaseResponse.failIfNotSuccessful(BaseResponse.java:34)

What's more strange, another pipeline from the same MS, is passing that stage. They both are using the same Sonarquebe user and token, and the same stage syntax.

Sonarqube version: 6.7.1 (build 35068).

Notes:

* I also tried this, but it didn't work.
* The logger for org.sonarqube.ws.client not working. it doesn't write anything to the log.

# Answer
Problem: The execution user/group had permissions to run “Execute Analysis” Solution: Update the default template with permissions and push to projects (Bulk Apply Template)
> 0 votes

---
Tags: jenkins-pipeline, sonarqube
---


# Question
Title: Jenkins Pipeline Docker Agents with official Jenkins Docker image
I'm using the official `jenkins:lts` image to host a Jenkins instance, with the Docker control socket forwarded inside the container to allow Jenkins to start new containers, and now I'm trying to run a pipeline that starts with

```
pipeline {
    agent {
        docker {
            image 'toolchain:2011'
            args '-v ${WORKSPACE}:${WORKSPACE} -w ${WORKSPACE}'
…

```

Running this job tries to execute the `docker` command inside the Jenkins container, which doesn't work because the package isn't installed. The regular Docker plugin works fine, as it uses the control socket directly instead of trying to run external programs.

I suspect I'm not the first person to do this -- how would I set up a Jenkins instance inside a container that can execute a pipeline with a "docker" agent?

# Answer
> 3 votes
You have to extend the `jenkins/jenkins:lts` to install the Docker client:

```
FROM jenkins/jenkins:lts

# Switch to root
USER root

ENV DOCKER_VERSION      docker-18.06.3-ce
ARG DOCKER_GID=993 # put the correct docker gid

# Download and install docker client
RUN wget --quiet -O- \
  https://download.docker.com/linux/static/stable/x86_64/${DOCKER_VERSION}.tgz | \
  tar zx --strip-components=1 -C /usr/local/bin docker/docker \
  && groupadd -g ${DOCKER_GID} docker \
  && usermod -aG docker jenkins

# Switch back to jenkins user
USER jenkins

```

You can now declare pipelines with docker agents.

# Answer
> 0 votes
I used something like that in the simple flask app image:

```
stage('Deploy') {
        steps {
            sh 'docker build -t flask-sample:latest .'
            sh 'docker run -d --rm --name hello_app_flask -p 4010:4000 flask-sample:latest'
}

```

But I don't know wether it is approppriate option. Maybe, someone knows the 'proper' way to do it.

# Answer
> 0 votes
What are the permissions set to Jenkins docker container you are using in the instance?

This is one of the solution. You can use the following docker-compose along with the Dockerfile.

docker-compose.yml

```
version: "3.7"

services:
  jenkins:
    build:
      context: .
      dockerfile: Dockerfile
    image: jenkins
    container_name: jenkins
    restart: always
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "/etc/sysconfig/docker:/etc/sysconfig/docker"
      - "/home/ubuntu/jenkins/volume:/var/jenkins_home"
    ports:
      - "8443:8080"
    privileged: true

```

Dockerfile

```
FROM jenkins/jenkins:lts

#Switching to root user
USER root
RUN apt update && \
    apt install sudo curl jq apt-utils -y

RUN curl -sSL https://get.docker.com/ | sh
RUN  sudo apt-get install docker-compose -y

#install aws
RUN sudo apt-get install -y python-pip python-dev build-essential
RUN pip install awscli --upgrade --user
RUN export PATH=~/.local/bin:$PATH
#updating path to support aws
ENV PATH="~/.local/bin:${PATH}"

#php installation
#----------------
RUN apt install ca-certificates apt-transport-https
RUN wget -q https://packages.sury.org/php/apt.gpg -O- | sudo apt-key add -
RUN echo "deb https://packages.sury.org/php/ stretch main" | sudo tee /etc/apt/sources.list.d/php.list

RUN apt update
RUN apt install -y php7.2
RUN apt-get install -y php7.2-common php7.2-opcache php7.2-curl php7.2-mbstring php7.2-mysql php7.2-zip php7.2-xml

#install composer
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/bin --filename=composer

#install phpunit
RUN curl -sSL https://phar.phpunit.de/phpunit.phar -o phpunit.phar
RUN chmod +x phpunit.phar
RUN mv phpunit.phar /usr/local/bin/phpunit

```

Create a volume in the same directory named `volume`

This will allow you to run the docker command without any issue.

---
Tags: docker, jenkins, jenkins-plugins
---


# Question
Title: How to add shell script to jenkins freestyle job
I have the below shell script:

```
du -sh /bbhome/shared/data/repositories/* |sort -h |tail -20 |
while IFS= read -r line;do
        DIR=`echo $line | awk '{print$2}'`
        Rep=`cat $DIR/repository-config |grep 'project\|repo' |  tr '\n' ' '`
        Size=`echo $line | awk '{print $1}' `
        echo $Size $Rep
done

```

How can I run it thought Execute shell step in a Jenkins freestyle job?

I need also to add an ssh command to the environment. This should be executed using passwordless authentication.

Note: I don't want to connect to the environmnet and run this shell, but want to execute it directly from the box.

# Answer
For executing the command in the Jenkins Freestyle Job.

```
#!/bin/bash

du -sh /bbhome/shared/data/repositories/* |sort -h |tail -20 |
while IFS= read -r line;do
        DIR=`echo $line | awk '{print$2}'`
        Rep=`cat $DIR/repository-config |grep 'project\|repo' |  tr '\n' ' '`
        Size=`echo $line | awk '{print $1}' `
        echo $Size $Rep
done

```

For ssh

```
ssh -o StrictHostKeyChecking=no user@host '
   command_1 to execute
   command_2 to execute
   ./script.sh to execute
   command_3 to execute
'

```
> 1 votes

# Answer
You can specify the script in the "execute shell" step of the build. *Where* it gets executed will depend on what labels the job needs, and which nodes those labels are associated with.

It sounds like you want to execute this script on an agent. As long as the agent is available, you can request it via a label in the job configuration. It's a bit difficult to determine this without having access to your setup, but if *e.g.* you have a set of agents on Debian boxes, you could attach the label `debian` to them, and specify that the job should run on `debian` nodes in the job configuration.

If on the other hand it should be run on `master` (where Jenkins itself is running), you can specify the label `master`.

You thus need to put your script into the "Execute Shell" box that Jenkins provides you, perhaps declaring what shell and options you want to use:

```
/bin/bash -xe
du -sh /bbhome/shared/data/repositories/* |sort -h |tail -20 |
while IFS= read -r line;do
        DIR=`echo $line | awk '{print$2}'`
        Rep=`cat $DIR/repository-config |grep 'project\|repo' |  tr '\n' ' '`
        Size=`echo $line | awk '{print $1}' `
        echo $Size $Rep
done

```

> I need also to add an ssh command to the environment. This should be executed using passwordless authentication.

For this to happen, you need ssh keys to be exchanged with the host onto which you want to ssh into.
> 1 votes

---
Tags: jenkins-pipeline, shell-script
---


# Question
Title: Jenkins pipeline fails with No such property error
I have Jenkins container (version 2.138.1) with a pipeline which uses the plugins:

Job DSL version 1.7.0

Repository Connector version 1.1.3

P4 version 1.8.14

it fails with the below error:

```
11:23:17 ERROR: (int_dbp_ci.groovy, line 7) No such property: IntDBCIProperties for class: int_dbp_ci

```

what's strange, in the same machine, in other Jenkins container, for the same job with the same exact syntax and plugins version, it works. any idea what might cause the issue?

# Answer
> 1 votes
The issue was with one of the options present in global security configuration “ Enable script security for job DSL scripts”, which needs to be unchecked.

---
Tags: jenkins-pipeline, jenkins-plugins
---


# Question
Title: Relation/interplay of deployment pipeline stages (dev-stage-prod) with blue/green deployment mechanisms?
I'm building some new infrastructure in AWS and plan on having separate accounts for dev, stage, and prod environments. I'd like to implement a blue/green deployment strategy at least for production, and I've seen a few different ways this could be done, an obvious one being CodeDeploy's built-in blue/green deployment capability.

Someone suggested to me that the stage/prod environments could be thought of as the blue and green environments. They are, after all, supposed to be identical environments, but I hadn't thought of stage being something that could act as a production platform. I think stage is supposed to be for end-to-end testing, final verifications by QA, demos, etc.

So, I believe stage should continue to be for internal purposes only. But we may need two production accounts. At any given time each of them would be expected to contain a combination of inactive and active services (blues and greens). DNS (Route 53) would be in a shared account, and when a release occurs it would switch routings from resources in one account to resources in the other one.

Then, it kind of seems like having stage is less useful when you have blue/green. When a new release is in the inactive state and going through the most comprehensive tests, it's essentially what stage does. And we don't even really have QA--developers and our customers are the QA. So can we get rid of it? I have heard praises sung of having a straight-to-production strategy whenever automated testing is taken seriously enough.

Is this sensible?

# Answer
Yes. With no QA and wanting to have an internal environment separated from production this makes sense. However, a couple comments:

* You should go for whatever is going to make your business money instead of going for an enterprise standard. While some of the common practices may not be sensible yet, keep in mind that as your company grows (e.g. a dedicated QA role), they might eventually be very useful.
* If you get rid of any stages, you are essentially eliminating a development environment. Straight to production can be great, or greatly harmful.
> 1 votes

---
Tags: amazon-web-services, continuous-deployment
---


# Question
Title: Is it a bad idea to use a web app including nginx, gunicorn and flask in a single docker container?
I want to deploy a web app as a docker container. Currently I need to set up nginx, gunicorn and flask to realize this. There are many guides around to do it in docker but always divide it in two docker containers:

1. flask, gunicorn
2. nginx

Is it possible and a good idea to use all three components in just a single docker container?

# Answer
While this is possible it is not a best practice. Whenever possible, containers should contain a single process. The reasoning for this is:

* It fosters a modular design where you can more easily plug and play components. What if you decide not to use nginx and use asp.net core in the future? If everything is tightly coupled in one container then the entire container will need to be reworked any time one of these components changes. Look into microservices for more information on the benefits a modular design.
* It is easier to parse container logs as each container will have its own distinct output.
* If a process crashes and everything is in one container, the entire container will crash. If you have them broken out across multiple containers, recovery is simplified. While this may not be applicable for running standalone containers, it is very useful if the containers are running in a Kubernetes Pod as the Pod can restart.
> 1 votes

---
Tags: docker, python, nginx, flask
---


# Question
Title: Why does my Terraform config require variables in multiple places?
My folder hierarchy looks like this:

```
/
  -> live/
    : main.tf
    : variables.tf
  -> modules/
    -> logs/
      : logs.tf
      : variables.tf

```

In my `main.tf` I have:

```
module "logs" {
  source = "../modules/logs"

  env = var.env
  ...
}

```

In my `logs.tf` I have:

```
resource "datadog_monitor" "logs-fatal" {
  env = var.env
  ...
}

```

In both `variables.tf` files I have:

```
variable "env" {default = "test"}

```

If I remove this from either of them, I'm given an `Unsupported argument` error - why? It's unclear to me in this scenario which variable will be taking precedence and why both sets are needed.

# Answer
Each module has its own set of input variables.

The input variables for the root module are set when you run Terraform, e.g. on the command like with the `-var` option.

The input variables for any child module, like your `module "logs"` here, are set inside the `module` block.

Each variable declaration is valid only for the module it appears in. You happen to have given both of these variables the same name, which is fine and sensible as it helps a human reader understand that they represent the same concept, but Terraform considers these two variables entirely separate and requires you to declare and set both of them.

As an analogy, consider a module as being somewhat equivalent to a function in a programming language. Each function defines its own arguments, and two functions might have an argument of the same name but they are still distinct by virtue of belonging to different functions:

```
function main(env: string) {
  logs(env)
}

function logs(env: string) {
  // "env" in here is "test" because it was passed through
  // the arguments of both of these functions, not because
  // the arguments happened to have the same name in both
  // functions.
}

// Calling main here is, for the purposes of this analogy,
// corresponding to running: terraform apply -var="env=test"
main("test")

```
> 1 votes

---
Tags: terraform, terraform-module
---