# Question
Title: Ansible authorized copy from remote source to remote destination
How to use ansible authorized\_key to authorize a ServerA (not the controller machine) to access Server B.

Scenario: Need a playbook to execute from a ansible controller that should append id\_rsa.pub of a specific user from a remote ssh ServerA (no the controller machine ) to ServerB.

In our case the ServerA count is 20 while ServerB count is 200. What i have tried so far and failed.

```
---
# tasks file for passwordless
- name: Play to setup Passowrdless
  hosts: destnode
  remote_user: "{{ ssh_user }}"
  vars_prompt:
    - name: "ssh_user"
      prompt: "Please specify ssh user name"
      private: no

  tasks:
  - name: Copy pub key from SourceNode to Destnode
    authorized_key:
      user: "{{ ssh_user }}"
      state: present
      key: "{{ lookup('file', lookup('env','HOME') + '/.ssh/id_rsa.pub') }}"
    become: yes
    become_user: "{{ ssh_user }}"
    delegate_to: "{{ hostvars.sourcenode }}"
    when: ssh_user is defined
    register: copyout

  - name: Print copyout value
    debug:
      msg: "{{ copyout }}"

```

# Answer
> 1 votes
> To append id\_rsa.pub of a specific user from a remote ssh ServerA (no the controller machine ) to ServerB.

1) Fetch the public keys from ServerA

```
- host: ServerA
  vars:
    public_keys_dir: <PUB_KEYS_DIR>
    specific_user:
      - user1
      - user2
      - userN
  tasks:
    - name: Fetch pub keys
      fetch:
        src: "/home/{{ item }}/.ssh/id_rsa.pub"
        dest: "{{ public_keys_dir }}/{{ item }}-ServerA.id_rsa.pub"
        flat: yes
      loop: "{{ specific_user }}"

```

2) Configure authorized keys at ServerB

```
- host: ServerB
  vars:
    public_keys_dir: <PUB_KEYS_DIR>
    my_remote_user: admin
  tasks:
    - name: Set up authorized_keys
      authorized_key:
        user: "{{ my_remote_user }}"
        key: "{{ lookup('file', '{{ item }}')}}"
      with_fileglob:
        - "{{ public_keys_dir }}/*-ServerA.id_rsa.pub"

```

(not tested)

# Answer
> 0 votes
```
---
# tasks file for passwordless
- name: Play to setup Passowrdless
  remote_user: "{{ ssh_user }}"
  vars_prompt:
    - name: "ssh_user"
      prompt: "Please specify ssh user name"
      private: no
  hosts: all
  tasks:
  - name: Get pub key value from edgenode
    #hosts: edgenode
    command:
      cat $HOME/.ssh/id_rsa.pub
    register: pubkey
    changed_when: false
    when: "'group_edgenode' in  group_names"

  - name: Print copyout value
    debug:
      msg: "{{ pubkey.stdout }}"
    when: "'group_edgenode' in  group_names"

  - name: Append pub key value to datanodes
    #hosts: datanode
    lineinfile:
      path: $HOME/.ssh/authorized_keys
      #line: "{{ pubkey.stdout }}"
      #line: "{{ hostvars['192.168.0.103']['pubkey']['stdout'] }}"
      line: "{{ hostvars[item]['pubkey']['stdout'] }}"
      #line: "{{ groups['group_datanode'] | map('extract',hostvars, stdout) }}"
      state: present
      owner: "{{ ssh_user }}"
      #group: domain
      mode: 0600
      backup: yes
      insertafter: EOF
    #when: inventory_hostname in groups['datanode']
    with_items: "{{ groups['group_edgenode'] }}"
    when: "'group_datanode' in  group_names"

```

In this solution edgenodes are ServerA while datanodes are ServerB. Its tested and is working fine.

---
Tags: ansible, ansible-inventory, ansible-galaxy
---


# Question
Title: Elastic Beanstalk give error Root Object doesn't exist in Jenkins
I create a Jenkins job which uploads my war file to elastic beanstalk. But when I start to build it gives error `java.lang.IllegalArgumentException: Root Object doesn't exist`

Below is my configuration:

# Answer
> 1 votes
This error occurs because you used a variable ($) and wildcard name (\*.war).

Elastic Beanstalk plugin does not support Variable or wildcard name in the Value.

I suggest a use full path of Jenkins job instead of `${WORKSPACE}` and full war name like `ROOT.war` instead of `*.war`

---
Tags: jenkins, amazon-web-services, elastic
---


# Question
Title: Vagrant inside a VM to control the VM host
# Background

I have an application which comes with a Vagrant setup that is designed for this usage scenario:

```
Laptop (Windows, Virtualbox host) => edit source code
+----> Vagrant VM (Linux) => build and run

```

Plain, default, simple, works. The sources are on a shared/synced directory, i.e., they are usually edited on the Windows host and then compiled and ran inside the VM.

Unfortunately, I prefer to use Linux development tools myself. Right now, I have this setup:

```
Laptop (Windows)
+----> Vagrant VM
+----> My personal VM (Linux, non-vagrant)

```

So my editor etc. run in the personal VM, and I have a common directory shared through Virtualbox. Again, everything works just fine.

Here is the problem:

To control the vagrant stuff, I need to enter the `vagrant` commands in the Windows shell, which is slowing me down. For example, to run the build inside the vagrant VM, I would prefer to use a hotkey in my editor to start `vagrant ssh build.sh` instead of switching to the CMD.EXE and entering that command. Unneccessary context switch.

# Question

I don't wish to open the can of worms that is "virtualization inside virtualization" (i.e., I won't run Linux Virtualbox inside a Windows-hosted VM); so this keeps me with these two alternatives:

Is there a way to control the "Windows-Vagrant" that is running on the Windows host from inside a Linux VM (which itself is not under Vagrant control), i.e., to use `vagrant up` or `vagrant ssh`?

Alternatively, is it possible to install "Linux-Vagrant" in my existing VM right away (i.e. the Debian version) and have it create its VM in the *Windows* host?

Or is there a completely different way to solve this problem in an elegant fashion?

# Answer
Is the answer as simple as setting up a dual-booted machine? Linux partitions for development tasks and as the VM host, Windows partitions for normal computing use.

* Laptop
    + Windows partition
    + Linux Partition
        - Vagrant VM
> 1 votes

# Answer
One possible but kind of ugly and very brittle workaround could be to use winrm to launch the vagrant commands on windows from linux 'remotely'.

This need a networking setup for the VM so it can talk to the host through tcp/ip (configuration done on the virtualization you use), configuring winrm on the windows host (if your company allows it) and then using something like pywinrm (first found with a quick search) or any other tool in your language of choice.
> 1 votes

# Answer
I also wish to know this answer. Did you come up with anything yet? The only way I can think of is to install/create a proxy like service on the host that exposes the virtualbox/vmware command line for vagrant to use. Might also be possible for a small service on the host that listens/reads from a socket/file for incoming commands while the vm writes the commands to the socket/file. This socket/file would be in the shared directory. You would probably need 2 sockets (one for read, one for write). This could be turned into a vagrant plugin that would initially extend virtualbox plugin.

I am literally just spitballing here though. You probably can't stand up adhoc tcp servcies on you laptop but you might be able to get by with a socket/file based mechanism.
> 0 votes

---
Tags: vagrant, virtualbox
---


# Question
Title: Override Ansible task parameters from role
I'd like to be able to define a task with default parameters:

```
- name: Create a new user
  user:
    name: "default"
    password: "password"
    state: present

```

Then, from my playbook, I'd like to be able to do something like this:

```
  roles:
    - role: common
      vars:
        name: "frank"
        shell: "/bin/zsh"

```

...and have it create the user "frank", with the password "password" and the /bin/zsh shell. Is it possible to override task parameters from an Ansible playbook? (For the record, the above doesn't work, I tried it.)

# Answer
Defining default value for ansible variables and overriding them via ansible playbooks can be achieved using Jinja2 templating.

```
- name: Create a new user
  user:
    name: "{{ username_variable | default('default_value') }}"
    password: "{{ password_variable | default('default_value') }}"
    state: present

```

Then if these variables are passed from the playbook it will be used, or else the default values.

```
  roles:
    - role: common
      vars:
        username_variable: "frank"
        password_variable: "/bib/zsh"

```
> 2 votes

---
Tags: ansible
---


# Question
Title: How to connect to host during a Docker build?
I have a Dockerfile which I want to run some tests inside of as part of a multistage build. The tests attempt to make a connection to localhost, however it appears that they're not able to connect to the specific port (which is ready on the host machine) inside the RUN command of the Dockerfile. Is such a thing possible, and if so how do I achieve it? Obviously external connections can be made at build time (to download dependencies and such) and loopbacks at run time

```
FROM python:3.7-alpine as tester
COPY . .
RUN python -m unittest discover

```

And for reference, the Python code checking the connection

```
def setUp():
    timeout = 120.0
    port = 15903
    host = 'localhost'
    start_time = time.perf_counter()
    while True:
        try:
            with socket.create_connection((host, port), timeout=timeout):
                break
        except OSError as ex:
            time.sleep(1.0)
            if time.perf_counter() - start_time >= timeout:
                raise TimeoutError('Waited too long for the port {} on host {} to start accepting connections.'.format(port, host)) from ex

```

# Answer
> 4 votes
From your container's perspective `localhost` is the container itself. If you want to communicate with the host machine, you will need the IP address of the host itself. Depending on your Operating System and Docker configuration, this IP address varies.

**edit** `host.docker.internal` is recommended from 18.03 onward.

Here is an example of communicating with the Docker host from a container on Mac:

```
# Dockerfile

FROM python:3.7-alpine

RUN ping host.docker.internal -c 4

```

```
sending build context to Docker daemon  562.7kB
Step 1/2 : FROM python:3.7-alpine
 ---> a93594ce93e7
Step 2/2 : RUN ping host.docker.internal -c 4
 ---> Running in 54ce465f7386
PING host.docker.internal (192.168.65.2): 56 data bytes
64 bytes from 192.168.65.2: seq=0 ttl=37 time=0.735 ms
64 bytes from 192.168.65.2: seq=1 ttl=37 time=1.898 ms
64 bytes from 192.168.65.2: seq=2 ttl=37 time=0.720 ms
64 bytes from 192.168.65.2: seq=3 ttl=37 time=0.643 ms

--- host.docker.internal ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 0.643/0.999/1.898 ms
Removing intermediate container 54ce465f7386
 ---> b2f32c26a135
Successfully built b2f32c26a135

```

---
Tags: docker, dockerfile, python
---


title: What is DOCKER_CONTENT_TRUST flag?

text:
# Question
Title: What is DOCKER_CONTENT_TRUST flag?
I am trying to understand this flag DOCKER\_CONTENT\_TRUST=1 but not able to figure it out. How does it effect pushing images and pulling images, let's say from docker hub?

# Answer
DOCKER\_CONTENT\_TRUST=1 this is env variable to enable Docker content trust by default it is in disable mode.

read more about DCT in official docs DCT docs

For specific build/push/pull you can use as docker command line flag as well

--disable-content-trust=false or --disable-content-trust=true
> 2 votes

---
Tags: docker
---


# Question
Title: Docker and Kubernetes integrations compatibility matrix?
Two example integrations of Docker and Kubernetes are OpenShift and Rancher Labs AFAIK.

Sadly but in fact we have not completely escaped us from the dependency hell.

**Question: is there an established source of information which distributions here package which versions (like Ubutu/Debian version chronicles on Wikipedia)?**

*Background.*

Not-so-obvious facts for newbies (judged by my learnings so far) are (defining acceptable technical usability to a level that you do no need hours to debug even 101 tutorials):

* Kubernetes is very dynamic project and it is not so easy to deploy it as-is (but seem to get better with each version, also thanks to kubeadm I would say)
* Kubernetes does not support any Docker versions coming after the community fork to moby and Docker CE.

See also: https://stackoverflow.com/questions/44657320/which-docker-versions-will-k8s-1-7-support

# Answer
Every Kubernetes release has an **External Dependencies** section in the respective Changelog. E.g.:

CHANGELOG-1.14:

> The list of validated docker versions has changed. 1.11.1 and 1.12.1 have been removed. The current list is 1.13.1, 17.03, 17.06, 17.09, 18.06, 18.09.

CHANGELOG-1.13:

> The list of validated docker versions remain unchanged at 1.11.1, 1.12.1, 1.13.1, 17.03, 17.06, 17.09, 18.06 since Kubernetes 1.12.

CHANGELOG-1.12:

> The list of validated docker versions was updated to 1.11.1, 1.12.1, 1.13.1, 17.03, 17.06, 17.09, 18.06.

*and so on...*
> 4 votes

---
Tags: docker, kubernetes, integration
---


# Question
Title: How do I create a Docker container which just waits to be shelled into?
I'm testing a Docker container for development. And at the moment I just want to provision it with some libraries. After which I want to connect to it and test that I have all the libraries I need.

At the moment, my Dockerfile is just

```
FROM ubuntu:18.04

RUN apt update
RUN apt install -y git curl cmake libx11-dev libglu1-mesa-dev libxrandr-dev libxinerama-dev libxcursor-dev libxi-dev zlib1g-dev libasound2-dev libgtk2.0-dev libjack-jackd2-dev jq

```

But once it finishes building it terminates.

What CMD should I put at the end to keep it alive so I can shell in from another terminal?

# Answer
Disclaimer: This is very much an anti-pattern since it looks like you are trying to create something similar to a VM, or at the very least a pet that you modify in place rather than making all your changes via code and redeploying a new container for each change.

That disclaimer aside, you just need a command that will hang indefinitely. The most common one I've seen in a `tail -f /dev/null`, e.g.:

```
FROM ubuntu:18.04

RUN apt update \
 && apt install -y \
      cmake \
      curl \
      git \
      jq \
      libasound2-dev \
      libglu1-mesa-dev \
      libgtk2.0-dev \
      libjack-jackd2-dev \
      libx11-dev \
      libxcursor-dev \
      libxinerama-dev \
      libxi-dev \
      libxrandr-dev \
      zlib1g-dev

CMD [ "tail", "-f", "/dev/null" ]

```

I've also rearranged the Dockerfile to merge the update and install commands, this is a best practice for Dockerfile's since it avoids having a stale `apt update` command used from the cache when you change the `apt install` command months later. Also, putting each package on a separate line is better for version control to see what changed if you add a single new package.

When using this, you would `docker run -d --name your_container your_image` to run it in the background, and then `docker exec -it your_container /bin/bash` to open a shell.
> 1 votes

---
Tags: docker, containers
---


# Question
Title: Is FaaS required to have an official "Cloud"?
I'm in the process of refining my DevOps skills, but my "Cloud" is more like a lot of scripted virtual machines and monitoring the system. For example, if I have Oracle VM Server, and I install Ansible, Docker, Kubernetes, etc., and I can do CI, provision VMs, containers, monitoring with Elastic...does that qualify as a Cloud?

What I might provision:

Developer needs CI. The developer uses Team Foundation Server which has CI calling Jenkins or something else, deploying.

Developer's App starts getting hit, a lot, hey, we need x, I scale up with more powerful VM. I scale out if the application is engineered to allow it. Database scaling, sharding, caching more, indexes, the whole DBA part.

\*To me this is more DevOps. You understand both Dev and Ops world and know when Ops can't help anymore per se, and when an application needs engineering.

There are other things with the scripting and log monitoring.

But, I do not have true Cloud software like Azure, Oracle Cloud, and I don't have Faas.

I don't want to misrepresent myself as a Cloud Engineer or like title.

# Answer
Google’s free computing course says:

> The US National Institute of Standards and Technology created a definition of cloud, although, there is nothing US specific about it. Here it is, cloud computing is a way of using I.T. that has these five equally important traits. First, you get computing resources on-demand and self-service. All you have to do is use a simple interface and you get the processing power, storage, and network you need, with no need for human intervention. Second, you access these resources over the net from anywhere you want. Third, the provider of those resources has a big pool of them and allocates them to customers out of that pool. That allows the provider to get economies of scale by buying in bulk and pass the savings on to the customers. Customers don't have to know or care about the exact physical location of those resources. Fourth, the resources are elastic. If you need more resources you can get more, rapidly. If you need less, you can scale back. And last, the customers pay only for what they use or reserve as they go. If they stop using resources, they stop paying.

This seems to be taken from the document at https://www.nist.gov/news-events/news/2011/10/final-version-nist-cloud-computing-definition-published

Using DevOps software with virtual machines, with or without containers, can happen without using cloud. In a similar manner, you can open a corporate account on a public cloud and not do DevOps at all. This happens more than we might expect when big companies are migrating legacy systems into the cloud where they simply have sysadmins or other Ops folks use the web interfaces to the cloud IaaS to manually commission VMs that they then manually configure. I have met teams that are convinced that they are doing DevOps when they are simply doing Ops with a few specialist Cloud Engineers who handle setting Virtual Private Cloud (VPS) meaning private connectivity into the cloud (e.g, VPN or a dedicated connection) and IAM (Identity and Access Management).

IMHO cloud and devops are most definitely not the same thing but are at their best when combined.
> 1 votes

---
Tags: terminology, cloud
---


# Question
Title: Ansible: Copying different files to different guests for use with the same role
I have a role, let's call it apache2. The role installs apache2 and all of its dependencies.

Now I have two machines, let's call them M1 and M2. Both M1 and M2 need to have apache2 installed. However, each of them needs their own set of files installed into the web root. So M1.yml looks like:

```
---
- hosts: M1
    - role: apache2

```

M2.yml looks like:

```
---
- hosts: M2
    - role: apache2

```

Obviously I can't just put the files into the role's `files` directory, as they are different for each. Options of which I am aware:

1. Create a `post_tasks` section for each playbook that copies over the necessary files and configures the server.
2. Create a `copy_files` role that takes the files as a variable

The reason I don't like #1 is because I might have a bunch of similar roles (apache2, nginx, ftp, etc.) each of which requires installation and files to be copied over. I'd like to keep each "together", meaning that the "install apache2" and "copy apache2 files to guest" tasks are both specified near each other in the file.

The reason I don't like #2 is because it seems heavyweight to basically just wrap a task as a role.

Is there another option I'm not thinking of? And of the available options, which is considered best practice?

# Answer
> 1 votes
Another approach that I found uses the `include_role` directive. So instead of the playbook being:

```
---
- hosts: M2
  - roles:
    - role: apache2

```

It becomes:

```
---
- hosts: M2
  - tasks:
    - include_role: apache2

```

Then I can add tasks directly after that apache2 task that configure apache2:

```
---
- hosts: M2
  - tasks:
    - include_role: apache2

    - copy:
        # etc

```

---
Tags: ansible
---


# Question
Title: Will google cloud take money from me in this situation?
I registered google cloud account and want to use Text-To-Speech API on my own server with github.com/googleapis/google-cloud-php library.

On API page in block "Pricing" written that using less than 1 millon chracters/month is free. But lower is a note "Additional charges may also apply from infrastructure you use to call the API"

Question: This note applies to users google cloud VPS, right? If I will use API how I described (send requests from my own server) first million of characters will be absolutely free? Google wont take a cent?

# Answer
Yes, they mean the Compute Engine. So you pay Google only for API, not for server.
> 1 votes

---
Tags: google-cloud-platform
---


title: TFS Back-up Notifications

text:
# Question
Title: TFS Back-up Notifications
Our company uses TFS 2018. We have an annoying issue when there is TFS back-up failed,it sends notifications to hell lot of people which are invalid or unncecessary users. This list is in the domain Admins. I can't delete them manually.

Is there a script to delete the users or modify the sending list?

# Answer
> 1 votes
I believe your primary issue is that you have this group of users in the `[Team Foundation]\Team Foundation Administrators` group or possibly many users added with privileges to the Administration Console Users, which is unusual. To adjust who is part of the Team Foundation Administrators group:

1. RDP to the server that hosts the application tier for your TFS installation
2. Open the TFS Administration Console
3. Highlight the Application Tier
4. Click the Group Membership link
5. Double-click `[Team Foundation]\Team Foundation Administrators`
6. Adjust membership of the group

To learn more about configuring administrators from the TFS Administration console, see Microsoft's documentation on the topic.

---
Tags: tfs
---


# Question
Title: How trigger a build by a commit in a branch prefix?
On the Azure DevOps, in we `git` we have a schema of branches where we create a branch per `sprint`, for sample:

* sprint01
* sprint02
* sprint03 ...

We keep the 3 last `sprint` branches, and in this case when I create the `sprint4` branch, we will drop the `sprint01`. Each sprint We publish a version of our product every week. On the `build` pipeline definition on our project, we have defined the trigger to run over a push on the `sprint*` branch because we don't want to change the definition on each new version.

It works but it runs a build for each `sprint` branch and a release for each one. I would like to have just for that branch (with prefix `sprint`) that was pushed on the git. Is there any way to configure it?

My current configuration:

# Answer
It's recommended that you use a convention of prefixing your grouped references (like branches) with "folder" prefixes.

For example, when you create sprint branches, create them with the names:

```
sprints/sprint01
sprints/sprint02
sprints/sprint03

```

Then, configure your build trigger `Include` filters with this prefix:

`sprints/*`

... and then an `Exclude` filter for `master`, since by default it is also included.

Once configured, this should build only the sprint branch (topic branch) that was committed to.

Reference: https://docs.microsoft.com/en-us/azure/devops/pipelines/build/ci-build-git?view=azure-devops&tabs=designer
> 0 votes

---
Tags: azure-devops, build-pipeline, release
---


# Question
Title: Assign bug from different repo when creating a pull request
Say I have 2 repos...

And I have a work item (or bug, or whatever) in one repo, and my code in another repo (yes, I know it's silly, that's how my group rolls for historical reasons).

Is there a way to assign a bug from one to the other? So that when I create a PR in Repo1, I can link to a bug in Repo2?

# Answer
> 1 votes
There are actually many ways to link work items to other objects in Azure DevOps. Microsoft's documentation is pretty extensive on this topic. However, for the particular task at hand, linking code commits (I'll assume Git is in use for this example), can be done in several different ways.

1. In your commit message - By including the work item ID in the commit message prefixed with the hash mark `#` (i.e. `#12345`), when the commit is pushed to the server, the work item and commit are linked automatically. Then, when this commit is part of a pull request that is created, the linkage is made visible as part of the pull request.
2. Within the pull request during creation, or afterward when edited by the creator, you can add work items directly to the pull request by searching either by ID or by keywords.
3. Several other methods can be discovered through editing a work item or right in Visual Studio.

# Answer
> 0 votes
Work Items created in Projects, not in Repos.

And yes, you can link to PR also work items from another projects.

---
Tags: azure-devops
---


# Question
Title: In Azure Devops and TFS, is it possible to create a build artifact and release definition in a different collection than the original code repository?
I have code repo held in Collection 1 and, using this code repo, I am trying to create a Build artifact and Release definition in a different Collection 2.

Is this even possible?

# Answer
Since on-premise TFS and the newly renamed Azure DevOps Server 2019 still have the concept of collections, which is the highest level of separation of data you can have (separate physical databases), you'd need to treat a code repo in Collection 1 as an external repository rather than try to discover it as a known internal repository.

You can do this in the Build pipeline of a build in Collection 2 by selecting External Git (if your repo is Git, TFVC is not supported), then clicking `Add Connection` and entering the Git repository URL and login credentials like in the screenshot below. Once connected, your pipeline, if it produces artifacts, will store them in Collection 2, which can be used for a Release pipeline.
> 1 votes

# Answer
In Azure DevOps there is no Collection - there is a project and there are organizations which hold zero or more projects. There is also a distinction between the code repositories and the builds and between the builds and the release mechanism.

Within a build pipeline it is entirely possible to reference a different projects repository to pull into the build and that repository could be in GitHub, Gitlab, or another VSTS organization, and within a release it is entirely possible to include build artifacts from multiple build pipelines.

TFS on-premises depends highly on how up-to-date it has been kept. The most recent versions are feature equal or close to it with Azure DevOps and the above should be true. Milage may vary in older versions.

It isn't clear here in your question though what you are trying to achieve or what you've tried so far.
> 0 votes

---
Tags: continuous-integration, azure-devops, cicd, azure-pipelines, tfs
---


# Question
Title: How should I deploy my projects using Docker, Git and VPS?
I have a VPS and am able to connect trough ssh using Terminal. Every project I have are dockerized and have github repositories. Since I am new to deployments so far my deployment process is manual:

```
$ ssh root@xxx.xxx.xx.xxx
$ cd /root/www/project_dir
$ git pull
$ docker-compose up --build -d --force-recreate

```

I would like to know how should I correctly deploy dockerized github applications on VPS? Is there a better practice or script, maybe a tool?

# Answer
Here is a Webhook rule to catch Github release events and run a custom bash script. It is run by adnanh/webhook which is a small go app that can run your deployment script. You then configure GitHub to send release events as per this screenshot:

You can then use the github hub command line tool to create releases with a name that matches the regex in the hook rule e.g., `v1.2.3`. That will fire the release webhook. That can be caught by your deploy script running in a container on your server. Your deploy script can take the release number parsed from the release event payload. It can then git pull the git tag `v1.2.3` launch your code. Note that you don’t have to use the hub cli a release event will also be fired if you push a tag to your repo using regular git. We find it helpful to use the hub tool that creates the tag and a github release. You can attach automatically generated release notes using github\_changelog\_generator see this example script.

Using similar webhooks is how OCD can drive kubernetes from configuration in GitHub. We have OCD deploy all our apps on AWS based on configuration and tags in GitHub. You can use the same techniques to drive your small deployment scripts quite easily. All you need to do is create docker container with adnanh/webhook, the json config and your deploy script. Here is a where we install webhook into a container and here is where we copy our rules. As it is a go app it doesn’t need a lot of memory to be able to fully automate your software based on git webhook events.
> 2 votes

---
Tags: docker, docker-compose, deployment, github
---


# Question
Title: Load Balancing of Jenkins Nodes without Load Balancing Plugin?
we currently want to introduce a loadbalancing on the Jenkins server and our nodes. I have already created several nodes.

Currently it is also so that if a node is blocked accordingly, that the next free option is chosen. But only if a node is issued. Unfortunately, this is only part of what the whole task looks like.

Task to be solved:

* If a node is under load, the next free option should be selected automatically.
* If a node is issued, then the next inactive node should be taken.

Problem:

For security reasons, you do not want us to use the loadbalancing plugin. So there has to be a solution that we have through a script or some other solution

# Answer
> 1 votes
This is similar to: https://stackoverflow.com/questions/55023570/how-to-run-a-job-on-first-available-node/55567958

Have you considered Least Load plugin?

https://plugins.jenkins.io/leastload

By default Jenkins tries to allocate a jobs to the last node is was executed on. This can result in nodes being left idle while other nodes are overloaded. This plugin overrides the default behavior and assigns jobs to nodes with the least load.

Don't imagine there's a security issue there.

---
Tags: jenkins, jenkins-pipeline, jenkins-plugins, load-balance, jenkins2
---


# Question
Title: How to persuade developers to start using feature flag toggles?
Assuming that feature flag toggles are a good idea, and should be implemented into code that developers write. For example Etsy swear by them as a major part of their culture.

**What is a good way to persuade (and enforce) developers to start using feature flag toggles?**

More information about feature flag toggles is explained in Q: How to use feature flag toggles, Q: What are feature flag toggles and very extensively in Pete Hodgson's article on the subject on Martin Fowler's blog.

# Answer
Feature toggles are a common practice in high-velocity development because they de-couple *development* from *release.* Dev teams can "soft-release" a new feature to production, in a disabled state. This allows the feature to be released any time. If the feature is dependent on other work or preparation, it doesn't have to wait for a major release to go to production.

As far as "convincing" developers to use them, that's an exercise in making the case for the freedom it offers. My experience is that it's not a tough sell to developers. It's management that tends to be reluctant to try new things. Try this:

* Find a feature toggle framework. Management/the business may be more amenable to trying something supported by a common system
* Start small. Introduce the toggle system on a trial basis for a feature that will demonstrate its utility.
* Demonstrate the toggle's ability to do A/B testing. Enable the toggle for a subset of your web farm, then gather metrics on behavior. Tiny differences in page layout have been demonstrated to have huge effects on revenue for retail applications (e.g. Ebay, Amazon)
> 18 votes

# Answer
In an ideal world I think you roll out a new build and surprise! NOTHING changes. This is because all of your new features are behind switches that go out with the switch off.

Post-deployment you verify that your rolled-out service still works, the phones aren't ringing any more (unless ringing phones is your purpose, that is), etc. Once you are back to a known stable operation you begin enabling and verifying your newly deployed features.

Now for your answer: How would you like to work on a team where being on call is practically a no-brainer and our users love us because our sites and services are rock solid stable?

That's the team I want to work on.

You can stop reading here if you want.

Putting everything behind a feature switch seems like it can lead to spaghetti code everywhere. If you use IoC and are able to select between vNow/vNext/vPrevious then it comes down to maintaining your configuration. Yes more check ins, yes more classes (componentV1, componentV2, componentV3, etc.) but you actually have a more stable system? How? vNext is wonky? Switch back to vNow with your control tower. Been a week and vNow has a subtle bug? Same thing - go back to vPrevious just as easily.

No hassle, no worries, no lost sleep, no stress.

This isn't a pipe dream. I used to work there. Wish I could sell this to my current team.
> 8 votes

# Answer
A successful high-velocity development environment typically relies on a pretty strict automated system involving quality verifications with detection and rejection of faulty changes causing regressions.

Feature toggles offer the ability to commit even work-in-progress, untested changes without getting rejected for causing regressions in the integration branch. Which constitues a very good **incentive** to introduce the feature toggles very early in the life of the feature.

One of the disadvantages of deviating from true CI and moving feature development on feature branches is the lack of such incentive. Adding the feature toggle later on, when merging the feature branch into the integration branch is typically more difficult, like any late integration.
> 4 votes

# Answer
Developers (and usually development managers) usually look for two outcomes associated with framework: ease of management, and speed of deployment. You want to ship code faster, and easier.

Provide evidence that the approach works; try building a small POC using feature flags versus the old way. Case studies matter less to tactical people (developers\engineers) than strategic people (middle management\product designers).
> 2 votes

# Answer
The rationale to have feature toggles is not something for developers to decide. This is something for product owners to care. Developers enable this change in the most sustainable and safe manner. I critique this very question.
> 0 votes

---
Tags: culture, feature-flags, transformation
---


# Question
Title: What are the two project layouts offered by Rancher on installation?
During login after initial setup, Rancher CLI asks the following:

```
Do you want to continue connecting (yes/no)? yes
NUMBER    CLUSTER NAME                          PROJECT ID        PROJECT NAME   PROJECT DESCRIPTION
1         xxx                                   c-5tcls:p-b4r55   System         System project created for the cluster
2         xxx                                   c-5tcls:p-tmx85   Default        Default project created for the cluster

```

What is the difference and is there a way to make this choice non-interactively in a headless environment?

*Note*. For some reason, 1 and 2 get randomly swapped from time to time we run this.

# Answer
Project is a Rancher specific concept. A Project can have multiple Kubernetes namespaces.

The System Project has all the Kubernetes system components like kube-dns, ingress etc. Think of it like System Services running on a machine/laptop.

Default project is where you can launch your workloads to start with. Similar to a scratchpad.

You can create new projects too. For example, Web App, Database, etc. Or it could be based on different users using the cluster. Project-For-Bob, Project-For-John.
> 1 votes

---
Tags: rancher
---


title: ansible does not play roles and tasks

text:
# Question
Title: ansible does not play roles and tasks
I'm learning ansible and I'm having a problem, when I run my playbook, it only runs the gather\_fact task and does not play my roles. On the other hand when I run just a module with the ansible command -m or playing the playbook locally, there my roles are played. Do you have an idea ? I posted the question on the google group, but for the moment no answers. thank you in advance

# Answer
If instead of using the command line:
`ansible-playbook -u root --private -key ~ / .ssh / id_rsa centos7.yml -i xxx.xxx.xxx.xxx`,

I use:

`ansible-playbook -u root centos7.yml -i xxx.xxx.xxx.xxx`,

It works.
> 1 votes

---
Tags: ansible
---


# Question
Title: Best Practice Authenticating Jenkins to Bitbucket to Access Team Repository
What's the "best-practice" approach to authenticate a CI tool like Jenkins or Bamboo to BitBucket? I started with App Passwords because it's cheap and easy, but I hate that it's tied to my account because if I leave the company, the CI/CD integration with Bitbucket gets killed.

Is there anything that isn't scoped to a specific user?

# Answer
> 1 votes
In an enterprise environment (but even in a personal one, I assume tech savvy enough users to do that) it's generally a good idea to have dedicated accounts created specifically for automated tools execution.

I'd even recommend one account per tool because different tools may have different permission requirements and it's generally a good idea to limit the permission scope to the minimum required, for security reasons. In large enterprises this is often mandatory, prescribed by policies.

In some cases it might be necessary to have different tools executed under the same account, in which case such combo accounts would be required. Again, one per specific use case.

If you want a parallel - take a look at a Linux system, for example - different system accounts used for different purposes. Or service accounts, at Google for example.

---
Tags: jenkins, bitbucket
---


# Question
Title: unable to save register value even if the play is skipped!
I have situation where i fetch some ip address on the fly with this play, when env variable is passed as prod, i have to fetch ip in a diff way then when i am in QA env.

If env is prod the 1st play would get ip address and store in register **"kafkaip"** and i have put a when condition on the 2nd play it gets skipped which is fine as the env is diff.

But although its skip the 2nd play, it wipes out the register value and my set\_fact fails,, any way to overcome below scenario.

```
    - name: Get the Kafka InstancesPrivateIps
      local_action: shell aws cloudformation describe-stacks --stack-name "{{ stack_kafka }}" --query "Stacks[*].Outputs[?OutputKey=='KafkaLink'].OutputValue"
      register: kafkaip
      when:
      - stack_kafka is defined

    - name: Get the Kafka InstancesPrivateIps New Automation.
      shell: |
        aws  ec2 describe-instances --filter "Name=tag:StackName,Values=KafkaCluster" --query 'Reservations[].Instances[].PrivateIpAddress' --output text >  /tmp/kafka_ip.txt
      register: kafkaip
      when:
      - stack_kafka is defined
      - ( deployment_env == "QA" )
      delegate_to: localhost

    - name: setting kafka_host.
      set_fact:
        kafka_host: "{{ kafkaip.stdout }}"
      when: stack_kafka is defined

```

# Answer
> 1 votes
`register` will always populate your registered var even when the task is skipped, e.g. you will end up with something like the following:

```
"kafkaip": {
    "changed": false,
    "skip_reason": "Conditional result was False",
    "skipped": true
}

```

Since only the commands differ depending on your env, I would store the command in a variable and then execute it, something like the following.

```
- name: "Set command for env <> QA"
  set_fact:
    get_ip_command: >-
      aws cloudformation describe-stacks
      --stack-name "{{ stack_kafka }}"
      --query "Stacks[*].Outputs[?OutputKey=='KafkaLink'].OutputValue"
  when:
    - stack_kafka is defined
    - ( deployment_env != "QA" )

- name: "Set command for env = QA"
  set_fact:
    get_ip_command: >-
      aws ec2 describe-instances
      --filter "Name=tag:StackName,Values=KafkaCluster"
      --query 'Reservations[].Instances[].PrivateIpAddress'
      --output text
  when:
    - stack_kafka is defined
    - ( deployment_env == "QA" )

- name: Get the Kafka InstancesPrivateIps
  shell: "{{ get_ip_command }}"
  register: kafkaip
  when: stack_kafka is defined

- name: Register kafka_host
  set_fact:
    kafka_host: "{{ kafkaip.stdout }}"
  when: stack_kafka is defined

```
* **Note1**: there is an output redirection in your QA command which will prevent capturing the output as you wish, I left it out in my example.
* **Note2**: I used `set_fact` to create the command variable but you can put that in your vars or your inventory to have a leaner playbook.

---
Tags: ansible
---


# Question
Title: Is there an alternative to setting up the whole project workspace in a Jenkinsfile
I have a ROS project for which I'm looking to use Jenkins to set up a CI system.

The project has a large number of dependencies which take a long time to build (they can't be installed on the build executor machines), so we typically build them once, then let Catkin (ROS's build system, based on CMake) use the cached artifacts on subsequent builds.

The Jenkins setup we currently have does this:

* Clone repository at commit
* Create an empty Catkin workspace
* Clone and setup all our dependencies
* Build everything together
* Run tests

Due to building all our dependencies, this takes several hours.

What I'd like to do is this:

* Clone repository at commit
* Move it into an already built Catkin workspace containing all dependencies
* Build just our project
* Run tests

I was thinking of setting up a Docker container with all the dependencies ready to go, and then just running the build in that. Is that a typically recommended approach?

# Answer
Using Docker with Jenkins in a situation like this is a great way to

1. Keep Jenkins worker slave nodes clean (so you don't have to build or install anything on them)
2. Have all dependencies and environment in an easily manageable artifact (in this case, a docker image is the snapshot of the exact OS, filesystem, env and dependencies)
3. Vendor / keep copies of dependencies at a certain point in time (Example: You don't want to update at the same rate as the dependency)

If you have some kind of artifact management system (Example: Nexus) you can store your docker images in the artifact management system, and then pull them into Jenkins, and create a running container that has your repository in it. From there you can build project run tests inside the docker image. Just be sure you remember to move/copy all test results and built artifacts out of the docker container if you need them or use them later.

If you need more details or specifics please edit your question and I will edit my answer accordingly.
> 1 votes

---
Tags: docker, jenkins
---


# Question
Title: How to deploy my Express/React app to server with docker-compose and Dockerfiles
#

Hi, I'm trying to deploy my app to production with Docker, but I struggle a lot. Here is the structure from the root folder :

* a **server** folder : runs with Express on port 4000
* a **web** folder : a React app running on 3000 (locally), that needs to run on 80 on my server
* a **docker-compose.prod.yml** file in the root folder

What I'm currently trying :

* the **docker-compose.prod.yml** is running a Postgres service (port 5432), as well as web and server's respective Dockerfile
* a Dockerfile in both web and server folders
* I `yarn build` both server and web folders locally and send everything on my server
* on my server, I run `docker build -f web/Dockerfile.prod -t app_web ./web` and `docker build -f server/Dockerfile.prod -t app_server ./server` to build both images
* finally, I run `docker-compose -f docker-compose.prod.yml up`

**My problem :**

The `server` container throws `Error: Cannot find module '...'` although I run `yarn install --production` in its Dockerfile (it may be related to volumes?)

Also, when I run `docker run -it -p 80:80 app_web` and `docker-compose up`, the web container acts differently when visiting my server IP address : in the first case, the app displays well, in the second case, a page is listing my web folder content (all files and folders)

I'd really appreciate help as I'm a bit new to Docker in production (should I use docker machine? are my folders structure / docker-compose / Dockerfiles wrong?)

**docker-compose.prod.yml** :

```
version: "3"
services:
  db:
    image: postgres:latest
    restart: on-failure
    environment:
      POSTGRES_DB: ${DB_NAME}
      POSTGRES_USER: ${DB_NAME}
      POSTGRES_PASSWORD: ${DB_PASSWORD}
    ports:
      - 5432:5432
    volumes:
      - ./.db/data:/var/lib/postgresql/data

  server:
    container_name: app_server
    build:
      context: ./server
      dockerfile: Dockerfile.prod
    volumes:
      - ./server:/app/server
    ports:
      - 4000:4000
    depends_on:
      - db

  web:
    container_name: app_web
    build:
      context: ./web
      dockerfile: Dockerfile.prod
    volumes:
      - ./web:/app/web
    ports:
      - 80:80
    depends_on:
      - db

```

**server/Dockerfile.prod** :

```
FROM mhart/alpine-node:11 AS builder
WORKDIR /app/server
COPY package.json ./
RUN yarn install --production
COPY . .

FROM mhart/alpine-node
WORKDIR /app/server
COPY --from=builder /app/server .
CMD ["node", "/app/server/dist"]

```

**web/Dockerfile.prod** :

```
FROM mhart/alpine-node:11 AS builder
WORKDIR /app/web
COPY package.json .
RUN yarn install --production
COPY . .

FROM mhart/alpine-node
RUN yarn global add serve
WORKDIR /app/web
COPY --from=builder /app/web/build .
CMD ["serve", "-p", "80", "-s", "."]

```

# Answer
> 1 votes
I found the solution, eventually it was just a volume issue, I had to add a specific line for modules :

```
volumes:
      - ./server:/app/server
      - /app/server/node_modules

```

After that I no longer got the "Can't find module" error
Still not sure what the impact of this is though

---
Tags: docker, docker-compose
---


title: Why label k8s deployment?

text:
# Question
Title: Why label k8s deployment?
In the below deployment yml, the deployment itself has a label "app: hello". What would happen if I delete that line (line 6)?

I understand the functional need for *pods* to be labeled, because then the deployment can reference those pods by labels. In the below example, that means that I understand why lines 11-12 and 16-17 exist. But that doesn't answer why the *deployment* should be labeled.

```
apiVersion: apps/v1
kind: Deployment
metadata:
  name: helloweb
  labels:
    app: hello
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello
      tier: web
  template:
    metadata:
      labels:
        app: hello
        tier: web
    spec:
      containers:
        - name: hello-app
          image: registry.gitlab.com/itential/cloud-team/applications/hello-app
          ports:
            - containerPort: 8080
      imagePullSecrets:
        - name: gitlab-reg

```

# Answer
Labels lets you as user query for particular objects; from the docs:

> Labels enable users to map their own organizational structures onto system objects in a loosely coupled fashion, without requiring clients to store these mappings.

For example if you use some kind of deployment tool like Spinnaker; the tool labels every single object that it touched so it knows about it - can manage it.

When you query for a label you can use equality-based operators (`=`, `==` and `!=`). K8s doc should be good starting point, alternatively check out this article on labels.
> 1 votes

---
Tags: kubernetes, deployment
---


# Question
Title: Unable to ssh in a multi-machine vagrant environment
I created a three node multi-machine vagrant environment and am having issues ssh'ing from one vagrant vm to another.

Here is the Vagrantfile:

```
Vagrant.configure("2") do |config|
  config.vm.box = "centos/7"

  config.vm.define "master" do |master|
    master.vm.hostname = "master.local"
    master.vm.network "private_network", type: "dhcp"
  end

  config.vm.define "node1" do |node1|
     node1.vm.hostname = "node1.local"
     node1.vm.network "private_network", type: "dhcp"
  end

  config.vm.define "node2" do |node2|
    node2.vm.hostname = "node2.local"
    node2.vm.network "private_network", type: "dhcp"
  end
end

```

The hosts file (same on each node):

```
$ cat /etc/hosts
172.28.128.3    master.local    master
172.28.128.4    node1.local     node1
172.28.128.5    node2.local     node2

```

I can ping back and forth all day from any machine to the other but I cannot ssh from one vagrant vm to the other. The typical error message is (from node1 to master):

```
[vagrant@node1.local] $ ssh vagrant@172.28.128.3
Permission denied (publickey,gssapi-keyex,gssapi-with-mic)

```

SSH is running and the port is open.

The firewall is not running.

I am sure this has to do with ssh keys. I readily admit I am not an expert.
What am I doing wrong here folks?

# Answer
Following Vagrant file address this problem.

You can get all supporting key files along with this vagrant file at https://github.com/malyabee/IaaC/tree/master/ansible\_lab

```
$commonscript = <<-SCRIPT
sudo yum update -y
sudo yum install python2 epel-release -y
sudo yum install -y ansible
sudo echo "192.168.22.10    ansiblecontroller.example.com ansiblecontroller" >> /etc/hosts
sudo echo "192.168.22.11   node01.example.com   node01" >> /etc/hosts
sudo echo "192.168.22.12   node02.example.com      node02" >> /etc/hosts
SCRIPT

$nodescript = <<-SCRIPT
cat /vagrant/ansible_lab.pub >> /home/vagrant/.ssh/authorized_keys
SCRIPT

$ansiblescript = <<-SCRIPT
sudo yum install ansible -y
sudo cp -r /vagrant/ansible_lab /home/vagrant/.ssh/id_rsa
sudo chmod 400  /home/vagrant/.ssh/id_rsa
sudo chown vagrant:vagrant /home/vagrant/.ssh/id_rsa
SCRIPT

Vagrant.configure("2") do |config|
  config.vm.provision "shell", inline: "echo Hello"

  config.vm.define "ansiblecontroller" do |ansiblecontroller|
    ansiblecontroller.vm.box = "centos/7"
    ansiblecontroller.vm.provider "virtualbox" do |v|
          v.memory = 512
          v.cpus = 1
       end
    ansiblecontroller.vm.network "private_network", ip: "192.168.22.10", virtualbox__intnet: "mynetwork01"
    ansiblecontroller.vm.hostname = "ansiblecontroller.example.com"
    # Installing required packages for ansible controller node
    ansiblecontroller.vm.provision "shell", inline: $commonscript
    ansiblecontroller.vm.provision "shell", inline: $ansiblescript
  end

  config.vm.define "node01" do |node01|
    node01.vm.box = "centos/7"
    node01.vm.provider "virtualbox" do |v|
          v.memory = 512
          v.cpus = 1
       end
    node01.vm.network "private_network", ip: "192.168.22.11", virtualbox__intnet: "mynetwork01"
    node01.vm.hostname = "node01.example.com"
    # Installing required packages for  node01
    node01.vm.provision "shell", inline: $commonscript
    node01.vm.provision "shell", inline: $nodescript
  end
  config.vm.define "node02" do |node02|
    node02.vm.box = "centos/7"
    node02.vm.provider "virtualbox" do |v|
          v.memory = 512
          v.cpus = 1
       end
    node02.vm.network "private_network", ip: "192.168.22.12", virtualbox__intnet: "mynetwork01"
    node02.vm.hostname = "node02.example.com"
    # Installing required packages for  node01
    node02.vm.provision "shell", inline: $commonscript
    node02.vm.provision "shell", inline: $nodescript
  end
end

```
> 3 votes

# Answer
According to the docs one should use:

```
vagrant ssh [name|id]

```

If there is a single node then use `vagrant ssh` and in case of multi-node define the name or id of the VM, e.g. `vagrant ssh box1`

If one would like to ssh between boxes then one could create an ssh key and provision the private key to each box and add the public key to authorized\_keys file.

https://www.vagrantup.com/docs/provisioning/file.html

> ```
> Vagrant.configure("2") do |config|
>   # ... other configuration
>
>   config.vm.provision "file", source: "~/.gitconfig", destination: ".gitconfig"
> end
>
> ```
> 2 votes

# Answer
Try this link. You need to either execute one of the following:

* `ssh -i <pathto/private_key> <vagrant>@<ip>`
* `ssh -o PreferredAuthentications=password user@server-ip` (if you haven’t disabled password based authentication)
> 0 votes

---
Tags: vagrant
---


title: Kubernetes multi cloud

text:
# Question
Title: Kubernetes multi cloud
I'm exploring doing kubernetes on a multicloud setup the configuration I want is a master connected to nodes in multiple cloud platforms. In my limited research I haven't found many tools to make this happen, from what I can tell rancher seems like the best option, but I don't think this fits within the traditional usecase. My question is does this sound like something rancher was made for? Honestly it seems more of a kubeadm usecase but the caveat is I want to use multiple managed solutions where multiple masters all share a common node pool.

# Answer
> 1 votes
(Rancher employee) Kubernetes itself is really not meant for that use-case. A cluster is generally a set of machines in the same provider with close proximity (latency) to each other.

* Etcd (and therefore API/CLI/UI/scheduling/everything) performance depends heavily on the (worst-case) latency between all members.
* Only one cloud-provider integration (storage providers, L4 load balancers) can be configured and most assume nodes are all in the same "region" or similar concept.
* Commonly used network plugins assume adjacency and/or provide no encryption or authentication suitable for communicating across untrusted networks/the internet.
* Communication between pods/services within the cluster has no consideration for where the "nearest" place to reach a pod for that service is, so latency from a pod to another service can be large (however far apart the two worst nodes are) and unpredictable (sometimes close, sometimes far). (There is an option for nodePorts to *always* go to a local pod, but if there is no local pod it will just get dropped).

You *can* make a "custom" cluster and add nodes from wherever and do what you're asking in Rancher, but you're not going to have a good time (no matter what product is building the cluster).

---
Tags: kubernetes, rancher
---


# Question
Title: What is the cheapest way to get CI for a Qt project?
To build up my portfolio I'm writing an application to showcase all the best practices I'm aware of, including unit tests and continuous integration.

The project was originally started in C++, and I took advantage of buddy.works to compile it and run my tests there. Managing `CMakeLists` was a pain but I endured for the greater good.

After a while I noticed that Qt would have been a better fit for my purposes. Being this a non-commercial open source project I'm allowed to use Qt Creator for free. However, buddy.works (and almost all other free CI tools) only allow bare `GCC` build environments. In my case I'd need a fully configured Qt environment, as when I install the IDE, plus Android SDK and NDK.

I see buddy.works allows to run commands in a "private container", but I'm not sure if building a container myself (something I'd start looking into if I knew this would work).

I guess the safest choice would be to rent an Amazon EC2 instance, install on that everything I need (Qt, SDKs, Jenkins) and configure it to communicate with my GitHub or Bitbucket, but it would cost me (and I can't say how much exactly). I expect the CI process to only run like thrice per week, with an average build time that at most would take half an hour. I also liked the idea of the green build badge in the readme file, something buddy.works offered and with an EC2 instance shutting down could be trickier to achieve (if I was to purchase an instance that boots every day, runs CI and shuts down). However, a free alternative would be much appreciated.

Any hint about this? Thank you very much.

# Answer
> 2 votes
One of the major advantage of the container technology is standardisation: easily replicating the same environment wherever a specific container is used, be it, for example, on your own machine during development or on the CI machine during the execution of the CI/CD pipeline.

Building and using custom containers is a pretty standard way of including OS-level dependencies (like libraries and toolsets, for example) which aren't included in a service provider's default container images. Depending on the provider the choices for the base image you'd use *might* be limited.

If you have a CI/CD provider which accepts custom-built containers (not all of them do, at least not yet) and you can meet their custom container restrictions (if any) then building the custom container for your case is IMHO a valuable investment. Not only for the CI execution, but even for developing of your app and ultimately for showcasing the container technology as a development technology itself :)

# Answer
> 1 votes
You could use Jenkins along with Gitlab for your CI. Both are free. As far as the Qt project goes, you could use the `sh` or `bat` steps to run qmake, or cmake for that matter, from within the build job.

There are also a lot of plugins to extend Jenkins and integrate it with Gitlab. This includes plugins that allow you to trigger builds when pushing to a particular branch or even generating the build badge you'd like to have. See: Embeddable Build Status Plugin

---
Tags: continuous-integration, containers, amazon-ec2, software-engineering
---


# Question
Title: Securing communication between cloud VMs without private networks. Any strategy besides building own VPN? Spiped?
I am working on an application that requires a number of different services (web application + database + worker queues, XMPP server for messaging, redis, central authentication via LDAP, etc). For my first attempt at designing the thing, I am estimating that I will need ~15 instances of varying sizes: more RAM/SSD for the database, less so for web server and LDAP, more cores for the worker queues, and so on.

I don't want to use the big cloud Providers (AWS, Azure, GCP) for matters of cost. This is something I am still bootstrapping and GCP pricing calculator put this at ~$5800/month. A similar (in terms of CPU/RAM/SSD) spec in Hetzner cloud would be ~$430/month.

Of course, the difference in price also comes with some difference in features. Hetzner Cloud VM instances do not have any sort of private networking. So I am wondering how I can ensure that all those machines can be effectively protected from the public internet. Some of these services (web server, xmpp server) should also have a public IP, but ideally all other services should have all ports closed (including SSH)

I know that one of the answers is "build your own OpenVPN AS". However, (a) I never done it before and (b) I am under the impression this would mean yet more services to add to this infrastructure.

One thing I *did* do before was to connect machines via spiped. This does not give you a private network but at least ensures that only those with the key can connect to your public-facing machines.

So my question is: am I missing some alternative to these two above?

Also: docker. I have limited experience with Docker and Kubernetes, but if I went this route it seems I would end up going to the arms of the big providers, not to mention it would still be yet-another thing to learn and take time from my application development. Is Docker Swarm something that could solve this requirement?

# Answer
> 1 votes
I am going to suggest an approach that I would not necessarily recommend, because I do not consider it a manageable, scalable solution:

SSH port forwarding.

SSH is a Swiss Army knife in terms of its features. Virtually any port can be securely forwarded to any other port on a remote machine, and indeed through a remote machine to a target machine beyond. Documentation abounds, but I really like the drawings in this answer. I use ssh across bastion hosts regularly in a manner that mirrors the diagrams at the link.

As I said, I don't think this is a good solution, but it might serve to give you some ideas and it could be one component of a larger solution.

Good luck!

# Answer
> 0 votes
Another idea: ZeroTier.

I have not used it, but a fellow DevOps nerd recommends it.

---
Tags: security, cloud, networking
---


# Question
Title: Get Ansible to print lines for include_role(s)
I figured this would be trivial, but can't find the answer anywhere.

Let's say I have a playbook.yml that looks like:

```
- hosts: all

  tasks:
  - name: I'm a block!
    block:
    - name: I'm a role!
      include_role: name=some_role

```

And then I have a main.yml in roles/some\_role/tasks that looks like:

```
- name: I'm a task!
  git: repo=https://doesnexist.com/nope.git dest={{ install_path }}

```

What I would expect / like to be printed is:

```
BLOCK [I'm a block!] *********
ROLE [I'm a role!] *********
TASK [some_role : I'm a task!] *********
ok: [MACHINE]

```

Or something like that. Instead, what gets printed is:

```
TASK [some_role : I'm a task!] *********
ok: [MACHINE]

```

So why did I bother putting those names on the block and include\_role? Is there any way to get what I want, or at least something close to it? The debug module doesn't do what I want, since it is very poorly formatted.

# Answer
I was able to create a custom callback plugin. I started with the default callback code and modified it:

```
    def v2_playbook_on_task_start(self, task, is_conditional):

        if(task._role is not None and task._role != self._current_role):
            self._current_role = task._role
            self._display.banner("ROLE [" + self._current_role.get_name() + "]", color=C.COLOR_DEBUG)

        self._task_start(task, prefix='TASK')

```

Now the output looks like:

```
ROLE [some_role] *****************************************************

TASK [some_role : I'm a task!] ***************************************
ok: [MACHINE]

TASK [some_role : Some other task!] **********************************
ok: [MACHINE]

ROLE [another_role] **************************************************

TASK [some_role : I'm a task!] ***************************************
ok: [MACHINE]

```

With the role line being bolded. It's a work in progress, but it at least makes it a lot more obvious where roles are being executed. My determination on `block` is that it is impossible to do what I wanted, due to the way the block class is implemented in Ansible.
> 3 votes

# Answer
include\_role is a task and the name is printed (see below). block is not a task and name is not printed by any plugin.

The play

```
tasks:
  - name: "I'm a block!"
    block:
      - name: "I'm a role!"
        include_role: name=role5

```

gives:

```
PLAY [localhost]
TASK [I'm a role!]
TASK [role5 : I'm a task!]

```
> 0 votes

---
Tags: ansible
---


title: Code review with GitLab?

text:
# Question
Title: Code review with GitLab?
Is there a way to perform code review (for your own projects) using GitLab? For instance automated reviews, but also manual review where you can comment and propose changes at a review of a commit?

# Answer
> 2 votes
Yes, it's possible. Gitlab has the concept of Discussions (either single comments or discussion threads), which you can have on a variety of development items/activities, including the code itself (in which case they effectively are code reviews):

> The ability to contribute conversationally is offered throughout GitLab.
>
> You can leave a comment in the following places:
>
> * issues
> * epics
> * merge requests
> * snippets
> * commits
> * commit diffs

But note that only merge request discussions are presently considered resolvable.

Even though the check for a merge request can be configured to be automated, the starting of a review/discussion itself (if/when needed) is a manual operation. From Threaded discussions:

> To start a threaded discussion, click on the **Comment** button toggle dropdown, select **Start discussion** and click **Start discussion** when you’re ready to post the comment.

---
Tags: gitlab
---


# Question
Title: Serverless framework: how to specify a c++ lambda function with provided deployment package
I have followed the guide in https://aws.amazon.com/blogs/compute/introducing-the-c-lambda-runtime/ to package my c++ lambda function into a zip file. The zip file has the contents: function.zip bin function lib many .so files, etc. bootstrap

In the guide the lambda is deployed using the `aws lambda create-function` with the handler specified to be the same as the zip file name.

I would like to configure this lambda function in my serverless.yml and deploy it with the rest of my lambdas. I am not sure what the handler field below should be and using the zip file name didn't work out.

```
functions:
  function:
    runtime: provided
    package:
      include:
        - function.zip
    handler: function
    events:
      - http:
          path: function
          method: get
          cors: true
          authorizer: aws_iam

```

When I run `serverless deploy -v` I get warnings like:

> Serverless: WARNING: Entry for function@function could not be retrieved. Please check your service config if you want to use lib.entries.

# Answer
> 1 votes
I don't see any documentation from serverless.com that they support C and C++ (while AWS Lambda might do), I have asked about it now so that we can know:

https://forum.serverless.com/t/c-and-c-support/8045

("Supports Node.js, Python, Java, Go, C#, Ruby, Swift, Kotlin, PHP, Scala, & F#") but if you want to try then I think that you should try some other name than "function" for your function. I did not try this at home but all examples use some other name than "function" because that word might be reserved.

https://serverless.com/framework/docs/providers/aws/guide/resources/

If that still won't work, check your path to your code and maybe you also want to set up everything up with a CI/CD pipeline with GitLab for example. That's how I do it.

---
Tags: aws-lambda, serverless
---


title: i have questions about docker-compose

text:
# Question
Title: i have questions about docker-compose
postgres:

```
image: healthcheck/postgres:alpine
environment:
  POSTGRES_USER: postgres

```
* What is the default user of postgresql?
* How does hardcoding the postgres user improve security?
* What special rights does the postgres user have?
* What is the upside and downside of using an unprivileged user in this scenario?
* Is it necessary in our scenario to have an unprivileged user?

# Answer
> 1 votes
I searched for the **healthcheck/postgres:alpine** and I found the following repository with the **Dockerfile**:

https://github.com/docker-library/healthcheck/blob/master/postgres/Dockerfile.alpine

Assuming this is the correct repo for your docker image, the dockerfile specifies:

```
FROM postgres

COPY docker-healthcheck /usr/local/bin/

HEALTHCHECK CMD ["docker-healthcheck"]

```

So it extends the postgres docker image which is documented here:

https://hub.docker.com/\_/postgres

Specifically for the password they mention the following:

> POSTGRES\_PASSWORD This environment variable is recommended for you to use the PostgreSQL image. This environment variable sets the superuser password for PostgreSQL. The default superuser is defined by the POSTGRES\_USER environment variable.
>
> Note 1: The PostgreSQL image sets up trust authentication locally so you may notice a password is not required when connecting from localhost (inside the same container). However, a password will be required if connecting from a different host/container...

Read the whole documentation at the image homepage.

---
Tags: docker-compose, postgresql
---


# Question
Title: What does the RAILS_ENV environment variable do in Ruby on Rails?
I have this `docker-compose.yml`:

```
ports:
  - "3000:3000"
environment:
  RAILS_ENV: development

```
* What does the RAILS\_ENV do?
* What difference does it make, when RAILS\_ENV is changed to development?
* How does the change affect the automated build?

# Answer
RAILS\_ENV is en environment variable. Environment variable is a setting which may be read inside the container. Though, environment variables do exist on most of the operating systems https://en.wikipedia.org/wiki/Environment\_variable So the only specific here is that you can set an environment variable for the container and pass some values inside that way.

I assume you are working with docker-compose example for Ruby on Rails. So for the meaning of the RAILS\_ENV value you should refer to the Ruby on Rails documentation https://guides.rubyonrails.org/configuring.html#rails-environment-settings
> 1 votes

---
Tags: ruby-on-rails
---


title: How to avoid build scripts duplication

text:
# Question
Title: How to avoid build scripts duplication
Let's say you've got several code repositories on GitHub. You want to build, test and deploy them. You picked a continuous integration service (e.g. Travis CI, AppVeyor or something else) and wrote some scripts for building, testing and deploying.

If your projects are very similar, let's say all of them written in Golang, the result of the build is always a Docker image and deployment is always a push to Docker Hub, you don't want to duplicate the scripts across all of your repositories. How would you avoid that?

# Answer
> 1 votes
You could extract the common, shareable portions of the scripts which you'd package as either standalone scripts or libraries (depending on the scripting language you use and/or preferences), placed in a shared location (of course, backed by their own repository).

Then, in the scripts from your code repositories you'd replace the extracted portions with invocations of the scripts (or calls into the libraries) from the shared location.

Some CI/CD tools *might* be capable of directly invoking the scripts from the shared location, so if there aren't any specific portions in some of the code repo scripts you could drop them completely from those repos.

# Answer
> 0 votes
Hey Sergey and Welcome to the DevOps StackExchange.

I agree with @bgdnlp on using git submodules if you are really keen on keeping your bash scripts.

I however would recommend you look at other tooling to facilitate your transition into the world of CI/CD. While your stack is simple keeping a couple bash scripts handy is a sensible way to go about it. As your stack gets more complicated and the need for more testing, secrets management, monitoring and performance testing arises I think you will find that your bash scripts are a little lacking.

Where I work we use Jenkins to handle our builds which allows us to use shared libraries written in Groovy. We right one function and can now use it across many builds.

You could also do it with some newer tools like http://buildbot.net/

You could also convert your scripts to go or python and bundle them up in a package and install them each time on your build agent.

# Answer
> 0 votes
What I have seen working is to use a custom docker image with all the common tools bundled in during the building process. And it seems that Travis CI supports this as well

https://docs.travis-ci.com/user/docker/

That way you can always expect that the custom building modules are available during the building process.

---
Tags: continuous-integration, automation, shell-script
---


# Question
Title: Using HTTPs in application vs reverse proxy
Is there any difference if you use TLS in your application for example in node or java(spring), and if you use normal HTTP in your application and then set up Nginx as a reverse proxy and configure your TLS certificate throw Nginx? What would be the benefits of using one instead of another?

# Answer
You will find people from both camps (terminating TLS as soon as possible vs terminating TLS in the application code) but it's not as *serious* as for example tabs vs spaces.

The benefit of terminating TLS by reverse proxy is that application developer does not have to implement TLS in the application itself - possibly introducing bugs. Another benefit mentioned by jdog is the CPU cost; which is negligible these days - unless you are running 100s of micro-services and it all adds up to a large chunk of money.

Personally I always go with reverse proxy terminating the TLS unless there is a specific requirement against it (security reasons). I also make sure that service behind reverse proxy listens on the localhost / local network only - is not accessible directly form the internet.

You may find wikipedia article about TLS termination useful.
> 2 votes

# Answer
There is no direct technical benefit. Some people argue that it takes some cpu load of the Web server. However there are equally articles that claim its only a very small percentage of cpu required to handle TLS.

One major benefit is that a http only caching server (for example Varnish) can be put between http server and tls termination without changing either Web server or public IP address. I have chosen this for my own Web hosting setup, as it is not always feasible to change DNS records en masse.
> 0 votes

---
Tags: nginx
---


# Question
Title: DevOps Newbie - How to automate windows infrastructure deployment?
DevOps newbie here, so bear with me. I have a project where infrastructure deployment and application deployment is done using powershell and it is tied to VMWare vCenter.

For every client deployment, a deployment VM is created manually on their vCenter \>\> everything (application binaries and automation scripts) is copied manually to this VM \>\> DevOps run powershell script on this VM to create other VMs, configure them, and deploy application binaries.

What you would recommend to improve this process ?

# Answer
The first thing I can recommend for you is to remove all of the manual steps in your deployment pipeline. DevOps is all about automating as much as possible to allow for fast, frequent, and safe deployment of your infrastructure.

Second, if you haven't already, I would recommend using a build and deployment tool (Jenkins, Bamboo, GitLabCI, etc.) to compile your binaries and execute your automated deployment/configuration scripts. Having a central location for your deployments allows you to be able to scale more effectively, avoid repetition, and monitor your deployments more accurately. You should also store ALL of your scripts in some form of revision control ASAP (if they are not already).

Finally, I recommend you read up more on DevOps best practices and how it can affect your organization. In order for DevOps to be truly effective, your entire development team and company must adopt best practices and be 100% on board with a shift in how you manage your deployments.

Some reading I can recommend are:

The Phoenix Project by Gene Kim, Kevin Behr, and George Spafford

A telling of a fictional, but very realistic story of DevOps transforming a company and the opticals that the team faces to get there.

The DevOps Handbook by Gene Kim, Patrick Debois, John Willis, Jez Humble, and John Allspaw

Outlines the steps/stages to transform your company into a DevOps organization along with an assortment of real life DevOps transforming case studies.

Accelerate by Nicole Forsgren, Jez Humble, and Gene Kim

A plethora of empirical data and analysis on how DevOps has improved organizations both within IT and the entire company.

Continuous Integration by Paul Duvall, Steve Matayas, and Andrew Glover

Lays out the steps to take to move your organization to be able to deploy to the highest standards possible.
> 1 votes

---
Tags: terraform, packer, vmware
---


# Question
Title: In Jenkins when to use SSH site and when to use a node?
What are the advantages of each option? What are the disadvantages? Is there a best practice in this? I fail to see any critical differences between them.

In my current place of employment we use nodes exclusively, however they are all in our internal network.

I have a task that needs to be delegated to an external server. It does not need access to our GIT. Essentially, it will upload data to a file server, and does not need to access our servers, network, data, etc.

# Answer
> 1 votes
If I need to build, run checks, tests, ... I use a node.

If I want to open a connection to a server to deliver the built code to production for example, I use the sshCommand step from SSH Step plugin. When the code is delivered the ssh connection is closed.

Having a node always connected to your jenkins can be unpleasant. Let's say the server you connect to belongs to a partner and he wants to audit the times you connect to put the files on the server. In that case he cannot say. Having a node always connected also means you need to maintain the connection, imagine the server is configured to disconnect you after a while or the server reboots and the connection is not relaunched.

---
Tags: jenkins
---


title: Python Script in Jenkins

text:
# Question
Title: Python Script in Jenkins
I am trying to run a python script in Jenkins. My Jenkins job is integrated with Jira & on build its gives JIRA\_ISSUE\_KEY. I want to save the value of JIRA\_ISSUE\_KEY in a variable.

```
import subprocess
e = 'echo "$JIRA_ISSUE_KEY"'
x = subprocess.call(e, shell=True)

```

Whenever I am doing print e it gives o/p as "echo "$JIRA\_ISSUE\_KEY"" & when I am doing print e its gives o/p as 0.

My overall output in Jenkins console is

```
JIRA comment is added
Running as SYSTEM
Building in workspace /var/lib/jenkins/workspace/Jenkins Jira
Integration Test
[Jenkins Jira Integration Test] $ python /tmp/jenkins2864498002834652992.py
MCO-15498
echo "$JIRA_ISSUE_KEY"
0
Finished: SUCCESS

```

I want to save the value MCO-15498 to some variable. How can I do that?

# Answer
> 1 votes
using bash script (or pipeline step `sh`), I would do:

```
JIRA_ISSUE_KEY=$(python my_python_script.py)
echo "$JIRA_ISSUE_KEY"

```

---
Tags: jenkins, python
---


# Question
Title: Jenkins fails to start after restart because of google login plugin had an exception
Jenkins version: Jenkins ver. 2.32.1 Google Login Plugin version: 1.3

I got the following error after I restart my jenkins, can anyone suggest me how to solve it?

The thing is :

* I already make sure that in plugins directory I had the google-login plugin enabled
* I didn't change anything on the plugins, just straight restart
* It's been so long since I restart my jenkins

```
    hudson.util.HudsonFailedToLoad: org.jvnet.hudson.reactor.ReactorException: java.io.IOException: Unable to read /var/lib/jenkins/config.xml
        at hudson.WebAppMain$3.run(WebAppMain.java:248)
    Caused by: org.jvnet.hudson.reactor.ReactorException: java.io.IOException: Unable to read /var/lib/jenkins/config.xml
        at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:269)
        at jenkins.InitReactorRunner.run(InitReactorRunner.java:47)
        at jenkins.model.Jenkins.executeReactor(Jenkins.java:1110)
        at jenkins.model.Jenkins.<init>(Jenkins.java:926)
        at hudson.model.Hudson.<init>(Hudson.java:85)
        at hudson.model.Hudson.<init>(Hudson.java:81)
        at hudson.WebAppMain$3.run(WebAppMain.java:231)
    Caused by: java.io.IOException: Unable to read /var/lib/jenkins/config.xml
        at hudson.XmlFile.unmarshal(XmlFile.java:161)
        at jenkins.model.Jenkins.loadConfig(Jenkins.java:3015)
        at jenkins.model.Jenkins.access$1100(Jenkins.java:326)
        at jenkins.model.Jenkins$16.run(Jenkins.java:3033)
        at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
        at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282)
        at jenkins.model.Jenkins$7.runTask(Jenkins.java:1086)
        at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210)
        at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
        at java.lang.Thread.run(Thread.java:748)
    Caused by: jenkins.util.xstream.CriticalXStreamException: org.jenkinsci.plugins.googlelogin.GoogleOAuth2SecurityRealm : org.jenkinsci.plugins.googlelogin.GoogleOAuth2SecurityRealm
    ---- Debugging information ----
    message             : org.jenkinsci.plugins.googlelogin.GoogleOAuth2SecurityRealm
    cause-exception     : com.thoughtworks.xstream.mapper.CannotResolveClassException
    cause-message       : org.jenkinsci.plugins.googlelogin.GoogleOAuth2SecurityRealm
    class               : hudson.model.Hudson
    required-type       : hudson.model.Hudson
    converter-type      : hudson.util.RobustReflectionConverter
    path                : /hudson/securityRealm
    line number         : 485
    version             : not available
    -------------------------------
        at hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:356)
        at hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:270)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72)
        at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:65)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:50)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:134)
        at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
        at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1189)
        at hudson.util.XStream2.unmarshal(XStream2.java:114)
        at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1173)
        at hudson.XmlFile.unmarshal(XmlFile.java:159)
        ... 11 more
    Caused by: com.thoughtworks.xstream.mapper.CannotResolveClassException: org.jenkinsci.plugins.googlelogin.GoogleOAuth2SecurityRealm
        at com.thoughtworks.xstream.mapper.DefaultMapper.realClass(DefaultMapper.java:79)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:30)
        at com.thoughtworks.xstream.mapper.DynamicProxyMapper.realClass(DynamicProxyMapper.java:55)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:30)
        at com.thoughtworks.xstream.mapper.PackageAliasingMapper.realClass(PackageAliasingMapper.java:88)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:30)
        at com.thoughtworks.xstream.mapper.ClassAliasingMapper.realClass(ClassAliasingMapper.java:79)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:30)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:30)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:30)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:30)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:30)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:30)
        at com.thoughtworks.xstream.mapper.ArrayMapper.realClass(ArrayMapper.java:74)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:30)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:30)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:30)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:30)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:30)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:30)
        at com.thoughtworks.xstream.mapper.SecurityMapper.realClass(SecurityMapper.java:71)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:30)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:30)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:30)
        at hudson.util.XStream2$CompatibilityMapper.realClass(XStream2.java:282)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:30)
        at hudson.util.xstream.MapperDelegate.realClass(MapperDelegate.java:43)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:30)
        at com.thoughtworks.xstream.mapper.CachingMapper.realClass(CachingMapper.java:48)
        at hudson.util.RobustReflectionConverter.determineType(RobustReflectionConverter.java:461)
        at hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:327)
        ... 22 more

```

# Answer
What I would do is check read/write permission settings on `/var/lib/jenkins/config.xml`, then remove the xml in `/var/lib/jenkins/` that has the plugin's name in its filename. (backup it before just in case)
> 1 votes

---
Tags: jenkins
---


# Question
Title: AWS lambda: works locally but fails in deplyment
I have a node.js lambda function that adds a database entry and then calls another lambda function. The code works fine when I test locally via

```
serverless invoke local --function create --path mocks/create-event.json

```

but it fails when I invoke the deployment

```
aws lambda invoke  --function-name create --payload file://mocks/create-event.json out.txt

```

here is the relevant lambda code:

```
try {
    await dynamoDbLib.call("put", params);  //OK locally and in deployment
    var lambda = new AWS.Lambda();
    const req = {
      FunctionName: 'eqtor-api-dev-mysim',
      InvocationType: 'RequestResponse',
      LogType: 'Tail',
      Payload: JSON.stringify({ name: 'me' })
    };
    const result = await lambda.invoke(req).promise(); //OK locally but not in deployment
    console.log(result);
    return success(params.Item);
  } catch (e) {
    return failure({ status: false });
  }

```

The command shows

```
{
    "ExecutedVersion": "$LATEST",
    "StatusCode": 200
}

```

and in out.txt, I get

```
{"statusCode":500,"headers":{"Access-Control-Allow-Origin":"*","Access-Control-Allow-Credentials":true},"body":"{\"status\":false}"}

```

Any tips on how to debug this?

# Answer
I've resolved the issue now.

First of all, console.log(e) should be used to log the result. Then the error will appear in CloudWatch.

The problem was that the role that was used by the lambda did not permit invoking other lambdas. This can be fixed by granting the missing InvokeFunction permission for the role.
> 1 votes

# Answer
Generally speaking a good practice for AWS Lambda is to map what are the services and permissions required for it to run properly then create a dedicated role for it with the required policies. I also recommend not using roles on multiple Lambdas, it tends to get very insecure fast if you do.
> 0 votes

---
Tags: aws-lambda, node.js
---


title: Jenkins Shared lib not working properly

text:
# Question
Title: Jenkins Shared lib not working properly
Recently I tried to add Shared Libraries to my Jenkins instance to make it easier to maintain all code that runs in each project.

1. I have a repo `myorg/jenkins`

Inside of it I have Jenkinsfile and groovy file meant to contain libraries, Jenkinsfile looks like this:

```
@Library('utils@develop')_
import com.*

jenkinsSlave = "slave.myorg.com"

node(jenkinsSlave) {
    stage('Test') {
        def var = "kek"
        myorg.echoTest(var)
    }
}

```

I try to execute `echoTest` method there and I'm passing one argument.

Shared libs look like this (Jenkinsfile is in root directory of this repo):

```
src
└── com
    └── myorg
        └── Utils.groovy

```

Utils.groovy:

```
#!/usr/bin/groovy
package com.myorg

def echoTest(test) {
  println("Test + ${test}")
}

return this

```

In Jenkins I've added this repo to global libraries section and I pointed to main branch, which is develop in this case.

Unfortuantely, when I run build jenkins throws this:

```
groovy.lang.MissingPropertyException: No such property: myorg for class: groovy.lang.Binding
    at groovy.lang.Binding.getVariable(Binding.java:63)
    at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetProperty(SandboxInterceptor.java:264)
    at org.kohsuke.groovy.sandbox.impl.Checker$6.call(Checker.java:288)
    at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:292)
    at org.kohsuke.groovy.sandbox.impl.Checker.checke

[...]

```

I tried various configuration but so far I cannot make this work. I've tried to follow few tutorials about implementing this and I cannot pass through here.

Does someone notice any misconfiguration in this setup?

# Answer
You have not instantiated the Utils class in the myorg classpath so you cannot reference the method `echoTest()`. Try `Utils.echoTest` or just `echoTest()`
> 1 votes

# Answer
When you see `scriptsecurity.sandbox` error, that means you need to allow the script from Jenkins script approval page https:///scriptApproval/ Also I would recommend you not to import your package directly within your pipeline. Instead I would define a step under `./vars` folder and only there import the custom package
> 0 votes

---
Tags: jenkins, continuous-integration, jenkins-pipeline, jenkinsfile
---


# Question
Title: How to run/deploy devilbox (Docker image) docker-compose on GCP without a Dockerfile?
Devilbox comes with a docker-compose file to run a docker container: https://github.com/cytopia/devilbox

However, on the GCP Kubernetes Engine it is only possible to start up a new container via a Dockerfile. I searched for a translation/conversion tool of the docker-compose file to a Dockerfile. I could not find an easy way and also read that this is not the go to approach. Is there a way to let it run in the Kubernetes Engine or do I have to create a manual instance, upload and run the code manually? Any other approaches?

# Answer
Firstly, `docker-compose` is is a tool for defining and running multi-container Docker applications. (https://docs.docker.com/compose/). Dockerfile is single-container, if you want to run multi-containers on GKE you should convert docker-compose to kubernetes deployment.

You can manually convert or use http://kompose.io/ to convert automatically.
> 1 votes

---
Tags: docker, kubernetes, deployment, google-cloud-platform
---


title: Getting rid of redundant Ansible roles

text:
# Question
Title: Getting rid of redundant Ansible roles
I currently have a bunch of roles, like:

* apache2
* gcc
* gedit
* ...etc.

For each of those, I have a task file that looks like this:

```
---
- name: "Install {{ role_name }}"
  become: true
  package:
    name: "{{ role_name }}"
    state: present

```

Is there a way to abstract this to a higher level and have a single "Install package" role, and then somehow specify a bunch of specific packages to install per host in a variable or something?

# Answer
Is this what you're looking for?

```
- hosts: all
  vars:
    bunch_of_specific_packages:
      - apache2
      - gcc
      - gedit
  tasks:
    - package:
        name: "{{ item }}"
      loop: "{{ bunch_of_specific_packages }}"

```
> 2 votes

# Answer
Another way to do this would be to set a role variable in your host\_vars file called "role" and assign the role to install that way. You could also use the loop that Vladimir provided to install multiple roles.

```
tasks:
  - name: "Install Apache"
    become: true
    package:
      name: apache2
      state: latest
    when: role == "apache"

  - name: "Install GCC"
    become: true
    package:
      name: gcc
      state: latest
    when: role == "gcc"

```

You could also use tags to control what gets run in your play:

```
tasks:
  - name: "Install Apache"
    become: true
    package:
      name: apache2
      state: latest
    tags:
       - web
  - name: "Install GCC"
    become: true
    package:
      name: gcc
      state: latest
    tags:
       - gcc

```

And run your playbook setting the tags you require:

```
ansible-playbook example.yml --tags "web,gcc"

```
> 0 votes

---
Tags: ansible
---


# Question
Title: Why does Docker fail to start on AWS ECS instances. ECS stateful instance fails initialization. ECS stateful instance doesn't join the ECS cluster
The ECS instance fails to join the ECS cluster because it relies on docker to launch its ecs agent. If in this case docker fails to start then the whole process halts.

Docker fails to start in AWS ECS instances with the following message in /var/log/docker :

```
Error starting daemon: error initializing graphdriver: devicemapper: Non
existing device docker-docker--pool

```

Notice the following.

Instance AMI is a COPY of the AWS ECS optimized AMI with the embedded ECS agent(and docker config for ECS)

# Answer
> 1 votes
The key here is the behavior of AWS AMIs in the context of ECS. By default the AWS ECS Amazon Linux AMI optimized for ECS is in charge of VOLUME management but there's a problem if you copy that AMI and try to launch a new instance -- the SECONDARY volume that ECS uses to deploy docker on can go missing.

Troubleshooting steps:

* Check `/var/log/cloud-init-output.log` and see if you have an error message like:

> ERROR: /dev//dev/xvdcz is not a valid block device.
* Check your instance mounted volumes and make sure that the Secondary Volume exists. Usually if you see the error above it means that only `/dev/xvda` is mounted.

**FIX**:

Enforce your Launch Configuration, Launch Template or script to always deploy the secondary volume `/dev/xvdcz` **regardless** of AMI configuration. Do not rely on the AMI as unfortunately the AMI flow has changed and might not preserve all the necessary configuration.

---
Tags: docker, amazon-web-services, amazon-ec2, amazon-ecs, amazon-ami
---


# Question
Title: Gitlab - Is there any reason not to default enable 'Allow Users to Request Access' for Projects?
Is there any reason not to default enable 'Allow Users to Request Access' for Projects?

I can conceive that there may be a reason this is not enabled by default. But I am considering enabling it if possible. If not that may be a feature request.

Anybody know why it is default disabled?

# Answer
It's just a safeguard for preventing unwanted user accounts from accessing projects. It's disabled by default so that you can control from the start which accounts can access a project and not have unwanted spam of requests for access.

One example I can think of, is if you only wanted users to be able to use their company email/account to access projects, and not their personal account. It could even happen by accident that someone unknowingly requests access with the wrong account.
> 1 votes

---
Tags: gitlab
---


# Question
Title: TFS Service hook should not fire when work item is updated via REST API
I wrote a service hook on work item update but I don't want to fire the service hook when work item is updated from REST API, becz its the same data that other system had and it post to ADO and now ADO is posting back that data again, and it is creating duplicates, so any workaround to not to fire service hook when work Item is updated using the REST API or can bypass integration user, ?

# Answer
> 1 votes
The service hook will fire as part of the notifications process. You can suppress all notifications by setting the URI parameter `suppressNotifications` to `true`.

An example request URI might be:

```
PATCH https://dev.azure.com/fabrikam/_apis/wit/workitems/{id}?suppressNotifications=true&api-version=5.0

```

See the work item update API documentation for more information.

---
Tags: azure-devops, azure, tfs
---


# Question
Title: what is better between Jenkins & AWS CodeDeploy for CI/CD
I'm considering what is better and easier to use for CI/CD pipeline between Jenkins and AWS CodeDeploy.

e.g. Jenkins server request webhook to git repo(github), and deploy to EC2.

e.g. When push to git repo, deploy to EC2 go through codeDeploy.

Thank you for your help!!

# Answer
> 2 votes
Both are excellent tools, but it depends on your use cases for which is better. I would highly recommend using the entire AWS Ci/Cd suite (CodeBuild/CodePipeline/CodeDeploy) to gain the full benefit of using your resources in AWS. Here are pros for both:

**Jenkins**

* Multitude of customizability plugins
* Integration with all cloud providers (GCP, Azure, etc.)
* Open sourced
* Configuration as code through Jenkins Pipelines
* Able to manage your own Jenkins Server (vs AWS which manages it for you)
* Customizable GUI

**AWS CI/CD**

* No plugins (no dependency hell)
* Trivial integration with IAM (for both users and infrastructure)
* Easy integration with AWS Secrets Manager, KMS, and other secret management tools
* AWS managed infrastructure
* Infrastructure as code through CloudFormation
* Scalable out of the box

TLDR; Choose Jenkins if you want more control/configuration of your deployment infrastructure or you are integrating with many non-aws resources, otherwise, choose the AWS CI/CD suite for less overhead and better integration with AWS.

# Answer
> 1 votes
I know a good amount about Jenkins so I can give you specific information about that, but I know virtually nothing about AWS code deploy, that being said using my general AWS knowledge, there are a few things I think I can safely tell you to get started.

1. Jenkins is open source however you can get hosted Jenkins if you prefer a managed solution. CodeDeploy on the other hand is purely managed so I don't believe you have the option for something such as on prem or any other self hosted solutions.
2. Part of what gives Jenkins it's incredible power is the massive community following and subsequent plugins that are continuously churned out for Jenkins. In comparison I assume the AWS marketplace may be rather limited as to it's offerings but I can't offer any official word on this.

I hope that helps in some capacity.

---
Tags: amazon-web-services, jenkins-pipeline, cicd, aws-codedeploy
---