# Question
Title: Can't make MySQL 5.7 work with Docker Compose, no matter what
I'm configuring a docker compose file, and the first step is to configure the MySQL service (so I can further use in my app service), but no matter what, I cannot enter MySQL within the container, even when I set all environments. That's how my docker-compose.yml file looks like:

```
version: "3.3"

services:
  db:
    image: mysql:5.7
    volumes:
      - ./tmp/db:/var/lib/mysql
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: root
      MYSQL_DATABASE: wordpress
      MYSQL_USER: wordpress
      MYSQL_PASSWORD: wordpress

```

I've tried SEVERAL (I literally lost count of how many) other configurations, and none of them work, it just doesn't work.

If I go inside the container with `docker exec it ${CONTAINER_NUMBER} bash` and try to access MySQL, it will ALWAYS return:

```
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)

```

Or if I try with the user wordpress:

```
ERROR 1045 (28000): Access denied for user 'wordpress'@'localhost' (using password: YES)

```

I'm pretty confident that I spent more than 8 hours only changing the configuration, so the only thing I can think right now is: the image is broken.

I'm in a Ubuntu 19, not sure if this makes any difference.

# Answer
First of all: your `docker-compose.yml` file is absolutely fine. I just tested it, created the necessary dir for bind mount, started the compose project (`docker-compose up -d`) and was able to login into mysql (`docker-compose exec db bash -c "mysql -u root -proot"` \- worked with `wordpress` as well).

So now I'll take a guess (which is usually good...): you already have an initialized mysql filesystem in `./tmp/db` from a previous run (or copied from somewhere else). In this case, here is what you might have missed in the mysql docker image documentation in the section on environment variables:

> Do note that none of the variables below will have any effect if you start the container with a data directory that already contains a database: any pre-existing database will always be left untouched on container startup

If you want to have a look under the hood how this happens, you can study the image entrypoint script

So, as you may have now understood, your container starts but it is using the existing mysql filesystem with the previous settled dbs, data, users and passwords.

Simply stop your container, delete any files and directories in `./tmp/db` and restart from scratch: you should get your blank db install and be able to login in mysql with the users/pass defined in `docker-compose.yml`
> 5 votes

---
Tags: docker, docker-compose, containers
---


title: Nginx unable to get SSL cert to work

text:
# Question
Title: Nginx unable to get SSL cert to work
I'm trying to get a kubernetes cluster using the nginx ingress controller and cert manager to get an SSL cert from lets encrypt. Currently is appears the cert has been issued however I am unable to get it to load properly inside Chrome or via curl.

Below are some config files that show the current config and following that some describes.

```
apiVersion: cert-manager.io/v1alpha2
kind: Issuer
metadata:
  name: letsencrypt-prod
  namespace: cert-manager-prod
spec:
  selfSigned: {}
---
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
  name: letsencrypt-prod
  namespace: cert-manager-prod
spec:
  commonName: me@example.com <- This is not actually what I have just for example reasons
  secretName: letsencrypt-prod-tls
  issuerRef:
    name: letsencrypt-prod

```

```
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    # add an annotation indicating the issuer to use.
    kubernetes.io/ingress.class: "nginx"
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
    # needed to allow the front end to talk to the back end
    nginx.ingress.kubernetes.io/cors-allow-origin: "https://portal.<domain>.com"
    nginx.ingress.kubernetes.io/cors-allow-credentials: "true"
    nginx.ingress.kubernetes.io/enable-cors: "true"
    nginx.ingress.kubernetes.io/cors-allow-methods: "GET, PUT, POST, DELETE, PATCH, OPTIONS"
    # needed for monitoring - maybe
    prometheus.io/scrape: "true"
    prometheus.io/port: "10254"
    #for nginx ingress controller
    ad.datadoghq.com/nginx-ingress-controller.check_names: '["nginx","nginx_ingress_controller"]'
    ad.datadoghq.com/nginx-ingress-controller.init_configs: '[{},{}]'
    ad.datadoghq.com/nginx-ingress-controller.instances: '[{"nginx_status_url": "http://%%host%%:18080/nginx_status"},{"prometheus_url": "http://%%host%%:10254/metrics"}]'
    ad.datadoghq.com/nginx-ingress-controller.logs: '[{"service": "controller", "source":"nginx-ingress-controller"}]'
  name: prod-ingress
  namespace: production
spec:
  rules:
  - host: api.<domain>.com
    http:
      paths:
      - backend:
          serviceName: api
          servicePort: 8090
        path: /
  tls: # < placing a host in the TLS config will indicate a certificate should be created
  - hosts:
    - api.<domain>.com
    secretName: prod-ingress-cert # < cert-manager will store the created certificate in this secre

```

```
me@LAPTOP-LINUX:~/Projects/k8/tmp$ kubectl get certificates
NAME                READY   SECRET              AGE
prod-ingress-cert   False   prod-ingress-cert   30m

```

```
Name:         prod-ingress-cert
Namespace:    production
Labels:       <none>
Annotations:  <none>
API Version:  cert-manager.io/v1alpha2
Kind:         Certificate
Metadata:
  Creation Timestamp:  2020-01-26T21:29:07Z
  Generation:          1
  Owner References:
    API Version:           extensions/v1beta1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Ingress
    Name:                  prod-ingress
    UID:                   adec15ea-4cde-4376-8e6a-306586e57119
  Resource Version:        2780
  Self Link:               /apis/cert-manager.io/v1alpha2/namespaces/production/certificates/prod-ingress-cert
  UID:                     c86bf4aa-8435-4da0-a822-1a0e9d1df38e
Spec:
  Dns Names:
    api.<domain>.com
  Issuer Ref:
    Group:      cert-manager.io
    Kind:       ClusterIssuer
    Name:       letsencrypt-prod
  Secret Name:  prod-ingress-cert
Status:
  Conditions:
    Last Transition Time:  2020-01-26T21:29:07Z
    Message:               Waiting for CertificateRequest "prod-ingress-cert-1024021523" to complete
    Reason:                InProgress
    Status:                False
    Type:                  Ready
Events:
  Type    Reason        Age   From          Message
  ----    ------        ----  ----          -------
  Normal  GeneratedKey  30m   cert-manager  Generated a new private key
  Normal  Requested     30m   cert-manager  Created new CertificateRequest resource "prod-ingress-cert-1024021523"

```

That's everything I can think of that might be of use in resolving this issue. I should note I have a similar setup running on development (well did till I took the cluster down today) that is/was working.

# Answer
Issue ended up being that I was creating a self signed cert because I was using the wrong issuer. I had an issue before with the dev.yaml file but never updated for this new ENV.
> 1 votes

---
Tags: kubernetes, nginx, letsencrypt, cert-manager
---


# Question
Title: How do I set up certbot, with Nginx, both in a Docker container?
Title says it all I guess. I didn't even know what Ubuntu/Nginx/Docker/etc was a week ago but I've managed to set up Docker/Docker Compose, PHP-FPM, PHPMyAdmin + Gitlab on a reverse proxy through Nginx with somewhat difficulty. My Github repo documenting my steps.

Basically managed to do everything I wanted to do set-up wise. Last step is setting up SSL certs for my site. It wasn't as straightforward as I was hoping, I haven't been able to find any clear guides on what to do regarding setting up Cerbot and having it on automatic renewal through a docker container. Lot of the shorter tutorials seem to use custom Certbot docker images which I'm not sure why they're different, or they have custom shell scripts that make dummy certs and delete them for some reason. The Certbot docker hub page is kind of devoid of any official instructions as well...

Any pointers or instructions would be extremely appreciated, thanks.

# Answer
Answered my own question, documented my steps on my Github repo.

Recap:

Add Nginx config to desired domains:

```
location ~ /.well-known/acme-challenge {
        allow all;
        try_files $uri $uri/ /index.php;
    }

```

Use certbot staging to try out test certificates before running the real deal.

```
sudo docker run -it --rm -v /some/place/to/save/letsencrypt:/etc/letsencrypt -v /some/place/to/save/lib:/var/lib/letsencrypt -v /some/place/to/have/html:/data/letsencrypt certbot/certbot certonly --webroot --register-unsafely-without-email --agree-tos --webroot-path=/data/letsencrypt --staging -d example.com

```

Run the real thing once ready

```
sudo docker run -it --rm --v /some/place/to/save/letsencrypt:/etc/letsencrypt -v /some/place/to/save/lib:/var/lib/letsencrypt -v /some/place/to/have/html:/data/letsencrypt certbot/certbot certonly --webroot --email someemail@account.com

```

And then add in Nginx configs redirecting from http to https with ssl cert locations, updating nginx in docker-compose.yml to have access to certs. Go a tiny bit more in-depth about it in my repo
> 1 votes

# Answer
I went through one of the guides that linuxserver.io published to get started learning about how to setup the SSL encryption. I was able to use their guide to use my own images and get up and running Nginx with an SSL cert for my site.

Automating it into a build workflow can be a challenge if you are not using a static IP. Additionally, if you need to be able to accept external web traffic you will need to work with however controls your organizations firewall.
> 0 votes

---
Tags: docker, nginx, certbot
---


# Question
Title: What is Unit testing in the context of YML configuration Alerting rules?
I read a very short blog posting about Unit testing alerts with Prometheus

Can you explain me why I need unit testing for \*.yml? What is the benefit of unit testing for configuration files?

I not understand what are the gains when implementing that.

# Answer
The important part of the blog post you link to isn't testing the syntax of the YAML file; it's testing the semantics of the Prometheus configuration.

The core configuration in that post includes a fragment:

```
- alert: MyAlert
  expr: avg without(instance)(up) < 0.75
  for: 2m
  labels:
    severity: page

```

How do you know this is the right configuration? If it's misconfigured then either your on-call team gets paged when the system is healthy, or it doesn't get paged when the system is down; both are bad.

The test setup shown in the blog post would let you test your Prometheus alerting setup, with artificial sample data (the "bar" instance is up for 5 minutes, then down for 5 minutes, and it's expected to page after it's been down for longer than 2 minutes), before you deploy it for real and without actually paging anyone. That gives you some confidence your monitoring setup will do what you expect, before you deploy it for real.
> 3 votes

---
Tags: configuration, prometheus, unit-testing
---


title: Authentication Error in Kubernetes

text:
# Question
Title: Authentication Error in Kubernetes
I am new to Kubernetes. I am getting below error while creating deployments in Kubernetes.

```
<!--
You are authenticated as: anonymous
Groups that you are in:

Permission you need to have (but didn't): hudson.model.Hudson.Read
 ... which is implied by: hudson.security.Permission.GenericRead
 ... which is implied by: hudson.model.Hudson.Administer
-->

```

Can anyone give me solution for this?

I am using below command to setup deployment

```
 kubectl run my-app --image=gcr.io/<dockeraccount>/<dockerrepository>:dockerimagetag

```

# Answer
> 1 votes
I found this same error in StackOverflow, problably that you problem it's about your authentication *anonymous*

If help you: JENKINS Authentication Fails

---
Tags: docker, kubernetes, kubectl
---


# Question
Title: Kubernetes: How to expose a Pod/Deployment to the Nodes Network
Ok, let's assume I've got the following theoretical setup:

I want to somehow assign IP-Addresses accessible by the PCs on the left to my pods. I can set routes etc. as I need since this setup is purely for practice. My goal is for every Pod to be able to expose any service (HTTP(s)/FTP/...) on any port. E.g.: :22, :22 for SSH

My first thought was to create `ClusterIPs` for every pod and somehow use ingress to assign IPs to those services, but as I later found out this is not what ingress is intended for.

My next step was to look at `LoadBalancer`-type services and this is where I'm currently at. My current impression is that they somehow only work with AWS/Google Cloud/Other providers and always expose those selected services to the internet, which is not my intention. I only want them to be accessible from my internal network.

Any input would be appreciated.

# Answer
Some clarification first, the `ClusterIP` is only accessible inside the cluster this is not the right choice to solve your problem and the `LoadBalancer` depends on the integration with your network, real or virtual, to generate the correct resources and assign an IP address.

With that said, an easy way open a service port should be to use services of type `NodePort` it assigns a port directly on the machine, the same way you do when you open ports on a Linux machine, but with the difference that all the process is managed by Kubernetes. In that scenario, you only need to worry about the routes between the networks managed on your router.

You have more info about `NodePort` on the official Kubernetes documentation here

**UPDATED**

As per the last comments, for the use case, the use of `NodePort` should work and probably is a good option to use some DNS management like external-dns to not worry about how to setup the right IP.
> 1 votes

---
Tags: kubernetes, networking
---


# Question
Title: What is the boundary of DevOps engineer job role?
I will not get into what is DevOps since everyone have their own meaning.

But I would like to understand what is the boundary of DevOps engineer. Sometimes people ping me and ask to me that,

> 1. can you please help me setting up VPN in my laptop?

sometimes,

> 2. From the RDP machine, I am not able to open MySQL workbench, it hangs, can you fix it please?

sometimes,

> 3. Production application is having performance issues, why is it happening?

sometimes,

> 4. Can you please schedule the performance testing script?

sometimes,

> 5. can you setup the spark and hadoop cluster?

sometimes,

> 6. can you create users in MySQL?

I think we need an industry wide boundary, otherwise, the role of DevOps engineer is going in every direction.

# Answer
From the "how it should properly work"/"how it was meant" perspective there is no real "DevOps engineer" because DevOps is a methodology/culture which emphasizes also on technical practices which should stretch across as many roles as possible and feasible.

How can we apply the CALMS (Culture, Automation, Lean, Measure, Share) mantra here? (Taking into approach that depending on factors like maturity, complexity and resources in the context, approaches need to be adapted to what helps to solve problems in an efficient and sustainable way)

* Requirements engineer: can I automatically qualify maturity of requirements and track their delivery?
* Software engineer: what is required to run my software in test either productive environment?
* Architect: can I have a big picture based on live metrics?
* Tester: can I create virtual temporary infrastructure to support execution of my code?
* System administrator: how can I better organize my help desk / 1st user support and leverage self-help / automation solutions?

Somebody with a wish to focus on the methodology itself will be somewhere in-between all these guys and obviously become "jack of all trades, master of none".

Given this, my consideration how I would approach your customers' inquiries could be:

1. Please follow the instructions to setup VPN in this Wiki/code from this Git repo and if it's not working, document/show steps you have taken.
2. Please share more details on the context (here is not clear who owns which parts).
3. Let's allocate time to develop emergency plan for immediate action if required and also run scaling tests on a test system to model experiments, collect data and finally learn under which circumstances the performance problems appear. Then, we will know why it is happening. :-)
4. Follow from 3.? What does mean schedule from your perspective and if you mean regular execution, how often? So please let's go back to previous item :-)
5. Possibly, have you got more specs which infrastructure option (Cloud, on premises) should we use for that and what and what are you going to do? Here is a link to generic infrastructure planner, let's allocate time to discuss what is specific for Spark/Hadoop here and how will delivery to it look like.
6. If identity management is not part of our architecture, let's assess what are you going to do. An outcome could be that you include creation of users to your database scripts.

So the bottom line in DevOps culture regarding the role boundary is:

* professional communication first
* the boundary is to set boundaries inviting people to a professional conversation between adults, to get more conscious on both sides (!) about requirements, responsibilities and goals. Not just thinking in solutions and delegation but also make a plan and assess risks.

Therefore seasoned and successful professionals who are today maybe 50+ yrs old say sometimes jokingly "we have been doing DevOps from the very beginning" and partly, they are right.

To conclude and highlight and reflect on the real-world existence of such job-role, organizations who do not have a DevOps culture proceed to define DevOps roles as the following, for example:

* DevOps engineer is an advanced systems administrator who can code up the infrastructure and automate it completely.
* DevOps engineer is a tools administrator with coding skills to support the Dev team and ensure delivery towards production (CI/CD, deployment scripting but also Jira administration)
* DevOps engineer is a developer focusing on including infrastructure elements into development, like containers and cloud APIs.

After all, many companies still look for the "full stack engineer" who should obviously become additionally the DevOps engineer (along with "AI/ML, blockchain and foobar").

Finally, the role boundary, possibly not only in the DevOps context, but in this special context indeed very likely:

* is also in your responsibility
* may change between projects and employers or just from time to time.
> 11 votes

# Answer
AFAIK, there is no boundary for DevOps engineer. When it comes to questions you have put together, it is general misconception to consider DevOps engineer as administrators.

DevOps is change in culture, change in mindset and change in approach.

DevOps culture fills the gaps development and operations.

DevOps mindset allows engineers to try out of the box combinations to explore tools, technology, services to fasten the process to test, build and deploy product.

DevOps approach will allow engineers to come out of sticking to certain role and explore.
> 2 votes

---
Tags: culture, project-management, job-role, team-role
---


# Question
Title: Containerized (official) GitLab CE starts and ends up in a boot loop with LDAP configuration
I have been working on deployment of a GitLab container with Ansible.

At one point the whole thing broke with a terribly unspecific error message. After that point the container kept rebooting about once a minute.

The `docker logs <container>` command gave me the following output:

```
Thank you for using GitLab Docker Image!
Current version: gitlab-ce=12.7.2-ce.0

Configure GitLab for your system by editing /etc/gitlab/gitlab.rb file
And restart this container to reload settings.
To do it use docker exec:

  docker exec -it gitlab vim /etc/gitlab/gitlab.rb
  docker restart gitlab

For a comprehensive list of configuration options please see the Omnibus GitLab readme
https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md

If this container fails to start due to permission problems try to fix it by executing:

  docker exec -it gitlab update-permissions
  docker restart gitlab

Cleaning stale PIDs & sockets
Preparing services...
Starting services...
Configuring GitLab...
/opt/gitlab/embedded/bin/runsvdir-start: line 37: /proc/sys/fs/file-max: Read-only file system
Starting Chef Client, version 14.14.29
resolving cookbooks for run list: ["gitlab"]
Synchronizing Cookbooks:
  - postgresql (0.1.0)
  - registry (0.1.0)
  - mattermost (0.1.0)
  - consul (0.1.0)
  - gitaly (0.1.0)
  - gitlab (0.0.1)
  - letsencrypt (0.1.0)
  - runit (4.3.0)
  - crond (0.1.0)
  - redis (0.1.0)
  - nginx (0.1.0)
  - acme (4.0.0)
  - package (0.1.0)
  - monitoring (0.1.0)
  - praefect (0.1.0)
Installing Cookbook Gems:
Compiling Cookbooks...

================================================================================
Recipe Compile Error in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb
================================================================================

Psych::SyntaxError
------------------
(<unknown>): did not find expected key while parsing a block mapping at line 2 column 5

Cookbook Trace:
---------------
  /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/helpers/settings_helper.rb:105:in `block in from_file'
  /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:29:in `auto_vivify'
  /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/helpers/settings_helper.rb:105:in `from_file'
  /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/helpers/settings_helper.rb:105:in `block in from_file'
  /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:29:in `auto_vivify'
  /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/helpers/settings_helper.rb:105:in `from_file'
  /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/config.rb:22:in `from_file'
  /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb:26:in `from_file'

Relevant File Content:
----------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/helpers/settings_helper.rb:

 98:
 99:    def from_file(_file_path)
100:      # Throw errors for unrecognized top level calls (usually spelling mistakes)
101:      config_strict_mode true
102:      # Turn on node deprecation messages
103:      Gitlab::Deprecations::NodeAttribute.log_deprecations = true
104:      # Allow auto mash creation during from_file call
105>>     Gitlab::ConfigMash.auto_vivify { super }
106:    ensure
107:      config_strict_mode false
108:      Gitlab::Deprecations::NodeAttribute.log_deprecations = false
109:    end
110:
111:    # Enhance set so strict mode errors aren't thrown as long as the setting is witin our defined config
112:    def internal_set(symbol, value)
113:      if configuration.key?(symbol)
114:        configuration[symbol] = value

System Info:
------------
chef_version=14.14.29
platform=ubuntu
platform_version=16.04
ruby=ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/chef-client
executable=/opt/gitlab/embedded/bin/chef-client

Running handlers:
There was an error running gitlab-ctl reconfigure:

(<unknown>): did not find expected key while parsing a block mapping at line 2 column 5

Running handlers complete
Chef Client failed. 0 resources updated in 01 seconds

```

# Answer
While someone with knowledge about the innards of GitLab may have known what the error pointed to, it took me some time and bisecting my revision history to find the root cause.

I had introduced a Jinja2 conditional like this in the inlined YAML for LDAP servers:

```
gitlab_rails['ldap_servers'] = YAML.load <<-EOS # remember to close this block with 'EOS' below
main: # 'main' is the GitLab 'provider ID' of this LDAP server
  somekey: somevalue
  {% if some_variable is defined %}
  key: value
  {% endif %}
EOS

```

The `# remember to close this block with 'EOS' below` remark isn't the only caveat you have to worry about here. Indentation is too, as I found out.

Provided the condition became true, the about would have evaluated to:

```
gitlab_rails['ldap_servers'] = YAML.load <<-EOS # remember to close this block with 'EOS' below
main: # 'main' is the GitLab 'provider ID' of this LDAP server
  somekey: somevalue
    key: value
  EOS

```

... where the additional indentation tripped up the YAML parser and caused the unspecific error symptoms (and message).

Hope this helps someone else to avoid a lengthy analysis of the cause, if they run into a similar condition.
> 1 votes

---
Tags: gitlab
---


# Question
Title: What is a gating continuous integration (CI) system?
I often find myself in the position of mentioning gating CI systems and I thought it would be good to have a post I can easily reference explaining what that is. So here goes:

What is a gating CI system and how is it different from the traditional CI systems (i.e. the majority of the CI systems/tools presently out there)?

# Answer
A traditional CI system is **reactive**, it **detects regressions**, but requires human intervention for repairs.

Traditional CI relies on automated quality checks performed on the already integrated code - **after** changes are committed/merged into the integration branch (typically triggered by those events). The results of these checks indicate if the integration was successful, i.e. if the branch at that point is a good candidate for any subsequent steps towards delivery/deployment or if regressions occurred. Some regressions are considered catastrophic (for example build failures) and must be repaired ASAP as they're effectively blocking development on the branch. Unfortunately in most cases the repairs require human intervention, which can be a significant bottleneck.

By contrast a gating CI system is **proactive**, it **prevents regressions** and, with proper orchestration, can be entirely automated.

Gating CI performs the quality checks on **candidate** changes **prior** to them being committed/merged into the integration branch. If the checks are successful the changes are committed/merged into the branch, ideally in an automated manner, performed by the CI system itself, without human intervention. If the checks reveal regressions considered catastrophic/blocking the candidates are rejected (i.e. their commit/merge into the branch is prevented). Hence their **gating** characteristic - they will prevent "bad" changes from even reaching the branch.

A simple example would be a pull request (PR) based integration with one or more automated validations (for example a build and maybe a smoketest) hooked up in the PR processing which must be successful for the PR to be merged. A fairly common practice for many development teams.

Unfortunately rejecting "bad" changes alone is not sufficient to ensure the integration branch maintains a minimum quality level at all times.

It is always possible to have multiple candidates outstanding at the same time. But having multiple quality checks (on different candidates) active at the same time means that at least some of the checks do not take into account the other candidates in flight, leaving room for regressions due to changes interfering with each-other - see a simple example in How can successfully pre-verified changes cause regressions that should have been caught?.

It is for this reason that those using a PR-based system as mentioned above can probably recall that the branch can still see breakages once in a while.

This can be addressed with proper **orchestration** of the quality checks. The purpose of the orchestration is to completely eliminate the risk of interference between changes by controlling when each check is started/stopped and what candidate change(s) it covers, based on one or more inputs which can include: other candidates submissions/current states, results of other checks, actual commits/merges, availability of resources necessary for the checks, etc.

A very basic orchestration algorithm for the PR-based CI system mentioned above would be, for example, to only allow the merge if the tip of the branch at merge time is the same as it was when the automated PR validation started (i.e. no other change was committed/merged in the mean time). Effectively serializing PR processing, it could work great for small teams, but probably will not be able to keep up in a large scale project.

The orchestration algorithm (or lack thereof) is ultimately what dictates a gated CI system's scalability, performance ability to maintain the integration branch' quality level. An algorithm 100% efficient in preventing blocking regressions would no longer require human intervention for normal operation, a gated CI system using such algorithm can typically be entirely automated - a pipeline accepting change candidates at one end, filtering out regression-causing changes on a mid tap and outputting guaranteed stable branch tags at the other end.

It's probably worth mentioning that using a traditional or a gating CI only covers the integration stage and brings no fundamental restrictions on the CD (delivery/deployment) portion of a CI/CD pipeline - any traditional CI/CD tool can be plugged in behind a gated CI to perform additional (but non-gating) CI verifications (if needed) and the CD steps. Similarly any release strategy (see Should I use release branch to push my changes?) can be used with either of them.

**Note**: for distributed version control systems like `git` the context of the answer is the master/origin/central repository, not the local replicas.
> 3 votes

---
Tags: continuous-integration, terminology
---


# Question
Title: Is running docker-compose not supported in BitBucket?
**Aim**

The aim is to run docker-compose in BitBucket pipelines

**Attempt**

*bitbucket-pipelines.yml*

```
image: docker/compose:1.12.0

pipelines:
  default:
    - step:
        script:
          - docker-compose up --build

```

**Outcome**

```
The Docker Engine version is less than the minimum required by Compose.
Your current project requires a Docker Engine of version 1.13.0 or greater.

```

**Discussion**

According to this post it is not supported?

# Answer
> 11 votes
Here is my solution: (bitbucket-pipelines.yml)

```
pipelines:
  default:
    - step:
        script:
          - export DOCKER_COMPOSE_VERSION=1.18.0
          - export DOCKER_COMPOSE_URL=https://github.com/docker/compose/releases/download/${DOCKER_COMPOSE_VERSION}/docker-compose-$(uname -s)-$(uname -m)
          - curl -L $DOCKER_COMPOSE_URL > docker-compose
          - chmod +x docker-compose
          - mv docker-compose /usr/local/bin
          - docker-compose build --force-rm --no-cache --pull

```

# Answer
> 6 votes
We are less likely to see support of native docker-compose integration because pipelines API is a custom implementation of similar functionality, and also because possible security issues, but looks like more features are currently in development to implement more docker-compose-like features in this open issue.

UPDATE: added support for multi-container pipelines, as described in documentation

# Answer
> 3 votes
I managed to use docker-compose to push images to a remote host by using the following image.

tmaier/docker-compose:latest

This is the basic docker image with docker-compose installed.

My `bitbucket-pipelines.yml` looks like this:

```
- step:
    image: tmaier/docker-compose:latest
    script:
        - (umask  077 ; echo $DOCKER_PRIVATE_KEY | base64 -d > ./keys/key.pem)
        - export DOCKER_CERT_PATH=./keys/
        - export DOCKER_TLS_VERIFY="1"
        - export DOCKER_HOST=tcp://<DOCKER_HOST_IP>:2376
        - docker login --username $DOCKER_HUB_USERNAME --password $DOCKER_HUB_PASSWORD
        - docker-compose up -d <service>

```

I have checked-in in my repo the `ca.pem` and `cert.pem` needed by docker-compose in the `keys` directory. The private key is saved base64encoded, as a secure variable in Bitbucket pipelines environment variables, so I just decode it and paste it to `keys/key.pem` in the pipeline as the first step.

Keep in mind that i used `docker login` after setting the enviroment variables for the remote host.

# Answer
> 2 votes
**bitbucket-pipelines.yml**

```
image: python:3.8.1

pipelines:
  branches:
    "**":
      - step:
          name: Build
          services:
            - docker
          caches:
            - docker
            - pip
          script:
            - pip install docker-compose
            - docker network create dockernet
            - docker-compose build

definitions:
  services:
    docker:
      memory: 2048

```

---
Tags: bitbucket, docker-compose
---


# Question
Title: How can I test native macOS UI software in a container before deployment?
As part of the CI setup, I'd like to run (automated) tests on native macOS GUI software in a 'safe environment' (i.e. without messing with the host software installation too much). Virtual machines (using VirtualBox) work in principle, but they are fairly slow to start/stop and VirtualBox on macOS hosts doesn't seem to be very stable (it works great on Linux hosts though).

After reading macOS testing on Desktop: Docker containers vs. Homebrew installation I learned that using Docker works great for testing software on macOS. However, my understanding is that even with Docker Desktop for Mac (which uses Hyperkit to virtualize the OS as opposed to running VMs using VirtualBox), the software running in the container sees a Linux installation.

Would it be plausible to use Docker anyway (maybe I misunderstood it?) such that the containerized program sees a macOS installation or would some other solution (A plain `chroot`?) work better here?

# Answer
> 1 votes
I can confirm that running Docker on Mac works just fine, but according to this ServerFault question, it's not possible to just run MacOS in Docker.

(You might be able to run MacOS in a VM in a Docker container, but that's probably not what you want.)

You could purchase a MacMini, use it as a Jenkins slave and run automated tests on it as part of your build pipeline or something like that?

# Answer
> 0 votes
Back in the day I attended a meetup and they had a Mac Mini for testing Apple related stuff.

At my previous job we had docker images and these were run on Linux, Mac and Windows. The Windows docker environments were less stable, but it worked well on Linux and Mac. All developers were using a Mac and they used docker on a daily basis.

In conclusion, from a cost and DevOps perspective I would prefer using docker over using a Mac Mini as the mac mini will not be used in production will the same docker image should be used in dev, test, acc and prod (DTAP).

---
Tags: continuous-integration, containers, testing, virtualization, dtap
---


title: Minimal CMake Jenkinsfile failing

text:
# Question
Title: Minimal CMake Jenkinsfile failing
Why is this Jenkins build failing? How can I write a Jenkins file just to test the building of this simple project?

*DisplayImage.cpp*

```
#include <stdio.h>
#include <opencv2/opencv.hpp>

using namespace cv;

int main(int argc, char** argv )
{
    if ( argc != 2 )
    {
        printf("usage: DisplayImage.out <Image_Path>\n");
        return -1;
    }

    Mat image;
    image = imread( argv[1], 1 );

    if ( !image.data )
    {
        printf("No image data \n");
        return -1;
    }
    namedWindow("Display Image", WINDOW_AUTOSIZE );
    imshow("Display Image", image);

    waitKey(0);

    return 0;
}

```

*CMakeLists.txt*

```
cmake_minimum_required(VERSION 2.8)
project( DisplayImage )
find_package( OpenCV REQUIRED )
add_executable( DisplayImage DisplayImage.cpp )
target_link_libraries( DisplayImage ${OpenCV_LIBS} )

```

*Jenkinsfile*

```
pipeline {
    agent any
    stages {
        stage('build') {
            steps {
                cmakeBuild
                installation: 'InSearchPath'
            }
        }
    }
}

```

Output from Jenkins

```
WorkflowScript: 6: Expected a step @ line 6, column 17.
                   cmakeBuild
                   ^

WorkflowScript: 7: Expected a step @ line 7, column 31.
                   installation: 'InSearchPath'

```

# Answer
> 3 votes
Verify that you have the correct CMake Plugin installed. Also, add parenthesis to your cmakeBuild statement in your pipeline

```
pipeline {
  agent any
  stages {
    stage('build') {
      steps {
          cmakeBuild(
            installation: 'InSearchPath'
          )
      }
    }
  }
}

```

---
Tags: jenkins
---


# Question
Title: How to define Service Level Objectives without historical data?
Assuming I can define Service Level Indicators for a user journey are there any structured methods for setting a Service Level Objective within that SLA without any historical data to indicate what is realistic or feasible?

For **Availability** we can often turn to various mathematical approaches that enable us to understand the maximum theoretical availability of the system, although even this is tied back to historical data in the form of Mean Time between Failure (MTBF) and Mean Time to Recover (MTTR).

# Answer
Without much data you are really guessing, the general thought I have heard before is to start at 80% SLO then work your way up as you have data to back up your goal. There really is not set way of figuring out a SLO without some data. Even taking and using a week's data would be good enough to give you a reasonable idea.
> 1 votes

# Answer
I think the short answer is: You shouldn't define **SLOs**, if you don't have historical data.

With that being said, **SLIs** can be defined without historical data and correlated with KPIs and support metrics.

Here is a concrete example:

* Hipster store is about to launch.
* Product management, dev and SRE wants to define SLOs.
* They should start by defining SLIs, e.g site availability, sign-up success rate, checkout success rate.
* PM/execs should agree to some kind of 'soft' launch and gradual rollout (if at all possible), e.g internal to employees -\> partners -\> single state/coast/country.
* As the 'soft' and gradual rollout progresses, they can correlate swings in the SLIs with changes to KPIs (e.g revenue) and support metrics.

At this point, the hipster store leadership would have some information about how much customer unhappiness/pain (e.g support metrics) and KPIs (e.g revenue) are correlated with SLIs. The SLOs are the bars. One or multiple can be set and alerts defined around them. Rinse and repeat.

A follow-up question is: How to alert and what to alert on before SLOs are set? I'd take the following article in consideration - https://www.oreilly.com/ideas/reduce-toil-through-better-alerting.
> 0 votes

---
Tags: monitoring, sre, operations
---


# Question
Title: Azure DevOps - how to create a 2nd Microsoft-hosted Agent Pool
I am using Azure DevOps.

I have 5 Microsoft-hosted agents. And 10 pipelines. One pipeline in particular can get a bit grabby, so I want to restrict it to use just 2 agents.

But I want my other 9 pipelines to have access to the 3 remaining agents.

I figure Agent Pools is the right way to solve this. By default, Azure DevOps sets us up with 1 single "Azure Pipelines" pool with our self-hosted agents. I would like to split this into 2 pools.

However, when I go to "create new pool" I only get the option to add "self-hosted" agents.

How do I create a new Agent Pool with some of my existing Microsoft-hosted agents?

# Answer
> 1 votes
This is a good question because you would assume you could do what you are after, but as far as I know you can't yet. From the docs on sharing of parallel jobs across projects in a collection

> Currently, there isn't a way to partition or dedicate parallel job capacity to a specific project or agent pool.

I considered trying to do buckets via capabilities on a subset of my agents, and then companion demands in the pipelines, but that will not work either because hosted pipelines don't work with demands.

In my case we had some greedy pipelines that fan out into multiple parallel jobs, that we had to update to run sequentially with `dependsOn` for each job. I think if your case is relatively simple you could potentially check the queue length or number of running builds in a first stage but that could get complex and might be tough to scale.

Hopefully this feature gets added soon. I will look to see if I can find one to vote up on the developer community forums. I am also interested in other ideas on how to solve for this issue!

Edit found a feature vote on it here

---
Tags: azure-devops, azure-pipelines
---


# Question
Title: Is it possible to force to false or true the App.config transformation with arguments on Azure DevOps Pipeline?
Is it possible to force the TransformOnBuild to false using MSBuild parameters in Azure DevOps?

So I Use Azure DevOps Pipeline to build a clickOnce application for a staging environment

Here is my MSBuild Arguments

```
/target:publish /p:ApplicationVersion=$(Build.BuildNumber) /p:InstallURL=http://app-staging.example.com/ /p:PublishURL=http://app-staging.example.com/ /p:UpdateEnabled=true  /p:UpdateMode=Foreground  /p:ProductName="App Staging" /p:OutputPath="$(build.ArtifactStagingDirectory)\Publish\\"

```

Is it possible to force to false or true the App.config transformation with arguments?

# Answer
> 1 votes
Short answer, yes.

Azure Pipelines supports conditional steps. Find these under `Control Options` in any build step.

**How** you build your custom conditions will depend on what triggers your step to true/false.

The screenshot has an example of how we control a build step based on a build variable (defined in the `variables` tab of the pipeline).

---
Tags: azure-devops, azure-pipelines
---


title: Worker Node in Kubernetes

text:
# Question
Title: Worker Node in Kubernetes
I am new to Kubernetes. I have followed one document and setup master node in Kubernetes. Now I want to create a Worker node, so How can I create it? I don't know anything about to set up a worker node.

# Answer
If you are running on VMs or outside of a cloud provider then manually setting this up involves several steps. The tool you will need is kubeadm (see documentation linked) and you will need to init the nodes and then `kubeadm join` the nodes to the master.

If you are using a cloud provider, then follow the providers documentation on how to add additional nodes.

A great guide called "Kubernetes The Hard Way" is another good resource for learning how to setup a cluster without any automated tooling.
> 1 votes

---
Tags: kubernetes
---


# Question
Title: Building Jenkins pipeline with GitLab info
I am new to DevOps world and desperately looking for your help. We have multiple API depositors in GitLab subgroup, I am trying to establish a CI/CD pipeline, which will connect to GitLab subgroup,

1. Any new API added into the subgroup, the pipeline will be triggered.
2. If existing API code is updated into any branch pipeline should be triggered.

In GIT repository, I have to provide the API URL, should I write #of pipelines as number API?

# Answer
> 1 votes
Gitlab Subgroups are essentially only additional path identifiers on the Gitlab server.

For example if you have:

* Gitlab server on `localhost`
* Group: `test`
* Subgroup `subgroup1`
* Repository `myawesomerepo`

Then the path to your repository will be:

```
http://localhost/test/subgroup1/myawesomerepo

```

And cloning the repo will be

```
git clone git@localhost/test/subgroup1/myawesomerepo.git

```

So when you connect any Jenkins pipelines to the repo you will need to connect them to that full path.

Any changes being done or API's added to the Git repository itself can easily be monitored automatically to do some action via Jenkins.

However, since a Gitlab Subgroup is not a Git feature rather a Gitlab feature, you need to try and write you own logic to query on the full URL and to see if there were any new Subgroups/ API endpoints added.

---
Tags: continuous-integration, jenkins-pipeline, gitlab
---


# Question
Title: Issue getting HTTPS working for subdomain despite "Certificate issued successfully"
Part of my struggle to learn this, I think, is that `cert-manager` is updated pretty regularly so a lot of the tutorials, blog posts, questions, and courses I have read or taken are no longer relevant.

For example, when I initially started learning Kubernetes you had to make `certificate.yaml` that looked similar to this:

```
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
  name: test-domain-com
spec:
  secretName: test-domain-com
  issuerRef:
    name: letsencrypt-staging
    kind: ClusterIssuer
  commonName: test.domain.com
  dnsNames:
    - test.domain.com
  acme:
    config:
      - http01:
          ingressClass: nginx
        domains:
          - test.domain.com

```

This doesn't appear to be the case anymore since it seems to be extrapolated from the `issuer.yaml` and `ingress.yaml`.

Anyway, I have a certificate that "issued successfully", but still HTTPS is not working on the application when I navigate to it.

**issuer.yaml**

```
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
  name: test-domain-com
spec:
  secretName: test-domain-com
  issuerRef:
    name: letsencrypt-staging
    kind: ClusterIssuer
  commonName: test.domain.com
  dnsNames:
    - test.domain.com
  acme:
    config:
      - http01:
          ingressClass: nginx
        domains:
          - test.domain.com

```

**ingress.yaml**

```
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/add-base-url: "true"
    nginx.ingress.kubernetes.io/rewrite-target: /$1
    cert-manager.io/cluster-issuer: "letsencrypt-staging"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
  name: ingress
  namespace: default
spec:
  tls:
    - hosts:
        - test.domain.com
      secretName: test-domain-com
  rules:
    - host: test.domain.com
      http:
        paths:
          - path: /?(.*)
            backend:
              serviceName: client-cluster-ip-service
              servicePort: 3000
          - path: /api/?(.*)
            backend:
              serviceName: api-cluster-ip-service
              servicePort: 5000

```

**Certificate Logs**

```
$ kubectl describe certificates test-domain-com
Name:         test-domain-com
Namespace:    default
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"cert-manager.io/v1alpha2","kind":"Certificate","metadata":{"annotations":{},"name":"test-domain-com","namespace":"de...
API Version:  cert-manager.io/v1alpha2
Kind:         Certificate
Metadata:
  Creation Timestamp:  2020-01-30T17:49:12Z
  Generation:          6
  Owner References:
    API Version:           extensions/v1beta1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Ingress
    Name:                  ingress
    UID:                   d31883d9-4388-11ea-84ec-469fd0fd8fec
  Resource Version:        832245
  Self Link:               /apis/cert-manager.io/v1alpha2/namespaces/default/certificates/test-domain-com
  UID:                     d3222905-4388-11ea-84ec-469fd0fd8fec
Spec:
  Dns Names:
    test.domain.com
  Issuer Ref:
    Group:      cert-manager.io
    Kind:       ClusterIssuer
    Name:       letsencrypt-staging
  Secret Name:  test-domain-com
Status:
  Conditions:
    Last Transition Time:  2020-01-30T18:12:41Z
    Message:               Certificate is up to date and has not expired
    Reason:                Ready
    Status:                True
    Type:                  Ready
  Not After:               2020-04-29T17:12:40Z
Events:
  Type    Reason     Age   From          Message
  ----    ------     ----  ----          -------
  Normal  Requested  43m   cert-manager  Created new CertificateRequest resource "test-domain-com-566504257"
  Normal  Issued     42m   cert-manager  Certificate issued successfully

```

**Describe Secret**

```
$ kubectl describe secret test-domain-com
Name:         test-domain-com
Namespace:    default
Labels:       <none>
Annotations:  cert-manager.io/alt-names: test.domain.com
              cert-manager.io/certificate-name: test-domain-com
              cert-manager.io/common-name: test.domain.com
              cert-manager.io/ip-sans:
              cert-manager.io/issuer-kind: ClusterIssuer
              cert-manager.io/issuer-name: letsencrypt-staging
              cert-manager.io/uri-sans:

Type:  kubernetes.io/tls

Data
====
ca.crt:   0 bytes
tls.crt:  3566 bytes
tls.key:  1675 bytes

```

`ca.crt` is empty and not entirely sure if it should or shouldn't be. Looks like there is a Github issue about it.

**Errors in `cert-manager` log**

```
W0130 19:13:11.357728       1 client_config.go:543] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
E0130 19:14:21.811114       1 controller.go:230] cert-manager/controller/webhook-bootstrap/webhook-bootstrap/ca-secret "msg"="error decoding CA private key" "error"="error decoding private key PEM block" "resource_kind"="Secret" "resource_name"="cert-manager-webhook-tls" "resource_namespace"="cert-manager"
E0130 19:14:21.811257       1 controller.go:131] cert-manager/controller/webhook-bootstrap "msg"="re-queuing item  due to error processing" "error"="error decoding private key PEM block" "key"="cert-manager/cert-manager-webhook-tls"
rce_kind"="Challenge" "resource_name"="test-domain-com-566504257-1006673842-2382236481" "resource_namespace"="default" "type"="http-01"
E0130 19:16:21.262484       1 sync.go:184] cert-manager/controller/challenges "msg"="propagation check failed" "error"="wrong status code '404', expected '200'" "dnsName"="test.domain.com" "resource_kind"="Challenge" "resource_name"="test-domain-com-566504257-1006673842-2382236481" "resource_namespace"="default" "type"="http-01"
E0130 19:16:42.616965       1 controller.go:156] ingress 'default/cm-acme-http-solver-6xj55' in work queue no longer exists
E0130 19:16:42.837328       1 sync.go:59] cert-manager/controller/orders "msg"="failed to update status" "error"=null "resource_kind"="Order" "resource_name"="test-domain-com-566504257-1006673842" "resource_namespace"="default"
E0130 19:16:42.837364       1 controller.go:131] cert-manager/controller/orders "msg"="re-queuing item  due to error processing" "error"="Operation cannot be fulfilled on orders.acme.cert-manager.io \"test-domain-com-566504257-1006673842\": the object has been modified; please apply your changes to the latest version and try again" "key"="default/test-domain-com-566504257-1006673842"
E0130 19:16:43.687987       1 controller.go:193] cert-manager/controller/challenges "msg"="challenge in work queue no longer exists" "error"="challenge.acme.cert-manager.io \"test-domain-com-566504257-1006673842-2382236481\" not found"

```

The only thing that jumped out to me was:

`"msg"="error decoding CA private key" "error"="error decoding private key PEM block"`

Tried to dumping browser cache, incognito mode, etc... still comes up HTTP or "Not Secure" with "Fake LE Intermediate X1" certificate when navigating to HTTPS.

So what did I screwup/overlook this time and how do I resolve this?

# Answer
> 0 votes
Ok figured it out. It was due to using `staging` like so:

```
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
  namespace: cert-manager
spec:
  acme:
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    email: <email>
    privateKeySecretRef:
      name: letsencrypt-staging
    solvers:
      - http01:
          ingress:
            class: nginx

```

Changed it `prod` like so and that cleared up the issue:

```
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
  namespace: cert-manager
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: <email>
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
      - http01:
          ingress:
            class: nginx

```

Apparently that is the expected behavior when using `staging` according to this well written and accurate tutorial.

That being said I didn't initially think it worked. Even after a hard reload and empty cache + hard reload, HTTPS still wasn't working. Tried incognito and it worked, so closed all my browser instances, reopened, and HTTPS worked.

And also, `certificate.yaml` was not necessary at all.

---
Tags: kubernetes, ingress, cert-manager
---


# Question
Title: Applying manifests in Azure DevOps pipeline
Basically, I want the pipeline to `apply` some manifests before deploying images from ACR. No images need to be made. It would just be things like applying the `issuer.yaml`, `ingress.yaml`, and maybe a few other things that need to be run before the `client` and `api` images are pulled from ACR.

Was looking at the documentation here:

https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/kubernetes-manifest?view=azure-devops#overview

But just not really seeing what I need.

Suggestions for how to do this?

# Answer
Once again, come across the answer right after asking the question.

Needed to look at the `Kubectl task` and not the `Kubernetes manifest task`. Particularly the Commands section:

```
- task: Kubernetes@1
  displayName: kubectl apply using arguments
  inputs:
    connectionType: Azure Resource Manager
    azureSubscriptionEndpoint: $(azureSubscriptionEndpoint)
    azureResourceGroup: $(azureResourceGroup)
    kubernetesCluster: $(kubernetesCluster)
    command: apply
    arguments: -f mhc-aks.yaml

```
> 0 votes

---
Tags: azure-devops, azure, azure-pipelines, azure-kubernetes-service
---


title: Ansible - Copy multiple files

text:
# Question
Title: Ansible - Copy multiple files
I am trying to copy multiple files to the target like so:

```
  tasks:
  - name: "Copying files"
    copy:
      src: files/{{ item }}
      dest: /my/folder/
    with_items:
      - file.txt

```

The files directory is in the same directory as the Ansible playbook. However when running the script, I get this error:

```
docker: TASK [Gathering Facts] *********************************************************
    docker: fatal: [test]: FAILED! => {"msg": "failed to transfer file to /home/user/.ansible/tmp/ansible-local-123331or28t2a/tmp6st_fxv1 ~user/.ansible/tmp/ansible-tmp-1579990845.0734024-189911264147319/AnsiballZ_setup.py:\n\n\n"}
    docker:

```

Why is that? Why can Ansible not copy the files to the target machine?

# Answer
> 1 votes
Following you comment: no I cannot reproduce your problem. Since you did not provide the rest of your files (inventory, etc...) and I cannot guess where is your problem, here is a quick proof by example using the `php:7.0-apache` image.

* The inventory (`inventories/docker_test/hosts.yml`)

```
---
all:
  hosts:
    docker_test:
      ansible_connection: docker

```
* Launching the test container

```
docker run -d --rm --name docker_test php:7.0-apache

```
* The test `playbook.yml`

```
---
- hosts: docker_test
  gather_facts: false

  tasks:
    # Your image does not have python which is mandatory for ansible
    # Please note this is for demo only. You should get python installed
    # by extending the image through a Dockerfile and build it.
    - name: Dirty low-level command to get python installed
      raw: apt-get update && apt-get install -y python3

    - name: Gather facts now we can
      setup:

    - name: Copy a dummy file
      copy:
        dest: /etc/apache2/sites-available/dummy.conf
        content: "# I'm a dummy file"

```
* Playbook run and output

```
$ ansible-playbook -i inventories/docker_test/ playbook.yml

PLAY [docker_test] ********************************************************************************************************************************************************************************************************

TASK [Dirty low-level command to get python installed] ********************************************************************************************************************************************************************
changed: [docker_test]

TASK [Gather facts now we can] ********************************************************************************************************************************************************************************************
[WARNING]: Platform linux on host docker_test is using the discovered Python interpreter at /usr/bin/python3.5, but future installation of another Python interpreter could change this. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information.

ok: [docker_test]

TASK [Copy a dummy file] **************************************************************************************************************************************************************************************************
changed: [docker_test]

PLAY RECAP ****************************************************************************************************************************************************************************************************************
docker_test                : ok=3    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

```

# Answer
> -1 votes
Use the following source code for copy multiple files on your client machine.

```
---
  - name: Copy data to the client machine
    hosts: hostname
    become_method: sudo
    become_user: root
    become: true
    tasks:
      # Copy twice as sometimes files get skipped (mostly only one file skipped from a folder if the folder does not exist)
      - name: Copy UFO-Server
        copy:
          src: "source files path"
          dest: "destination file path"
          owner: root
          group: root
          mode: 0644
          backup: yes
        ignore_errors: true

```

Note:

1. If you are passing multiple paths by using variable then

src: "/root/{{ item }}"
2. If you are passing path by using a variable for different items then

src: "/root/{{ item.source\_path }}"

---
Tags: docker, ansible, provisioning
---


# Question
Title: Is the root user in a Docker image/container the same as the root user of the machine?
Is the root user in a Docker image/container the same as the root user of the machine? I am curious about the security implications of using the root user in the Docker image.

TMK, the container is run by a non-root user (often the user is called "docker") and that user launched the docker-dameon. And the docker daemon launches the container process.

# Answer
> 3 votes
No the root in container is a root-user of container. Remember containers runs isolated. DOcker uses cgroups and libvirt of kernel linux.

---
Tags: docker, docker-daemon
---


title: Git-based CI/CD Pipeline for Monorepo

text:
# Question
Title: Git-based CI/CD Pipeline for Monorepo
Are there established CI/CD strategies & best practices for working with a distributed team on a project that is utilizing monorepos?

I can provide two specific use-cases:

1. Working on content distribution, e.g. a daily/weekly HTML email newsletter, and
2. A multi-project directory for building LaTeX documents

Both of these scenarios will utilize the same CI/CD pipeline (build/send an email, compile and upload a LaTeX document) but with a growing & variable amount of content that makes independent repos per document undesirable.

Most of the projects using monorepos currently seem to be javascript projects with a lot of cross-dependencies on code. This is a step a way from that, but I'd still like to embrace the audit trail of version control with an automated development process for producing content.

Do these projects compile every piece of code on change? Is there a way to embrace a git diff or use some sort of 'flag' file in the repo root to use CI/CD tools with a monorepo?

# Answer
The fact that all the code is in a single repository doesn't mean that all the code is changing every time a commit is pushed. I would first make the "pathways" in the code explicit. E.g. perhaps you have a few subdirectories:

```
- App1/
  - Docs/
  - Code/
  - Tests/
  - Makefile
- App2/
  - Docs/
  - Code/
  - Tests/
  - Makefile

```

Assuming that App1 is independent from App2, you could trigger it's build by checking if anything in that directory changed (see https://stackoverflow.com/questions/424071/how-to-list-all-the-files-in-a-commit).

Most continuous integration tools have a git plugin or native feature that allows you to filter on a path. If you were using Jenkins, you could set up a few jobs, each with the included region to be be the pathway that should be triggered -- so, *e.g.* changes to `App1/*` would trigger `jobs/app1/` which executes `make -f App1/Makefile` or similar.

The trick then to maintaining this project is really in the Makefiles, or other build tool of your choice. You would write the dependencies in there.

You could also have a single global Makefile and use the `git diff-tree` trick in the question mentioned above:

```
if grep -q App1 `git diff-tree --no-commit-id --name-only -r <commit>` ; then
  # trigger pipeline App1
  cd App1
  make build
  make test
  make publish
fi

```
> 3 votes

# Answer
I created some showcase for monorepo with Gradle as build tool and CircleCI or Bitbucket pipelines as CI tool.

It's based on the same ideas as in accepted answer and I used similar setup in two projects till now.

See: https://github.com/zladovan/monorepo

Gradle specific stuff should be replaceable quite easily by something else. Just change `tools/ci/core/list-dependencies.sh`.
> 1 votes

---
Tags: continuous-integration, git, travis-ci, gitlab-ci-runner
---


# Question
Title: How to pass parameters to a container orchestrated by Kubernetes similar to running the container at the command line using the docker run command?
**Background**

I have a docker container that I run in the following fashion:

`docker run --rm my-custom-container:latest --username me --password yes --url "https://my.url"`

The container does what what it needs to do with no issues.

I have pushed my container image to a private image repository, and I am able to create a Kubernetes deployment. Kubernetes is able to pull the container from the private image repository for the container with no problems.

**Problem**

However, when Kubernetes goes to start the container it returns with an error stating that it cannot run the container.

This is due to a lack of parameters being passed to the container to run.

**Question**

How do I pass parameters to my container during a Kubernetes deployment?

# Answer
You can include the arguments in the deployment yaml. The official docs include examples and further customization.

```
spec:
   containers:
     image: myimage
     name: myContainerName
     args: ["me", "yes", "https://my.url"]
```
> 2 votes

---
Tags: docker, kubernetes, containers, configuration, orchestration
---


# Question
Title: Pipeline flow get stuck at “Notifying stash” after finishing all stages
All our CI projects are getting stuck in the Notify stash. eventually, it's passing, but we're trying to understand why it takes so much time. example:

```
17:03:01 Notifying Stash at "https://bitbucket"
17:15:44 Notified Stash for commit with id 925ceb567087f94c5b9d44d1330995860f11d1e5

```

what can cause such a delay? Bitbucket (Stash) Notifier Plugin version: 1.4.0

# Answer
> 0 votes
We removed the OCP plugin configuration from Jenkins

---
Tags: jenkins-pipeline, jenkins-plugins, bitbucket
---


# Question
Title: Is it possible to give only view permission for Jenkins monitoring-plugin
We have the monitoring-plugin installed: https://github.com/jenkinsci/monitoring-plugin

is there a way to give only view permissions once it is installed?

# Answer
No :). please see Jenkins issue 14750: https://issues.jenkins-ci.org/browse/JENKINS-14750
> 0 votes

---
Tags: jenkins-plugins
---


# Question
Title: Sonar Quality Gate behavior in Jenkins pipeline is unstable
I have some pipelines in Jenkins, with the SonarQube stage (Version: 6.7.1, build 35068) In some builds it is aborted due to time out:

```
12:19:01 Sleeping for 3 min 0 sec
[Pipeline] timeout
12:22:10 Timeout set to expire in 25 min
[Pipeline] waitForQualityGate
12:22:10 Checking status of SonarQube task 'XXXXXXXXXX' on server 'Sonar'
12:22:10 SonarQube task 'XXXXXXXXXX' status is 'PENDING'
12:47:10 Cancelling nested steps due to timeout

```

and in some other builds it pass:

```
13:56:47 Sleeping for 3 min 0 sec
[Pipeline] timeout
13:59:47 Timeout set to expire in 25 min
[Pipeline] {
[Pipeline] waitForQualityGate
13:59:47 Checking status of SonarQube task 'XXXXXXXXXX' on server 'Sonar'
13:59:47 SonarQube task 'XXXXXXXXXXXX' status is 'SUCCESS'
13:59:47 SonarQube task 'XXXXXXXXXXXX' completed. Quality gate is 'WARN'

```

Trigger command in bitbucket file:

```
    stage('Sonar Quality Gate') {
        sleep 180
        timeout(time: 25, unit: 'MINUTES') {
            def qg = waitForQualityGate()
                    if (!(qg.status == 'OK' || qg.status == 'WARN')) {
                        error "Pipeline aborted due to quality gate failure: ${qg.status}"
                    }
        }
    }

```

what might be the reason for it? how can I keep it stable? can it be network isssue?

# Answer
It was related at the end to Sonar DB performance (after getting AWR reports): https://stackoverflow.com/questions/59982345/how-to-maintain-cleanup-of-the-sonar-table-project-measures
> 0 votes

---
Tags: jenkins-pipeline, jenkins-plugins, sonarqube
---


# Question
Title: How to exclude a node in kubernets using podAntiAffinity?
I'm trying to exclude a set of nodes when I submit a job in kubernetes. The set of nodes do not (yet) have a label to differentiate them, so all I can exclude them on is hostname (for now).

All the examples on using podAntiAffinity use `labelSelector` which expects a label on the pod to differentiate it.

How can I limit my job from deploying to a specific set of nodes? The closest example I could scrape together from the docs is this non working example:

```
affinity:
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
    - labelSelector:               <--- this is wrong, there are no labels
      matchExpressions:
      - key: "kubernetes.io/hostname"
        operator: In
        values:
          - hostname1
          - hostname2

```

# Answer
> To exclude a set of nodes when submitting a job in kubernetes.

you should use `Node affinity` which is conceptually similar to `nodeSelector` –and will allow you to constrain which nodes your pod is eligible to be scheduled on, based on labels on the node and you should be able to use hostname.

On below example i run a job to 5 completions and all are started on node-01 of my cluster with node affinity defined.

Example : job.yaml

```
apiVersion: batch/v1
kind: Job
metadata:
  creationTimestamp: null
  labels:
    run: busybox
  name: busybox
spec:
  completions: 5
  template:
    metadata:
      creationTimestamp: null
      labels:
        run: busybox
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/hostname
                operator: In
                values:
                - k8s-node01
      containers:
      - image: busybox
        name: busybox
        resources: {}
      restartPolicy: OnFailure

```

Result :

```
ubuntu@k8s-master:~$ kubectl get all -o wide
NAME                READY   STATUS      RESTARTS   AGE     IP               NODE         NOMINATED NODE   READINESS GATES
pod/busybox-2gdp6   0/1     Completed   0          3m20s   192.168.85.204   k8s-node01   <none>           <none>
pod/busybox-l654d   0/1     Completed   0          3m55s   192.168.85.201   k8s-node01   <none>           <none>
pod/busybox-n7dbr   0/1     Completed   0          3m29s   192.168.85.203   k8s-node01   <none>           <none>
pod/busybox-pfc79   0/1     Completed   0          4m11s   192.168.85.200   k8s-node01   <none>           <none>
pod/busybox-r4xln   0/1     Completed   0          3m16s   192.168.85.205   k8s-node01   <none>           <none>

NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE   SELECTOR
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   31h   <none>

NAME                COMPLETIONS   DURATION   AGE     CONTAINERS   IMAGES    SELECTOR
job.batch/busybox   5/5           60s        4m11s   busybox      busybox   controller-uid=e12f136b-d828-4f7b-a49b-303499496a8a

```

I had two worker nodes on my cluster on job yaml i defined affinity to node01 only.

```
$ kubectl get nodes
NAME         STATUS   ROLES    AGE   VERSION
k8s-master   Ready    master   31h   v1.17.2
k8s-node01   Ready    <none>   31h   v1.17.2
k8s-node02   Ready    <none>   31h   v1.17.2

```
> 1 votes

---
Tags: kubernetes
---


# Question
Title: monitor for docker stats similar to htop?
I'm in the need for this and my google leaded me to this post on SO

I guess it's helpful to boost the beta for our sister site DevOps Stackexchagne here.

---

I quote the original question

`docker stats` gives a good overview of the cpu/mem/net io/disk io as text.

Does there exist a `htop` like monitor that shows the docker stats?

# Answer
bcicen on githup with tool `ctop` is working for me ref. https://github.com/bcicen/ctop
> 4 votes

---
Tags: monitoring
---


# Question
Title: Dump Kubernetes log files for all services/pods/jobs etc
We are remotely deploying a kubernetes cluster and when the installation fails, we would like to see the logs of all the services and migrations and pods etc. that we are trying to install. Instead of manually specifying each specific service etc. Is there an easy way of getting all the logs of the cluster and saving them in a tar file.

# Answer
> 1 votes
You are possibly looking for a third party stack on cluster to help you, if so there is long list you can explore and play to your need

Various Kubernetes logging tools you could consider from below list:

> Fluentd
>
> ELK Stack
>
> Stackdriver

I have used this link to setup EFK on my local cluster.

---
Tags: kubernetes
---


# Question
Title: how to control pod termination speed in kubernetes
I have a k8s deployment that I update by doing `kubectl apply -f spec.yml`. I use the default update strategy.

Whenever I update, new pods get created and the old ones **eventually** get deleted.

This is what `kubectl get pods` looks like a few seconds after calling `apply`:

```
NAME                      READY     STATUS        RESTARTS   AGE
server-243710380-7s7g0    1/1       Terminating   0          9m
server-243710380-ffzv1    1/1       Terminating   0          9m
server-243710380-gfs3h    1/1       Terminating   0          9m
server-3561703442-24dxm   1/1       Running       0          13s
server-3561703442-7jlds   1/1       Running       0          8s
server-3561703442-x5xw1   1/1       Running       0          11s

```

As you can see, the ones in `Terminating` status hang around, typically for about 30 seconds, before really disappearing. This happens despite the new pods reaching `running` status within a few seconds.

How can I control the time the old pods remain in `terminating` status? Specifically, I want them to terminate as quickly as possible.

# Answer
> 8 votes
There is a default grace period for pod termination of 30 seconds. See: https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods.

You can customize the grace period setting `terminationGracePeriodSeconds` at the pod spec level.

# Answer
> 1 votes
> Note : It is one of the Kubernetes **best practices to allow termination with grace**.

Kubernetes waits for a specified time called the termination grace period. By default, this is 30 seconds. It’s important to note that this happens in parallel to the preStop hook and the SIGTERM signal.

> NOTE : Kubernetes does not wait for the preStop hook to finish.**If your app finishes shutting down and exits before the terminationGracePeriod is done, Kubernetes moves to the next step immediately**.

If your pod usually takes longer than 30 seconds to shut down, make sure you increase the grace period. You can do that by setting the **`terminationGracePeriodSeconds`** option in the Pod YAML.

> It’s important that your application **terminate gracefully so that there is minimal impact on the end user** and the time-to-recovery is as fast as possible!

To change it to value lower or higher than default value of 30 seconds include the flag as below example.

Example :

```
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  containers:
  - image: nginx
    name: nginx
  terminationGracePeriodSeconds: 0

```

We can always make use of `kubectl explain` command to find supported options under spec.

For example : `kubectl explain deployment.spec.template.spec`

```
master $ kubectl explain deployment.spec.template.spec.terminationGracePeriodSeconds
KIND:     Deployment
VERSION:  apps/v1

FIELD:    terminationGracePeriodSeconds <integer>

DESCRIPTION:
     Optional duration in seconds the pod needs to terminate gracefully. May be
     decreased in delete request. Value must be non-negative integer. The value
     zero indicates delete immediately. If this value is nil, the default grace
     period will be used instead. The grace period is the duration in seconds
     after the processes running in the pod are sent a termination signal and
     the time when the processes are forcibly halted with a kill signal. Set
     this value longer than the expected cleanup time for your process. Defaults
     to 30 seconds.

```

---
Tags: kubernetes
---


# Question
Title: User data in original windows ec2 instance does not get executed in custom ami
I created a windows Ec2 instance with user data in it. In my ec2 instance I downlaoded git and installed it. Later in EC2 instance, I provided it with the user data which was supposed to download a git repository and place it in C:

This is my user data in original EC2 instance.

```
 <powershell>
Remove-Item C:\c -Recurse -ErrorAction Ignore
C:\"Program Files"\Git\bin\git.exe config --global http.sslVerify false
C:\"Program Files"\Git\bin\git.exe clone https://github.com/(somename)/(someapp).git /c
</powershell>
<persist>true</persist>

```

The EC2 instance worked without any problem. Then I decided to create a custom AMI from it. I created one and launched another instance from it. I then tried to run the instance to which it ran succesfully. But then in the new EC2 instance that I created from custom AMI, the user data from the original EC2 instance was not executed. I tried changing the setting of EC2 instance which was found in C://programdata . I configured it to run user data on every boot. But then still user data was not executed. I am not sure what's going on here. Any help will be appreciated.

# Answer
## User Data Persistance

If I create an image with Packer or other tool, I'll include normally a userdata initialization script that would allow connecting via WinRM. This is required as the default AMI settings wouldn't allow the tooling to connect. In this scenario Packer would create the instance from an AMI with my provided userdata.

However, when I then take an AMI of this configured instance and try to use it as a new instance, this new image has it's own userdata. If I wanted to run the exact same command, I would need to create the userdata in this new instance.

Userdata isn't persisted in your AMI, it's more an instance level configuration value.

### Follow-Up Comments on This

The goal should be to create your AMI with the tooling, for example git, and then your new instance should not even need to run that command in the first place as you've "baked" it into your "golden ami". Consider this in your approach. Also consider decoupling any installation directly from userdata and use something like AWS Simple Systems Manager Command documents to run the installation. This would decouple your installation code from your infrastructure allowing you to apply automatically as well as on demand without having to repeat your work.

## User Data Content

The code example above has a typo in the closing tag for PowerShell. I'd try replacing this and see if that's the root issue. If there's further issues after fixing that, post a comment on this answer and I'd be glad to look further.

Also as far as powershell syntax consider using join path for proper path parsing.

For example `$exe = Join-Path $ENV:ProgramData 'git/git.exe`

The. You can run

```
&$exe --config --global ....

```

I'd also recommend that you consider looking at Amazon systems manager documents. You can create a document that initializes a setting such as get config.

Then you create a AWS association to this document on a tag, at that point it will run automatically anytime a new instance is created without you having to continue to maintain it. It also allows you to update the document independently of the user data, and then ensure you can run it manually as needed as well. I'm a big fan of decoupling any type of configuration scripts from the userdata and using that as a last resort.

One last item...

```
Remove-Item C:\c -Recurse -ErrorAction Ignore

```

Highly recommend you revisit that. That's seems super easy to mess up and start deleting stuff you didn't mean to. I'd make sure you use a fully qualified path with `Join-Path` and also use the `-LiteralPath` instead of positional argument by default. If anyone created a folder c on c drive I'd reject that in a heartbeat in a code review.
> 2 votes

---
Tags: amazon-web-services, amazon-ec2, amazon-ami
---


# Question
Title: Ansible execute task on different hosts in order
Is it possible to use ansible to manage order of programs? Im trying to find an automated solution which will do the following steps:

1. run a program on HostA
2. run a command on HostB which requires HostA program to run
3. run a program on HostC if 2. step succeed
4. run a command on HostD which requires HostC program to run
5. run a command on HostE which requires HostC program to run

For example:

```
---
  - hosts: neededhosts
    roles:
      - startprogram
      - runcommand
      - startsecprgoram
    .
    .
    .

```

```
startprogram:
- name: start hostA program
  tasks:
    - name: start program
      hosts: HostA
      shell: /etc/init.d/HostAprogram.sh start

```

```
runcommand:
- name: run command on HostB
  tasks:
    - name: run command
      hosts: HostB
      shell: command which using host A

```

# Answer
> 8 votes
Q: *"**Is it possible to use Ansible to manage the order of programs?**"*

A: Yes. It's possible. For example, the playbook below does the job

```
- hosts: HostA
  tasks:
    - include_role:
        name: startprogram

- hosts: HostB
  tasks:
    - include_role:
        name: runcommand
      when: hostvars.HostA.HostAprogram_started|default(false)

- hosts: HostC
  tasks:
    - include_role:
        name: startsecprogram
      when: hostvars.HostB.HostBcommand_passed|default(false)

- hosts: HostD,HostE
  tasks:
    - debug:
        msg: command on {{ inventory_hostname }} which requires HostC program to run
      when: hostvars.HostC.HostCprogram_started|default(false)

```

---

Example of the roles for the purpose of testing

```
shell> cat roles/startprogram/tasks/main.yml
- debug:
    msg: /etc/init.d/HostAprogram.sh start
- set_fact:
    HostAprogram_started: true

shell> cat roles/runcommand/tasks/main.yml
- debug:
    msg: command which using host A
- set_fact:
    HostBcommand_passed: true

shell> cat roles/startsecprogram/tasks/main.yml
- debug:
    msg: /etc/init.d/HostCprogram.sh start
- set_fact:
    HostCprogram_started: true

```

Example of the output

```
ok: [HostA] => {
    "msg": "/etc/init.d/HostAprogram.sh start"
}
ok: [HostB] => {
    "msg": "command which using host A"
}
ok: [HostC] => {
    "msg": "/etc/init.d/HostCprogram.sh start"
}
ok: [HostD] => {
    "msg": "command on HostD which requires HostC program to run"
}
ok: [HostE] => {
    "msg": "command on HostE which requires HostC program to run"
}

```

---
Tags: ansible
---


# Question
Title: What is the DevOps Best Practices for Hybrid environments?
I want to learn best practices, and I hope you could help me with your expertise.

In a large organization that uses cloud services and taking advantage of Infrastructure as Code (IaC). My two questions are as follows:

1) What is the best way to patch a cloud IaaS or hybrid environment? I s there a way to do so without downtime at all? Please explain.

2) With the powerfulness of IaC and that we can provision environments in seconds. Is there even a point to patch? Why don’t we just keep provisioning whenever we want and decommission the older ones?

Any inputs will be greatly appreciated!

# Answer
There may be a significant speed difference between **provisioning** an environment and **bringing up** that environment. Especially when it comes to what you mentioned: IaaS and hybrid environments - those are VMs and bare metal servers - it may take many seconds/minutes for some of them to come up.

So there may still be value in patching those environments, provided, of course, that the respective patches can be done without downtime. That depends on the patches - upgrading the kernel on a linux server, for example, would require a reboot to take effect. Upgrading Python won't.

The intended usage for particular IaaS/hybrid environments is also an important aspect in this discussion. The downtime could be completely insignificant for a distributed app designed to tolerate restarts of individual servers in the pool it's running on but probably a big deal for a centralized server app serving thousands of users.
> 2 votes

---
Tags: azure-devops, cloud, infrastructure-as-code, servers, infrastructure
---


# Question
Title: Can an in-memory filesystem be created from inside a docker container with no special permissions?
I'd like to create an in-memory filesystem within a docker container running on a cluster where I can't assign the container any special permissions (e.g. the container can't have `CAP_SYS_ADMIN` capability necessary for using `mount`).

Is it possible to create an in-memory filesystem under docker? Will one of the user space filesystems provide this functionality?

Reference article(s):

UPDATE: After finding and reading the references added above it's looking more and more like the answer is a firm no. I'll leave this open a bit in case anyone wants to elaborate or correct me.

# Answer
> 3 votes
Kubernetes can mount a ram disk for you. The answer is right there in the standard RTFM guide.

https://kubernetes.io/docs/concepts/storage/volumes/#emptydir

Create an `emptyDir` volume and set `emptyDir.medium` to "Memory" to build, what I interpret from the documentation to be, a dynamically sized memory file system which counts against the containers memory limit.

If you're not running under kubernetes, whatever system that deploys the docker container would need to create the ram disk and mount that into the container, or give the container the special permissions discussed in the various reference articles in the question.

# Answer
> 0 votes
This means that you can start a container where the /host directory is the / directory on your host; and the container can alter your host file system without any restriction. This is similar to how virtualization systems allow file system resource sharing.

Docker containers are very similar to LXC containers, and they have similar security features. When you start a container with docker run, behind the scenes Docker creates a set of namespaces and control groups for the container.

---
Tags: docker, kubernetes
---


# Question
Title: How to filter by entity name using sensuctl?
Using sensuctl for sensu-go, I want to be able to search for entities by name, preferably using wildcards.

I've tried this:

`.\sensuctl.exe entity list --field-selector "testfilter in entity.name"`

This still shows all of the entities regardless of the name.

It seems that --field-selector may only be available in the licensed version and that may be why this isn't working.

# Answer
Using some shell commands in cmder to parse the json and filter the results I was able to filter the results. Here is the command:

```
.\sensuctl.exe dump entity --all-namespaces --format wrapped-json | jq ".metadata.name" | grep -i <filtertext>

```
> 1 votes

---
Tags: sensu
---


# Question
Title: Open Ports on Google Cloud Load Balancer
It seems that by default, Google Cloud load balancers expose a number of ports unecessarily. I haven't found a way to expose only 80/443 and every time I make one of their load balancers, the following ports are seen in an nmap:

```
PORT     STATE  SERVICE
25/tcp   open   smtp
80/tcp   open   http
110/tcp  open   pop3
143/tcp  open   imap
443/tcp  open   https
465/tcp  open   smtps
587/tcp  open   submission
993/tcp  open   imaps
995/tcp  open   pop3s
1720/tcp open   H.323/Q.931
8080/tcp open   http-proxy

```

Is there a way to block 25, 465, 587, 993 & 995? *Note that this question is aboout GCP load balancers, not firewalls.*

# Answer
> 5 votes
info from: https://cloud.google.com/load-balancing/docs/https#open\_ports

Open ports The external HTTP(S) load balancers are reverse proxy load balancers. The load balancer terminates incoming connections, and then opens new connections from the load balancer to the backends. The reverse proxy functionality is provided by the Google Front Ends (GFEs).

The firewall rules that you set block traffic from the GFEs to the backends, but do not block incoming traffic to the GFEs.

The external HTTP(S) load balancers have a number of open ports to support other Google services that run on the same architecture. If you run a security or port scan against the external IP address of a Google Cloud external HTTP(S) load balancer, additional ports appear to be open.

This does not affect external HTTP(S) load balancers. External forwarding rules, which are used in the definition of an external HTTP(S) load balancer, can only reference TCP ports 80, 8080, and 443. Traffic with a different TCP destination port is not forwarded to the load balancer's backend.

# Answer
> 4 votes
You can't add `deny` rules to GC firewall. The default policy is `Deny`. You can only add `allow` rules - allow everything you need and let everything else get rejected.

Since the ports you need to block are allowed by default, you simply need to remove them. Check the name of the default rule:

```
gcloud compute firewall-rules list [NAME …] [--regexp=REGEXP, -r REGEXP] [--filter=EXPRESSION] [--limit=LIMIT] [--page-size=PAGE_SIZE] [--sort-by=[FIELD,…]] [--uri] [GLOBAL-FLAG …]

```

and delete it with:

```
gcloud compute firewall-rules delete NAME [NAME …] [GLOBAL-FLAG …]

```

You may check here for more detailed explanation on how to handle Google Cloud firewall.

# Answer
> 4 votes
I looked for that too but I don't think you can as these are the ports used by Google to do LB:

> HTTP requests can be load balanced based on port 80 or port 8080. HTTPS requests can be load balanced on port 443.
>
> TCP Proxy Load Balancing supports the following ports: 25, 43, 110, 143, 195, 443, 465, 587, 700, 993, 995, 1883, 5222

From: GCP HTTP(S) LB and GCP TCP LB

# Answer
> 4 votes
It's not currently possible to restrict a GCP load balancer's ports and protocols used like you can with an AWS ELB. This is a feature request. https://issuetracker.google.com/issues/35904903

---
Tags: security, load-balance, google-cloud-platform
---


# Question
Title: Does building a Docker image require the Docker daemon to be running?
I am trying to get my feet wet with Docker/containers. I started with this tutorial:

*Getting Started · Spring Boot with Docker*

It works fine, but I was surprised it is necessary to have the "dockerd" daemon running to build an image. It connects to it using localhost and a port number as part of executing the Maven build. Why is this required (and can it be avoided)? I mean, in many cases the image will have to be run on a separate platform anyway. It is particularly annoying, because I am building on Windows where Docker requires you to have Hyper-V enabled, which causes other issues (for instance that you can't use VMware on the same machine).

Due to the problems, I resorted to building it all inside a Linux VM (which then ran the Docker daemon).

Another thing which might be related to the above: It seems the build process doesn't leave the image on disk; it is installed directly into the repository managed by the Docker daemon. I can then export the image from the command line, and I suppose (haven't tried) this image could be moved to another machine and run from there. But how do I build "just" the image without having it deployed anywhere, etc.? It just needs to produce the artifacts in the local file system and leave it at that. Maybe that would also remove the requirement for having Docker running?

The command the tutorial uses for running Maven is:

```
./mvnw install dockerfile:build

```

However, trying alternatives to "install" (such as "package", etc.) seemingly doesn't change the behavior.

# Answer
> 7 votes
Each command in the Dockerfile is processed as a separate step. Every step from the build process gets executed in a dedicated container. It starts from Step 1 (the first command in the Dockerfile). That step is run in a container and if it is successful, a commit to a temporary image (let's call it image A) takes place. Then, Step 2 is executed in a container which is run from image A ... and so on until the final step is executed successfully. And as Tensibai commented - how do you think to run a container without the daemon?

So, yes - building a docker image requires the docker daemon to be running.

# Answer
> 3 votes
The Docker client is a CLI interface to the dockerd REST API. Very little happens in the client itself. Docker performs build steps on the dockerd engine and changing this would be a non-trivial task. Each `RUN` step creates a temporary container to execute the command and gather the resulting container filesystem as a new layer. The build cache is maintained as images on the server. And each layer of the image is managed by the storage driver within the Docker engine.

There are various attempts to try to remove this dependency, but the biggest stumbling block for each will be the temporary containers used by `RUN` which require a container runtime and root access on the server to set up.

# Answer
> 0 votes
Rootless container builds without using a docker daemon are possible using the Buildah and Podman utilities.

N.B. Podman has vendored in the Buildah library so it's all Buildah under the covers.

# Answer
> 0 votes
With Kaniko, you can build Docker images without Docker daemon.

Not all possible Dockerfile options might be supported.

https://github.com/GoogleContainerTools/kaniko

---
Tags: docker, maven
---


# Question
Title: Run JMeter Test in Jenkins with Parameters
I have a JMeter test I would like to run in JMeter that have some parameters. I installed the performance-plugin for Jenkins to enable JMeter testing as described here:

http://jenkinsci.github.io/performance-plugin/RunTests.html

However, the instruction is not clear on how to enter command-line parameters which is essential for my test. Here is what I got so far:

Is it possible at all to enter parameters to run my test? Or does it have to be completely parameter free. IE, I need to hardcode all parameters into the test itself?

# Answer
> 1 votes
Yes, you can easily trigger your JMeter jmx suite via jenkins coupled with performance plugin. For triggering the build:

* If you're using windows execute the build as windows batch command, giving the same commands that you would give if you were running your JMeter script from your command line.
* To give parameters you can conveniently select the option in jenkins configuration 'This project is parameterised' and pass the parameters ensuring the same reference is used in your JMeter jmx file.
* You can use the \__P() function to make ramup, hostname or number of threads configurable and refer them using J command.

# Answer
> 0 votes
You can use this following command to run your performance test

```
jmeter -n -t GenericPerformanceScript.jmx > output.log &

```

---
Tags: jenkins
---


# Question
Title: Bare-Metal Kubernetes storage in enterprise
### Problem in a nutshell

What kind of storage for dynamic provisioning in a bare-metal kubernetes should i use.

### Our configuration

* Each of our kubernetes nodes (worker and master) are running on VMs. The VMs themselves run on a VCenter on top of an Hyperconverged architecture
* So far the cluster have no persistent storage configured (other than the host storage)
* We do have an HDFS (Hadoop Distributed File System) running in bare metal as well, to make it short, hdfs is a *distributed* and *Highly Available* storage.

### What do i want

I've seen Ceph&Rook being a popular solution. But if i understand it properly it required each kubernetes nodes (if i want it to be fault resistant) to have an attached physical storage, which will all contains more or less the same datas.

### What's the problem with that

If each nodes have 1Tb of storage, i will have 1Tb of storage as a ceph&rook storage, and any time i want to add a node to the cluster i will be forced to add 1To of storage to the new node as well.

Ceph will duplicate data across the cluster to make it available in case of failure. But since the cluster is running on top of a hyperconverged architecture, the cepth duplication will be duplicated again by the hyperconverged architecture... you may start to see the problem.

The above example makes no sense to me, especially when i have an HDFS next to this cluster.

My question is what is my best option to use the HDFS as a dynamic storage provisioner?

# Answer
> 1 votes
A storage solution for kubernetes is a pain for those of us who chose to run kubernetes on premises.

What storage solution one picks would certainly depend on potential usage, a solution that will perform well for simple file operations would not pan out very well if you want to run a database in kubernetes.

Given your configuration, I think "vSphere Cloud Provider" is your best option

> VMware offers a Cloud Provider known as the vSphere Cloud Provider (VCP) for Kubernetes which allows Pods to use enterprise grade persistent storage.

architecture overview of vSphere Cloud Provider

If you don't want to run databases you can use NFS using HDFS setup you have or any other NFS server.

**Disclaimer:** I have not used vSphere Cloud Provider or ceph myself. Personally, I have used NFS from our storage server for production and it works fine for moderate database loads. We also have a large gluster implementation supporting close to thousand volumes, however, based on issues we have seen with gluster I will not suggest that to anyone.

On a side note, you don't "have" to use all of your cluster nodes as "storage nodes" for Ceph & Rook. Only storage nodes will have restrictions on them. If you are going with Ceph & Rook, I will suggest to use dedicated storage nodes otherwise it may impact performance of your workloads.

---
Tags: kubernetes, kubernetes-volume
---


# Question
Title: kubernetes: how to set "top node" and "top pod" permissions?
Hoping for some pointers, I have a question: what role/permissions do I need to bind to a user that has ClusterRole=view in order to let that user use `top node` and `top pod` commands in kubectl?

My team are using kubectl version 1.15.x (client) on a 1.15.7 cluster (AKS), although I'm not sure if that's relevant information. I tried to find this in the k8s documentation, but could not find the permission-reference description, only some examples and guidelines. If you know where to get the full k8s reference, I'm equally glad and will post the solution.

Note, I also posted this on StackOverflow, but they pointed me here, which indeed looks much more suited.

Thanks in advance!

Regards, Ludo

# Answer
Thanks to arghya-sadhu's comments on stackoverflow, I've got it working:

```
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: get-pod-and-node
  labels:
    rbac.authorization.k8s.io/aggregate-to-view: "true"
rules:
- apiGroups: ["metrics.k8s.io"]
  resources: ["pods", "nodes"]
  verbs: ["get", "watch", "list"]

```

(I'm aggregating to the standard 'view' role)
> 3 votes

---
Tags: kubernetes, permissions, role-based-access-control
---