# Question
Title: GitLab keeps loading and finally fails when deploying a dockerized node.js app
**GitLab Job Log**

```
[0KRunning with gitlab-runner 13.2.0-rc2 (45f2b4ec)
[0;m[0K  on docker-auto-scale fa6cab46
[0;msection_start:1595233272:prepare_executor
[0K[0K[36;1mPreparing the "docker+machine" executor[0;m
[0;m[0KUsing Docker executor with image gitlab/dind:latest ...
[0;m[0KStarting service docker:dind ...
[0;m[0KPulling docker image docker:dind ...
[0;m[0KUsing docker image sha256:d5d139be840a6ffa04348fc87740e8c095cade6e9cb977785fdba51e5fd7ffec for docker:dind ...
[0;m[0KWaiting for services to be up and running...
[0;m
[0;33m*** WARNING:[0;m Service runner-fa6cab46-project-18378289-concurrent-0-31a688551619da9f-docker-0 probably didn't start properly.

Health check error:
service "runner-fa6cab46-project-18378289-concurrent-0-31a688551619da9f-docker-0-wait-for-service" timeout

Health check container logs:

Service container logs:
2020-07-20T08:21:19.734721788Z time="2020-07-20T08:21:19.734543379Z" level=info msg="Starting up"
2020-07-20T08:21:19.742928068Z time="2020-07-20T08:21:19.742802844Z" level=warning msg="could not change group /var/run/docker.sock to docker: group docker not found"
2020-07-20T08:21:19.743943014Z time="2020-07-20T08:21:19.743853574Z" level=warning msg="[!] DON'T BIND ON ANY IP ADDRESS WITHOUT setting --tlsverify IF YOU DON'T KNOW WHAT YOU'RE DOING [!]"
2020-07-20T08:21:19.764021012Z time="2020-07-20T08:21:19.763898078Z" level=info msg="libcontainerd: started new containerd process" pid=23
2020-07-20T08:21:19.764159337Z time="2020-07-20T08:21:19.764107864Z" level=info msg="parsed scheme: \"unix\"" module=grpc
2020-07-20T08:21:19.764207629Z time="2020-07-20T08:21:19.764179926Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
2020-07-20T08:21:19.764319635Z time="2020-07-20T08:21:19.764279612Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0  <nil>}] <nil>}" module=grpc
2020-07-20T08:21:19.764371375Z time="2020-07-20T08:21:19.764344798Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
2020-07-20T08:21:19.969344247Z time="2020-07-20T08:21:19.969193121Z" level=info msg="starting containerd" revision=7ad184331fa3e55e52b890ea95e65ba581ae3429 version=v1.2.13
2020-07-20T08:21:19.969863044Z time="2020-07-20T08:21:19.969784495Z" level=info msg="loading plugin "io.containerd.content.v1.content"..." type=io.containerd.content.v1
2020-07-20T08:21:19.970042302Z time="2020-07-20T08:21:19.969997665Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.btrfs"..." type=io.containerd.snapshotter.v1
2020-07-20T08:21:19.970399514Z time="2020-07-20T08:21:19.970336671Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.btrfs" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
2020-07-20T08:21:19.970474776Z time="2020-07-20T08:21:19.970428684Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.aufs"..." type=io.containerd.snapshotter.v1
2020-07-20T08:21:20.019585153Z time="2020-07-20T08:21:20.019421401Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.aufs" error="modprobe aufs failed: "ip: can't find device 'aufs'\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n": exit status 1"
2020-07-20T08:21:20.019709540Z time="2020-07-20T08:21:20.019668899Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.native"..." type=io.containerd.snapshotter.v1
2020-07-20T08:21:20.019934319Z time="2020-07-20T08:21:20.019887606Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.overlayfs"..." type=io.containerd.snapshotter.v1
2020-07-20T08:21:20.020299876Z time="2020-07-20T08:21:20.020218529Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.zfs"..." type=io.containerd.snapshotter.v1
2020-07-20T08:21:20.021038477Z time="2020-07-20T08:21:20.020887571Z" level=info msg="skip loading plugin "io.containerd.snapshotter.v1.zfs"..." type=io.containerd.snapshotter.v1
2020-07-20T08:21:20.021162370Z time="2020-07-20T08:21:20.021121663Z" level=info msg="loading plugin "io.containerd.metadata.v1.bolt"..." type=io.containerd.metadata.v1
2020-07-20T08:21:20.021406797Z time="2020-07-20T08:21:20.021348536Z" level=warning msg="could not use snapshotter aufs in metadata plugin" error="modprobe aufs failed: "ip: can't find device 'aufs'\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n": exit status 1"
2020-07-20T08:21:20.021487917Z time="2020-07-20T08:21:20.021435946Z" level=warning msg="could not use snapshotter zfs in metadata plugin" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin"
2020-07-20T08:21:20.021581245Z time="2020-07-20T08:21:20.021533539Z" level=warning msg="could not use snapshotter btrfs in metadata plugin" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
2020-07-20T08:21:20.030531741Z time="2020-07-20T08:21:20.030427430Z" level=info msg="loading plugin "io.containerd.differ.v1.walking"..." type=io.containerd.differ.v1
2020-07-20T08:21:20.030639854Z time="2020-07-20T08:21:20.030604536Z" level=info msg="loading plugin "io.containerd.gc.v1.scheduler"..." type=io.containerd.gc.v1
2020-07-20T08:21:20.030779501Z time="2020-07-20T08:21:20.030736875Z" level=info msg="loading plugin "io.containerd.service.v1.containers-service"..." type=io.containerd.service.v1
2020-07-20T08:21:20.030865060Z time="2020-07-20T08:21:20.030833703Z" level=info msg="loading plugin "io.containerd.service.v1.content-service"..." type=io.containerd.service.v1
2020-07-20T08:21:20.030955439Z time="2020-07-20T08:21:20.030912981Z" level=info msg="loading plugin "io.containerd.service.v1.diff-service"..." type=io.containerd.service.v1
2020-07-20T08:21:20.031027842Z time="2020-07-20T08:21:20.030998003Z" level=info msg="loading plugin "io.containerd.service.v1.images-service"..." type=io.containerd.service.v1
2020-07-20T08:21:20.031132325Z time="2020-07-20T08:21:20.031083782Z" level=info msg="loading plugin "io.containerd.service.v1.leases-service"..." type=io.containerd.service.v1
2020-07-20T08:21:20.031202966Z time="2020-07-20T08:21:20.031174445Z" level=info msg="loading plugin "io.containerd.service.v1.namespaces-service"..." type=io.containerd.service.v1
2020-07-20T08:21:20.031286993Z time="2020-07-20T08:21:20.031253528Z" level=info msg="loading plugin "io.containerd.service.v1.snapshots-service"..." type=io.containerd.service.v1
2020-07-20T08:21:20.031370557Z time="2020-07-20T08:21:20.031312376Z" level=info msg="loading plugin "io.containerd.runtime.v1.linux"..." type=io.containerd.runtime.v1
2020-07-20T08:21:20.031709756Z time="2020-07-20T08:21:20.031650044Z" level=info msg="loading plugin "io.containerd.runtime.v2.task"..." type=io.containerd.runtime.v2
2020-07-20T08:21:20.031941868Z time="2020-07-20T08:21:20.031897088Z" level=info msg="loading plugin "io.containerd.monitor.v1.cgroups"..." type=io.containerd.monitor.v1
2020-07-20T08:21:20.032929781Z time="2020-07-20T08:21:20.032846588Z" level=info msg="loading plugin "io.containerd.service.v1.tasks-service"..." type=io.containerd.service.v1
2020-07-20T08:21:20.033064279Z time="2020-07-20T08:21:20.033014391Z" level=info msg="loading plugin "io.containerd.internal.v1.restart"..." type=io.containerd.internal.v1
2020-07-20T08:21:20.034207198Z time="2020-07-20T08:21:20.034120505Z" level=info msg="loading plugin "io.containerd.grpc.v1.containers"..." type=io.containerd.grpc.v1
2020-07-20T08:21:20.034316027Z time="2020-07-20T08:21:20.034279582Z" level=info msg="loading plugin "io.containerd.grpc.v1.content"..." type=io.containerd.grpc.v1
2020-07-20T08:21:20.034402334Z time="2020-07-20T08:21:20.034369239Z" level=info msg="loading plugin "io.containerd.grpc.v1.diff"..." type=io.containerd.grpc.v1
2020-07-20T08:21:20.034482782Z time="2020-07-20T08:21:20.034452282Z" level=info msg="loading plugin "io.containerd.grpc.v1.events"..." type=io.containerd.grpc.v1
2020-07-20T08:21:20.034564724Z time="2020-07-20T08:21:20.034533365Z" level=info msg="loading plugin "io.containerd.grpc.v1.healthcheck"..." type=io.containerd.grpc.v1
2020-07-20T08:21:20.034645756Z time="2020-07-20T08:21:20.034617060Z" level=info msg="loading plugin "io.containerd.grpc.v1.images"..." type=io.containerd.grpc.v1
2020-07-20T08:21:20.034722695Z time="2020-07-20T08:21:20.034689037Z" level=info msg="loading plugin "io.containerd.grpc.v1.leases"..." type=io.containerd.grpc.v1
2020-07-20T08:21:20.034800005Z time="2020-07-20T08:21:20.034770572Z" level=info msg="loading plugin "io.containerd.grpc.v1.namespaces"..." type=io.containerd.grpc.v1
2020-07-20T08:21:20.034873069Z time="2020-07-20T08:21:20.034837050Z" level=info msg="loading plugin "io.containerd.internal.v1.opt"..." type=io.containerd.internal.v1
2020-07-20T08:21:20.036608424Z time="2020-07-20T08:21:20.036525701Z" level=info msg="loading plugin "io.containerd.grpc.v1.snapshots"..." type=io.containerd.grpc.v1
2020-07-20T08:21:20.036722927Z time="2020-07-20T08:21:20.036684403Z" level=info msg="loading plugin "io.containerd.grpc.v1.tasks"..." type=io.containerd.grpc.v1
2020-07-20T08:21:20.036799326Z time="2020-07-20T08:21:20.036769392Z" level=info msg="loading plugin "io.containerd.grpc.v1.version"..." type=io.containerd.grpc.v1
2020-07-20T08:21:20.036876692Z time="2020-07-20T08:21:20.036844684Z" level=info msg="loading plugin "io.containerd.grpc.v1.introspection"..." type=io.containerd.grpc.v1
2020-07-20T08:21:20.037291381Z time="2020-07-20T08:21:20.037244979Z" level=info msg=serving... address="/var/run/docker/containerd/containerd-debug.sock"
2020-07-20T08:21:20.037493736Z time="2020-07-20T08:21:20.037445814Z" level=info msg=serving... address="/var/run/docker/containerd/containerd.sock"
2020-07-20T08:21:20.037563487Z time="2020-07-20T08:21:20.037522305Z" level=info msg="containerd successfully booted in 0.069638s"
2020-07-20T08:21:20.087933162Z time="2020-07-20T08:21:20.087804902Z" level=info msg="Setting the storage driver from the $DOCKER_DRIVER environment variable (overlay2)"
2020-07-20T08:21:20.088415387Z time="2020-07-20T08:21:20.088327506Z" level=info msg="parsed scheme: \"unix\"" module=grpc
2020-07-20T08:21:20.088533804Z time="2020-07-20T08:21:20.088465157Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
2020-07-20T08:21:20.088620947Z time="2020-07-20T08:21:20.088562235Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0  <nil>}] <nil>}" module=grpc
2020-07-20T08:21:20.088709546Z time="2020-07-20T08:21:20.088654016Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
2020-07-20T08:21:20.092857445Z time="2020-07-20T08:21:20.092749940Z" level=info msg="parsed scheme: \"unix\"" module=grpc
2020-07-20T08:21:20.092962469Z time="2020-07-20T08:21:20.092914347Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
2020-07-20T08:21:20.093060327Z time="2020-07-20T08:21:20.093013905Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0  <nil>}] <nil>}" module=grpc
2020-07-20T08:21:20.093142744Z time="2020-07-20T08:21:20.093102173Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
2020-07-20T08:21:20.149109416Z time="2020-07-20T08:21:20.148965236Z" level=info msg="Loading containers: start."
2020-07-20T08:21:20.159351905Z time="2020-07-20T08:21:20.159146135Z" level=warning msg="Running modprobe bridge br_netfilter failed with message: ip: can't find device 'bridge'\nbridge                167936  1 br_netfilter\nstp                    16384  1 bridge\nllc                    16384  2 bridge,stp\nip: can't find device 'br_netfilter'\nbr_netfilter           24576  0 \nbridge                167936  1 br_netfilter\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n, error: exit status 1"
2020-07-20T08:21:20.280536391Z time="2020-07-20T08:21:20.280402152Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.18.0.0/16. Daemon option --bip can be used to set a preferred IP address"
2020-07-20T08:21:20.337028532Z time="2020-07-20T08:21:20.336889956Z" level=info msg="Loading containers: done."
2020-07-20T08:21:20.435200532Z time="2020-07-20T08:21:20.435033092Z" level=info msg="Docker daemon" commit=48a66213fe graphdriver(s)=overlay2 version=19.03.12
2020-07-20T08:21:20.436386855Z time="2020-07-20T08:21:20.436266338Z" level=info msg="Daemon has completed initialization"
2020-07-20T08:21:20.476621441Z time="2020-07-20T08:21:20.475137317Z" level=info msg="API listen on [::]:2375"
2020-07-20T08:21:20.477679219Z time="2020-07-20T08:21:20.477535808Z" level=info msg="API listen on /var/run/docker.sock"

[0;33m*********[0;m

[0KPulling docker image gitlab/dind:latest ...
[0;m[0KUsing docker image sha256:cc674e878f23bdc3c36cc37596d31adaa23bca0fc3ed18bea9b59abc638602e1 for gitlab/dind:latest ...
[0;msection_end:1595233326:prepare_executor
[0Ksection_start:1595233326:prepare_script
[0K[0K[36;1mPreparing environment[0;m
[0;mRunning on runner-fa6cab46-project-18378289-concurrent-0 via runner-fa6cab46-srm-1595233216-1bd30100...
section_end:1595233330:prepare_script
[0Ksection_start:1595233330:get_sources
[0K[0K[36;1mGetting source from Git repository[0;m
[0;m[32;1m$ eval "$CI_PRE_CLONE_SCRIPT"[0;m
[32;1mFetching changes with git depth set to 50...[0;m
Initialized empty Git repository in /builds/xxx.us/backend/.git/
[32;1mCreated fresh repository.[0;m
[32;1mChecking out 257ffdf2 as stage...[0;m

[32;1mSkipping Git submodules setup[0;m
section_end:1595233333:get_sources
[0Ksection_start:1595233333:restore_cache
[0K[0K[36;1mRestoring cache[0;m
[0;m[32;1mChecking cache for stage node:14.5.0-alpine-2...[0;m
Downloading cache.zip from https://storage.googleapis.com/gitlab-com-runners-cache/project/18378289/stage%20node:14.5.0-alpine-2[0;m
[32;1mSuccessfully extracted cache[0;m
section_end:1595233334:restore_cache
[0Ksection_start:1595233334:step_script
[0K[0K[36;1mExecuting "step_script" stage of the job script[0;m
[0;mln: failed to create symbolic link '/sys/fs/cgroup/systemd/name=systemd': Operation not permitted
time="2020-07-20T08:22:14.844844859Z" level=warning msg="[!] DON'T BIND ON ANY IP ADDRESS WITHOUT setting -tlsverify IF YOU DON'T KNOW WHAT YOU'RE DOING [!]"
time="2020-07-20T08:22:14.846663310Z" level=info msg="libcontainerd: new containerd process, pid: 57"
time="2020-07-20T08:22:14.906788853Z" level=info msg="Graph migration to content-addressability took 0.00 seconds"
time="2020-07-20T08:22:14.907996055Z" level=info msg="Loading containers: start."
time="2020-07-20T08:22:14.910877638Z" level=warning msg="Running modprobe bridge br_netfilter failed with message: modprobe: ERROR: ../libkmod/libkmod.c:556 kmod_search_moddep() could not open moddep file '/lib/modules/4.19.78-coreos/modules.dep.bin'\nmodprobe: ERROR: ../libkmod/libkmod.c:556 kmod_search_moddep() could not open moddep file '/lib/modules/4.19.78-coreos/modules.dep.bin'\n, error: exit status 1"
time="2020-07-20T08:22:14.912665866Z" level=warning msg="Running modprobe nf_nat failed with message: `modprobe: ERROR: ../libkmod/libkmod.c:556 kmod_search_moddep() could not open moddep file '/lib/modules/4.19.78-coreos/modules.dep.bin'`, error: exit status 1"
time="2020-07-20T08:22:14.914201302Z" level=warning msg="Running modprobe xt_conntrack failed with message: `modprobe: ERROR: ../libkmod/libkmod.c:556 kmod_search_moddep() could not open moddep file '/lib/modules/4.19.78-coreos/modules.dep.bin'`, error: exit status 1"
time="2020-07-20T08:22:14.989456423Z" level=warning msg="Could not load necessary modules for IPSEC rules: Running modprobe xfrm_user failed with message: `modprobe: ERROR: ../libkmod/libkmod.c:556 kmod_search_moddep() could not open moddep file '/lib/modules/4.19.78-coreos/modules.dep.bin'`, error: exit status 1"
time="2020-07-20T08:22:14.990108153Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.18.0.0/16. Daemon option --bip can be used to set a preferred IP address"

time="2020-07-20T08:22:15.029286773Z" level=info msg="Loading containers: done."
time="2020-07-20T08:22:15.029664106Z" level=info msg="Daemon has completed initialization"
time="2020-07-20T08:22:15.029823541Z" level=info msg="Docker daemon" commit=23cf638 graphdriver=overlay2 version=1.12.1
time="2020-07-20T08:22:15.048665494Z" level=info msg="API listen on /var/run/docker.sock"
time="2020-07-20T08:22:15.049046558Z" level=info msg="API listen on [::]:7070"

# Keeps loading and finally fails after a couple of seconds

```

**gitlab-ci.yml**

```
cache:
  key: '$CI_COMMIT_REF_NAME node:14.5.0-alpine'
  paths:
    - node_modules/

stages:
  - release
  - deploy

variables:
  TAGGED_IMAGE: '$CI_REGISTRY_IMAGE:latest'

.release:
  stage: release
  image: docker:19.03.12
  services:
    - docker:dind
  variables:
    DOCKER_DRIVER: overlay2
    DOCKER_BUILDKIT: 1
  before_script:
    - docker version
    - docker info
    - echo "$CI_JOB_TOKEN" | docker login --username $CI_REGISTRY_USER --password-stdin $CI_REGISTRY
  script:
    - docker build --pull --tag $TAGGED_IMAGE --cache-from $TAGGED_IMAGE --build-arg NODE_ENV=$CI_ENVIRONMENT_NAME .
    - docker push $TAGGED_IMAGE
  after_script:
    - docker logout $CI_REGISTRY

.deploy:
  stage: deploy
  image: gitlab/dind:latest
  services:
    - docker:dind
  variables:
    DOCKER_COMPOSE_PATH: '~/docker-composes/$CI_PROJECT_PATH/docker-compose.yml'
  before_script:
    - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
    - eval $(ssh-agent -s)
    - echo "$DEPLOY_SERVER_PRIVATE_KEY" | tr -d '\r' | ssh-add -
    - mkdir -p ~/.ssh
    - chmod 700 ~/.ssh
    - ssh-keyscan $DEPLOYMENT_SERVER_IP >> ~/.ssh/known_hosts
    - chmod 644 ~/.ssh/known_hosts
  script:
    - rsync -avR --rsync-path="mkdir -p ~/docker-composes/$CI_PROJECT_PATH/; rsync" ./docker-compose.yml root@$DEPLOYMENT_SERVER_IP:~/docker-composes/$CI_PROJECT_PATH/
    - ssh root@$DEPLOYMENT_SERVER_IP "echo "$CI_REGISTRY_PASSWORD" | docker login --username $CI_REGISTRY_USER --password-stdin $CI_REGISTRY; docker-compose -f $DOCKER_COMPOSE_PATH rm -f -s -v $CI_COMMIT_REF_NAME; docker pull $TAGGED_IMAGE; docker-compose -f $DOCKER_COMPOSE_PATH up -d $CI_COMMIT_REF_NAME;"

release_stage:
  extends: .release
  only:
    - stage
  environment:
    name: staging

deploy_stage:
  extends: .deploy
  only:
    - stage
  environment:
    name: staging

```

**Dockerfile**

```
ARG NODE_ENV

FROM node:14.5.0-alpine

ARG NODE_ENV

ENV NODE_ENV ${NODE_ENV}

# Set working directory
WORKDIR /var/www/

# Install app dependencies
COPY package.json package-lock.json ./
RUN npm ci --silent --only=production

COPY . ./

# Start the application
CMD [ "npm", "run", "start" ]

```

**docker-compose.yml**

```
version: '3.8'

services:
  redis-stage:
    container_name: redis-stage
    image: redis:6.0.5-alpine
    ports:
      - '7075:6379'
    restart: always
    networks:
      - my-proxy-net

  stage:
    container_name: xxx-backend-stage
    image: registry.gitlab.com/xxx.us/backend:latest
    build: .
    expose:
      - '7070'
    restart: always
    networks:
      - my-proxy-net
    depends_on:
      - redis-stage
    environment:
      VIRTUAL_HOST: backend.xxx.us
      VIRTUAL_PROTO: https
      LETSENCRYPT_HOST: backend.xxx.us

networks:
  my-proxy-net:
    external:
      name: mynetwork

```

**Update 1**

I got a warning on the page claims I have used over 30% of my shared runner minutes. Maybe it is about not having enough minutes.

**Update 2**

The `release` stage gets completed successfully.

**Update 3**

Before I get into this problem, I deployed once successfully. I decided to test that commit once again and see if it succeeds, but it fails!

# Answer
> 1 votes
I fixed the issue. In my case, it was `PORT` (absolutely) and `HOST` (maybe) environment variables I defined manually in the GitLab CI/CD Variables section. It seems `PORT` and maybe `HOST` are two reserved environment variables for GitLab and/or Docker.

By the way, I couldn't find anything in docs state not using those variables names.

---
Tags: docker, docker-compose, gitlab, cicd, node.js
---


# Question
Title: Releasing so many changes at once to production
My company has a mobile app which we release a new update every 2-3 weeks. For each release, there are about 50-60 Jira tickets attached. The amount of code change between the release are very high. We do not follow any CI/CD, so releases happen once or twice every month after approval from CAB.

Is having too many changes in a single release a bad way of release management?

# Answer
> 2 votes
This question as it is posed is quite leading - the use of adjective "bad" to describe what you are doing implies that there is a "good" way. Nobody can answer whether your process is "good" or "bad" but you, and it should be judged on the effectiveness of the process. Perhaps this way of doing things *is* good for your environment...

However, what we can say is that the situation you describe goes against the principles of continuous delivery and continuous integration.

For example, describing the principle of "work in small batches", the Continous Delivery website states:

> In traditional phased approaches to software development, handoffs from dev to test or test to IT operations consist of whole releases: months worth of work by teams consisting of tens or hundreds of people.

This sounds like your situation.

It goes on to state:

> In continuous delivery, we take the opposite approach, and try and get every change in version control as far towards release as we can, getting comprehensive feedback as rapidly as possible. Working in small batches has many benefits. It reduces the time it takes to get feedback on our work, makes it easier to triage and remediate problems, increases efficiency and motivation, and prevents us from succumbing to the sunk cost fallacy.

So, by this comparision, yes, your release management is "bad", because it doesn't work in small batches, or continuously. (Again, with the caveat that any process that is executable effectively is a good process).

Another widely-used reference text, the Google Site Reliability Engineering Book has a whole chapter on release engineering. It says:

> We have embraced the philosophy that frequent releases result in fewer changes between versions

Your process seems to run counter to this statement, and though you don't specify what level if testing or canary deployments you use, one may surmise from the fact that there is no continuous integration that this is very small, if any. Therefore, by "SRE logic", every new deploy is high-risk, because it contains a large amount of untested changes.

There is another aspect which increases the risk in doing things "your" way. It is not reasonable to expect that a CAB will be able to make an informed decision on whether or not a given build should be released, without feedback on the performance or behaviour of the application in it's target environment (*i.e.* prod). They will be making decisions based on predictions and assumptions, rather than data.

For all these reasons, and probably many more, I think the consensus of this community would be that the answer to your question would be "yes, this is a bad way of doing release engineering".

A better way would be to:

1. Have a minimum of testing in place, and work on extending that with every deploy
2. Start to automate the tests as far as possible and have clear documentation for how to test it when automation is currently infeasible.
3. delivery artifacts continuously and run canary deploys to a test environment; work on aligning that environment as closely as possible with the production environment
4. Aim for the option of deploying any given build, and have a means for rolling back in the case of problems, to reduce the fear associated with deployment.

Smaller, and more deployments mean smaller changes at a time, which are easier to fix, and observing the deployment procedure continuously will give better insight into its reliability and where to improve things.

Automate, measure, improve. Repeat.

---
Tags: release-management
---


# Question
Title: How can I check when a subnet was created?
There's an unexpected subnet that isn't anywhere in my project I think it was created manually but I can't find when it was created to help descover what it is for and if it's still needed?

# Answer
On CloudTrail, select **Event history**, choose **Event name** from the first drop-down, and **CreateSubnet** from the second. Then fill in your timeframe.

If you don't have CloudTrail enabled, you will probably be out of luck, as I don't see a timestamp field available via the EC2 API.

References:

https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-a-trail-using-the-console-first-time.html

https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-subnets.html
> 3 votes

---
Tags: amazon-web-services
---


# Question
Title: Clarification about Operational Support System
I just started my career as a BA. As the first task, I have to study about operational support systems like ansible tower. I could not find any references to study about this. What are the similar operational support systems to ansible tower? Are there any references to study about them.

# Answer
The Wikipedia definition of Operational Support System (OSS) makes it sound like a term of art from telecommunication companies. I haven't seen this come up in systems administration over many years of working in this area. The top vendors in the OSS space are Ericsson, Nokia, and Huawei. I never see those guys trying to help me manage cloud deployments, so I don't feel there's much overlap with what Ansible Tower does.

Ansible Tower provides a GUI for working with the anisble configuration management tool. While this provides similar functionality to what it seems an Operational Support System to be, I doubt Ansible Tower is being used as an OSS by anyone. It wasn't mentioned in any of the product comparisons I googled for OSS vendors.

Tools similar to Ansible Tower would be in categories like deployment tools or infrastructure visualization. A few examples of these:

There's also overlap with CI/CD tools.
> 2 votes

---
Tags: ansible, automation
---


title: Gitlab CI push to docker-registry fails

text:
# Question
Title: Gitlab CI push to docker-registry fails
I have GitLab server and a separate `docker-registry`. I have created a `testuser` with a `testpassword`.

My `.gitlab-ci.yml`

```
image: docker:latest
services:
  - docker:18.09.7-dind

variables:
  DOCKER_HOST: tcp://docker:2375/
  DOCKER_DRIVER: overlay2
    # See https://github.com/docker-library/docker/pull/166
  DOCKER_TLS_CERTDIR: ""
  REGISTRY: myregistry:5000/golang-test

stages:
  - build
 # - test
before_script:
  - docker login -u testuser -p testpassword  myregistry:5000

build_project:
    stage: build
    tags:
      - docker-ci
    script:
      - docker ps
      - docker build -t $REGISTRY .
      - docker push $REGISTRY

```

When CI pipeline runs, I get following error.

> WARNING! Using --password via the CLI is insecure. Use --password-stdin. Error response from daemon: Get https://myregistry:5000/v2/: dial tcp: lookup worker1 on 192.168.123.456:53: no such host ERROR: Job failed: exit code 1

```
concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "docker-ci"
  url = "https://gitlab.sample.com:8000/"
  token = "89MfxRTby83ny3eTkTf2"
  tls-ca-file = "/etc/gitlab-runner/certs/gitlab.sample.com.crt"
  executor = "docker"
  pre_build_script = "export DOCKER_HOST=tcp://docker:2375"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
  [runners.docker]
    tls_verify = false
    image = "ubuntu:18.04"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache"]
    shm_size = 0

```

This was after suggested modifications here.

I can successfully login to my `docker-registry` using `docker login -u testuser -p testpassword https://myregistry:5000` from gitlab running server.

# Answer
Are you sure that your registry is publicly available? Checklist:

* Check if the domain is reachable and responding the right IP address -- `nslookup myregistry`
* Check if your port is open or not blocked any kind of software (let's say via firewalls or AWS security group) -- `telnet myregistry 5050`
* Check if your registry is not restricted for a particular CIDR of IP
* Test your login locally and ensure that there is no problem with it
> 3 votes

---
Tags: docker, continuous-integration, gitlab, gitlab-ci-runner
---


# Question
Title: Running a command as a specific user on an ec2 using ssm
## Background

I want to automatically configure an EC2 via a build, using the aws cli and ssm.

The (manual) setup for the container looks something like this:

* Create an EC2
* Run a shell script as root
* Run a shell script as a specific user

## Question

Is it possible to run a command on an ec2 utilising a tool such as `aws ssm send-command` specifying the linux user which will execute the command?

Assuming the command is a shell script, would specifying the user inside the script do the same job? e.g using `sudo su my_user`

# Answer
I didn't find anything on AWS or boto3 docs that allows for that, but I was able to execute as a different user using the `runuser` command. In theory, you could do the same thing with a combination of `sudo` and `su` commands, but this one is pretty simpler.

For that, you can do as follows:

```
runuser -l  userNameHere -c '/path/to/command arg1 arg2'

```

Since send-command executes as root, you don't have any issues.

**Note:** I thought that send-command uses in some way a session managed by the *SSM Session Manager*, but I was wrong. I spent a good time configuring *SSM Session Manager* preferences and tagging IAM resources according to this doc and this one, but send-command always execute as root as far I saw.

Sources:
> 10 votes

---
Tags: amazon-web-services, linux, aws-cli
---


# Question
Title: Azure DevOps Pipelines running on incorrect branches
I'm attempting to build different branches with different YAML configuration files in each branch. However, when I commit to a branch that should only trigger 1 pipeline it triggers both.

Development YAML:

```
    trigger:
    - master_Development

    pool:
      vmImage: 'windows-latest'

    variables:
      solution: '**/*.sln'
      buildPlatform: 'Any CPU'
      buildConfiguration: 'Release'
      buildPackageDirectory: 'packages'
      nugetConfig: 'SW.Api.Net/nuget.config'

    steps:

    - task: NuGetToolInstaller@0
      displayName: 'Use NuGet 4.4.1'
      inputs:
        versionSpec: 4.4.1

    - task: NuGetCommand@2
      displayName: 'NuGet restore'
      inputs:
        restoreSolution: $(solution)
        feedsToUse: config
        nugetConfigPath: $(nugetConfig)
        externalFeedCredentials: TelerikFeed
        restoreDirectory: $(buildPackageDirectory)

    - task: VSBuild@1
      displayName: 'Build solution'
      inputs:
        solution: $(solution)
        msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactStagingDirectory)\\"'
        platform: $(buildPlatform)
        configuration: $(buildConfiguration)

    - task: VSTest@2
      displayName: 'Test Assemblies'
      inputs:
        testFiltercriteria: 'TestCategory=Unit Test'
        platform: $(buildPlatform)
        configuration: $(buildConfiguration)

    - task: PublishBuildArtifacts@1
      displayName: 'Publish Artifact: drop'
      inputs:
        PathToPublish: $(build.artifactStagingDirectory)
        ArtifactName: 'drop'
      condition: succeededOrFailed()

Production YAML:

trigger:
- master

pool:
  vmImage: 'windows-latest'

variables:
  solution: '**/*.sln'
  buildPlatform: 'Any CPU'
  buildConfiguration: 'Release'
  buildPackageDirectory: 'packages'
  nugetConfig: 'SW.Api.Net/nuget.config'

steps:

- task: NuGetToolInstaller@0
  displayName: 'Use NuGet 4.4.1'
  inputs:
    versionSpec: 4.4.1

- task: NuGetCommand@2
  displayName: 'NuGet restore'
  inputs:
    restoreSolution: $(solution)
    feedsToUse: config
    nugetConfigPath: $(nugetConfig)
    externalFeedCredentials: TelerikFeed
    restoreDirectory: $(buildPackageDirectory)

- task: VSBuild@1
  displayName: 'Build solution'
  inputs:
    solution: $(solution)
    msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactStagingDirectory)\\"'
    platform: $(buildPlatform)
    configuration: $(buildConfiguration)

- task: VSTest@2
  displayName: 'Test Assemblies'
  inputs:
    testFiltercriteria: 'TestCategory=Unit Test'
    platform: $(buildPlatform)
    configuration: $(buildConfiguration)

- task: PublishBuildArtifacts@1
  displayName: 'Publish Artifact: drop'
  inputs:
    PathToPublish: $(build.artifactStagingDirectory)
    ArtifactName: 'drop'
  condition: succeededOrFailed()

```

Why would the pipelines not be separately triggering they target different branches?

# Answer
> 1 votes
I have solved this issue, it had to do with the path to the pipeline.

I created a directory called pipelines/dev and pipelines/prod in the root of my repo. And I pointed either build pipeline to the respective yaml config file. Now it will only build on the commit of the specific branches.

---
Tags: azure-devops, azure-pipelines, yaml
---


title: Who should write Docker files?

text:
# Question
Title: Who should write Docker files?
Once again company debate rages on who should write Docker files (including compose, interchangeable for this post).

The scenario is common in my work (developer and consultant) and I wanted to know if there is a hard and fast rule for the creation and management of Docker files. I have operated in the following three scenario:

1.) Dev-ops writes and manages the Docker files

Pros: they don't have surprises for production and ultimately control the final instance Cons: Devs are less informed on Docker/server, have to wait for certain changes or ask permission or need an ops team person to do something like add an environment variable

2.) Developers write and maintain Docker files

Pros: Devs keep on working, collisions and conflicts are resolved inside that team Cons: Dev-ops often is handed a bad or troubling Docker file for production. Dev ops also spends time fixing developer problems on the Docker file. Developers generally do not focus on security with respect to instance.

3.) Hybrid approach: Devs write the Docker file for their use in conjunction with dev-ops policy or needs. Dev-ops bases a different production docker file on the developers working Docker file but add in secret handling, Terraform/IaC/CI+CD needs, and takes responsibility for Docker files running on instance.

In this scenario dev-ops teams watched our (developer) files and used the differentials to ask questions or provide insight into changes.

--

The problem I run into over and over (regarding Docker files):

* Devs can't make simple changes for various reasons (scenario #1)
* Changes by devs upset dev-ops (all scenario)
* Dev-ops needs "2 weeks notice for any change" or similar unacceptable delay (scenario 1)
* Developers spend too much time learning how to be infrastructure compliant and lose productivity (scenario 1)
* Dev-ops get junk files (scenario2)
* liability for security and other ops related tasks are shared between departments (much infighting when things go wrong) (problematic in all scenario)

The list of experienced problems is quite long, but I hope this is enough information to establish a dialog on this topic from the professionals who handle Docker the most and take responsibility for uptime.

**23July Responding to @MLu:**

Dev ops is "Developer Operations", as in the deploy and infra side of things. "Working together" always happens, but these are two distinct career paths and domain of expertise (developer and operations). As the Wikipedia entry underscores my view, there is note regarding context dependence on the paradigm. Dev writes the code, Ops puts to instance, in the middle there is collaboration. So I don't agree at all that "DevOps model is exactly the blend of these two roles.", not in my 17 years on many teams.

One of the teams was considered "SRE" while another liked "Devops", and another "Infra (not even answering people who use devops)". Each team had their own ideas but the same problems in the end servicing the development aspect of projects.

Development and "Ops" as in CI/CD need to work together, but for some reason only these "ops" teams in all scenario were the acknowledged pain point. Why is this so common?

If DevOps is born of developer practices (agile, reliance on SCM, DRY, etc) and usually staffed by a high percentage of ex-developers why is the staffing diaspora in build/ship/pipeline generally slow to react or respond, require long lead time, and are generally the most common blocker on projects. I question the ivory castle declarations from these ops teams. When a team is overloaded, under performing, and diverges from established norms then the (the dogma of) "X needs to do the Y" could be a solution or grasping at straws?

I do agree with @MLu's commentary on management of Docker files via PR and other approaches outlined. I read the response as a hybrid approach that I prefer.

Frustration is rooted in a few projects where I was "Devops" and we had cloud based products with complicated CI/CD and other operations. Yet we never had devops problems. As soon as a dedicated team handles these elements things would slow down, uptime suffered, and the product release cycle was more stressful. A cadre of us are discussing alternatives to the current paradigms because we are repeating the same problem over and over, both in an org and across different jobs.

# Answer
Isn't *DevOps* .. you know .. *Devs* and *Ops* working together, ideally in the same team?

What you refer to as *Dev-ops* in your post sounds very much like the traditional *Operations* \- developers build something and hand it over to Operations for deployment. Well that's not DevOps.

DevOps model is exactly the blend of these two roles. Not necessarily in one person but ideally in one team. Some people in the team write the actual code and tests, some maintain CI/CD pipelines, some write packaging / build / Docker files, some look after deployment. All the above ideally driven by CI/CD so that most people in the team can focus on writing code. Building, packaging, testing, and deployment happens automatically.

In our team the Dockerfiles are in the same repo as the software sources we work on and any developer can update it if needed (through a PR of course). The deployment files (AWS CloudFormation, Terraform, etc) are in another repo and again anyone can update these through a PR. Once any change (source, Dockerfile, CloudFormation, ...) is committed and pushed to git it automatically builds and deploys to Dev environment.

Single click in the CI/CD interface can then promote it to Test and another click to Qual and then to Prod.

**TL;DR** What you describe is not *DevOps* but instead a traditional *Dev* and traditional *Ops* fighting each other ;)
> 9 votes

# Answer
I agree with MLu although my experience is generally in small teams where roles tend to get a bit blurred anyway. Taking the 'infrastructure as code' approach that led to dev ops, the dev ops team should be collaborating with the dev team just like the dev team collaborate internally to ensure features work together. Perhaps having separated teams is the problem and instead you should have project based teams that include all specialities working together. Again my experience is in small startup environments so this may be a big shift in a larger organisation used to departmentalisation.
> 2 votes

---
Tags: docker, amazon-web-services, operations
---


# Question
Title: What are the pros and cons of Azure Artifacts vs. Artifactory?
We have a requirement where we need to develop a custom product where the software component life cycle needs to be managed i.e. upload, download, recall etc. one option is to go with Custom API leveraging Azure Artifacts and the other option is to go with Artifactory. Can we have some insight on what are the Pros and Cons of these two solution. some of the links which provides info are https://www.g2.com/products/azure-artifacts/competitors/alternatives In addition to that i would also like to understand how is their support towards package managers mainly cocoapods, maven etc

# Answer
> 6 votes
**JFrog Artifactory pros**

Artifactory supports 25+ package types, while Azure supports only Maven, npm, NuGet and Python. You can get Docker and Helm with Azure Container Registry but it’s a separate tool.

Artifactory is fully hybrid - available both for on-premise installations and as SaaS - with full parity between the versions.

With Artifactory you can avoid vendor lock-in - not depending just on one cloud vendor for all of your needs. That also means not being limited to any vendor’s stack for your database, access management, etc. Proxy any remote repository - not just the official ones (like Maven Central and PyPI.org)

Artifactory is optimized for multi-site operations - Supports any network topology, over multiple, geographically distant sites and data centers, through different options for replication including pull, push, event-based and multi-push replication.

Rich metadata & AQL - Artifactory provides full metadata capabilities for all major package formats for both artifacts and folders – including custom metadata. Users can tag artifacts and folders with searchable properties. Artifactory also collects build information for build artifacts as part of the CI server integration (see below). AQL gives you unprecedented flexibility in how you search for artifacts. It offers a simple way to formulate complex queries that specify any number of search criteria, filters, sorting options and output fields.

Build info - Artifactory stores exhaustive metadata for all build artifacts deployed to it. With this “bill of materials” it is easy to faithfully reproduce a build, and trace the cause of issues to reduce time to resolution (TTR), even if the build is already in production.

**Azure Artifacts pros**

if your CI is already built on Azure DevOps, the initial set up for Azure Artifacts may be a bit quicker. The basic functionality of hosting artifacts, so that they can be resolved by other builds will work for you.

*Disclaimer: I'm a JFrog employee.*

# Answer
> 5 votes
Microsoft Artifact makes a lot of sense if you are using other Microsoft Azure products such as their DevOps solutions. If you are using Azure, Microsoft Artifact will integrate well with the rest of the platform, have better support (only 1 vendor to deal with), and should have better performance since the data will all be in their network. Additionally, you would not be billed for a data out transfer to create and store the artifact.

On the other hand Artifactory is designed to be more technologically agnostic. It will integrate with major cloud providers (Azure, Google, AWS) and supports many different types of technologies. However, there will be some setup and configuration to get it up and running if using the On-Prem version.

# Answer
> 2 votes
If you're using Java/Maven/Gradle, Azure Artifacts only supports ONE external repository: Maven Central. If your package is not in that repository, you are pretty much bound to use Artifactory or Nexus, or some similar alternative.

---
Tags: artifactory, azure-artifacts
---


# Question
Title: Find relevant domain/hosted zone under which to host subdomain
## Context

In Terraform, to create a new domain record, I first need to find the zone under which it should be hosted. E.g. If I want to create a DNS record for `mysite.example.com`, I'd get the zone `example.com` and create a record for `mysite` under it, like so:

```
variable zone {
    type = string
}
variable subdomain {
    type = string
}
variable targets {
    type = list(string)
    default = ["devops.stackexchange.com"] # just defaulting this stuff to reduce code in this sample
}
data aws_route53_zone this {
    name = var.zone
}
resource aws_route53_record this {
    zone_id = data.aws_route53_zone.this.zone_id
    name = "${var.subdomain}.${var.zone}"
    type = "CNAME"
    ttl = 100
    records = var.targets
}
output fqdn {
    value = aws_route53_record.this.fqdn
}

```

If I wanted to create `devops.mysite.example.com` then I'd either call this module with:

```
zone = "example.com"
subdomain = "devops.mysite"

```

Or, if I had a hosted zone for `mysite.example.com`, I'd call it with:

```
zone = "mysite.example.com"
subdomain = "devops"

```

It's possibly that I could go with the fist approach (e.g. where `example.com` is my only hosted zone) only for someone to later add a hosted zone for the `mysite` subdomain. If that occurred I wouldn't want to have to change my code; since the requirement is still the same; I'd just want Terraform to see that it was still possible to fulfill my requirement (i.e. to create a DNS CName record pointing `devops.mysite.example.com` at `devops.stackexchange.com`) and adjust accordingly.

# Question

Is there a way to have Terraform determine the appropriated hosted zone for a given record? E.g. So I could call my module passing `devops.mysite.example.com` and have the system work its way through the hosted zones recursively until it finds the correct one, then creates the appropriate entry there (or if no appropriate zone was found, error)?

# Answer
Short answer: I don't believe that is possible.

*In my opinion*, I suspect this problem has more to do with process and responsibilities than it does with Terraform.

You said:

> It's possibly that I could go with the fist approach (e.g. where example.com is my only hosted zone) only for someone to later add a hosted zone for the mysite subdomain. If that occurred I wouldn't want to have to change my code...

Terraform is best when *at least most* of its associated resources are Terraform-managed. That means you should be managing your zones, too, not just your records.

Which begs the question: why isn't your zone in Terraform?

If your zone was Terraform-managed, the impact of that change should be evident in a `terraform plan` to anyone who might come along to change it. Even if it references a zone in a different remote state file, a tool like runatlantis.io could show that diff in the Github PR before merging.

Let's say this is the worst-case scenario: you managed the records, it's a different team that creates and manages the zones, and they don't want to use Terraform. In that case, perhaps your DNS records shouldn't be in Terraform, either. Terraform is arguably too stateful for that. Perhaps a few python boto3 scripts can do everything you need it to.
> 1 votes

---
Tags: amazon-web-services, terraform, dns
---


title: Passing replicas number into container

text:
# Question
Title: Passing replicas number into container
I have a statefulset with an init container which creates openshift objects upon scaling. I need to pass it the replicas number from the same yaml definition. However the replicas filed doesn't seem to be working with fieldRef. Is there another way to do it?

# Answer
If you launch the statefulset with the following key set

```
{"spec":{"template":{"spec":{"serviceAccountName":"myserviceaccount"}}}}

```

you can run

```
curl \
https://kubernetes/apis/apps/v1/namespaces/<NAMESPACE>/statefulsets/<STATEFULSET> \
-k -H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
| jq '.spec.replicas'

```

to extract the replica count. This, of course, relies on the service account (myserviceaccount in this example) to have appropriate permissions to actually query the k8s API for this information, as well as having curl and jq available in the container.

You could then setup some kind of background job in the container to regularly make this request and update an env var or file or whatever.
> 4 votes

# Answer
Define the number of replicas in the `values.yaml` and pass it anywhere in resources (statefulset)

**Update 1:** You can neither have a dynamic environment variable nor refer to the replicas field in STS. So the only way to get an updated count of STS in each pod is to redeploy with the new value of replicas in `values.yaml`:

```
replicas: 3

```

**Update 2:**

According to this link, you can mount a secret now to each pod and stay up to date about replicas count.
> 2 votes

# Answer
There are two ways either increase the number in your DeploymentConfig directly or use `oc scale statefulset <name_of_statfulset> --replicas=2`

```
apiVersion: apps.openshift.io/v1
kind: DeploymentConfig
metadata:
  labels:
  app: jekyll
  name: jekyll
  namespace: test
spec:
  replicas: 1 <----------- here increase the number

```
> 0 votes

# Answer
I suggest you try to pass this information as an update to the statefulset.

See this for reference: https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/#updating-statefulsets
> 0 votes

---
Tags: yaml, openshift
---


# Question
Title: How to produce a map from the cartesian product of two lists to use with for_each on a resource with Terraform
I need to create a datadog synthetics monitors with the resource `datadog_synthetics_test`

I need to monitor multiples cluster, with multiple point of view :

```
# Clusters
variable "datadog_gke_clusters" {
  default = [
    # NPRD
    {
      name        = "cluster01"
      environment = "nprd"
      url = {
        private = "https://private.domain-priv.com"
        public  = "https://public.domain.com"
      }
    },
    # PROD
    {
      name        = "cluster02"
      environment = "nprd"
      url = {
        private = "https://private.domain-priv.com"
        public  = "https://public.domain.com"
      }
    }
  ]
}

# Point of views
variable "gke_slo" {
  description = "Response time threshold in ms to respond to a request."
  default = [
    {
      zone              = private
      monitor_threshold = 50
    },
    {
      zone              = public
      monitor_threshold = 100
    },
  ]
}

```

`for_each` need a map or a list of strings

```
resource "datadog_synthetics_test" "gke-monitoring" {
  for_each = ... # cartesian product of lists

  type    = "api"
  subtype = "http"
  ....
}

```

There is a function `[setproduct(sets...)][1]` which produce a list which combine values of each list, but not a map.

I would need to produce a map which look like this :

```
  {
    "cluster01-private" = {
      name        = "cluster01"
      environment = "nprd"
      url = {
        private = "https://private.cluster01.domain-priv.com"
        public  = "https://public.cluster01.domain.com"
      }
      zone              = private
      monitor_threshold = 50
    },
    "cluster01-public" = {
      name        = "cluster01"
      environment = "nprd"
      url = {
        private = "https://private.cluster01.domain-priv.com"
        public  = "https://public.cluster01.domain.com"
      }
      zone              = public
      monitor_threshold = 100
    },
    # PROD
    .... etc
  }

```

How could I do that ?

# Answer
In 2 steps, you can first create your cartesian product in a variable:

```
locals {
  datadog_gke_clusters_by_pov = flatten([
    for cluster in var.datadog_gke_clusters : [
      for pov in var.gke_slo_ : {
        cluster    = cluster
        pov        = pov
      }
    ]
  ])
}

```

And then use this new variable in your resource block:

```
resource "datadog_synthetics_test" "gke-monitoring" {
  for_each = {
    for valuue in local.datadog_gke_clusters_by_pov : "${valuue.cluster.name}-${valuue.pov.zone_name}" => valuue
  }
...
# here you can use each.value.cluster.* and each.value.pov.*
}

```

Have a good day!
> 3 votes

# Answer
As in Terraform, a loop "products" a result (a map, a list, an object...), in this case, you have to avoid nested loop:

```
# Clusters
variable "datadog_gke_clusters" {
  type = list(object({
    name        = string
    environment = string
    url = object({
      private = string
      public  = string
    })
  }))
  default = [
    # NPRD
    {
      name        = "cluster01"
      environment = "nprd"
      url = {
        private = "https://private.domain-priv.com"
        public  = "https://public.domain.com"
      }
    },
    # PROD
    {
      name        = "cluster02"
      environment = "nprd"
      url = {
        private = "https://private.domain-priv.com"
        public  = "https://public.domain.com"
      }
    }
  ]
}

# Point of views
variable "gke_slo" {
  type = list(object({
    zone              = string
    monitor_threshold = number
  }))
  description = "Response time threshold in ms to respond to a request."
  default = [
    {
      zone              = "private"
      monitor_threshold = 50
    },
    {
      zone              = "public"
      monitor_threshold = 100
    },
  ]
}

locals {
  result = {
    for tuple in setproduct(var.datadog_gke_clusters, var.gke_slo): "${tuple[0].name}_${tuple[1].zone}" => {
      name              = tuple[0].name
      environment       = tuple[0].environment
      url               = tuple[0].url
      zone              = tuple[1].zone
      monitor_threshold = tuple[1].monitor_threshold
    }
  }
}

output "result" {
  value = local.result
}

```

Here we use `setproduct` to produce a `list(list(tuple(cluster,pov))` and then we can iterate on the first list to produce the expected result.
> 2 votes

# Answer
Your loop is being nested.

Use this instead.

```
# Clusters
variable "datadog_gke_clusters" {
  type = list(object({
    name        = string
    environment = string
    url = object({
      private = string
      public  = string
    })
  }))
  default = [
    # NPRD
    {
      name        = "cluster01"
      environment = "nprd"
      url = {
        private = "https://private.domain-priv.com"
        public  = "https://public.domain.com"
      }
    },
    # PROD
    {
      name        = "cluster02"
      environment = "nprd"
      url = {
        private = "https://private.domain-priv.com"
        public  = "https://public.domain.com"
      }
    }
  ]
}

# Point of views
variable "gke_slo" {
  type = list(object({
    zone              = string
    monitor_threshold = number
  }))
  description = "Response time threshold in ms to respond to a request."
  default = [
    {
      zone              = "private"
      monitor_threshold = 50
    },
    {
      zone              = "public"
      monitor_threshold = 100
    },
  ]
}

locals {
  result = {
    for tuple in setproduct(var.datadog_gke_clusters, var.gke_slo): "${tuple[0].name}_${tuple[1].zone}" => {
      name              = tuple[0].name
      environment       = tuple[0].environment
      url               = tuple[0].url
      zone              = tuple[1].zone
      monitor_threshold = tuple[1].monitor_threshold
    }
  }
}

output "result" {
  value = local.result
}

```
> 0 votes

---
Tags: terraform
---


# Question
Title: Rename build pipeline .yml files in Azure DevOps
I have a .NET Core application that has three build pipelines. (Windows, Linux, macOS) In each of them, the application gets assembled a little different. This part works so far.

Azure created for each pipeline a .yml file in the root folder of the repository.

* azure-pipelines.yml (Linux Build)
* azure-pipelines-1.yml (Windows Build)
* azure-pipelines-2.yml (macOS Build)

I would like to rename them to something like *azure-pipelines-**platform**.yml*

But when I do so, the configuration in Azure gets broken. I can rename the build configuration, but not the file. And I find no way to configure the link between .yml file and Build config in DevOps

Is it generally not possible to rename the .yml files?

# Answer
> 5 votes
According to this Azure Q&A it should be possible to change the name of the yaml file by clicking on "Edit in the visual designer".

# Answer
> 6 votes
As suggested by @030, I have prepared two screenshots that show how to connect the .yml files with the build pipeline.

1. Rename the .yml file in the repository
2. Open the pipeline editor
3. Select the file from the dropdown
4. Select the in step 1 renamed \*.yml file and click on `save`.

After the existing .yml is assigned, the pipeline works as before the renaming.

# Answer
> 6 votes
This has been fixed in recent version of Azure DevOps. Currently you can easily rename or move the pipeline in the list of pipelines while clicking on 3 dots and select Rename/move

and change your location or move the pipeline

---
Tags: azure-devops, azure-pipelines, yaml
---


title: Why should I store artifacts?

text:
# Question
Title: Why should I store artifacts?
Why should I store artifacts? I'm doing it but i don't know if it still make sense.

* My builds are fairly quick
* I do git tag versioning
* None of my builds depends on another. (all independent micro-services)

Any compelling reason to keep this "Artifactory code" in my pipelines?

# Answer
> 3 votes
One argument in support of storing artifacts is that it promotes consistency when deploying across different environments.

From *Continuous Delivery* by Humble and Farley:

> Every time you compile the code, you run the risk of introducing some difference. The version of the compiler installed in the later stages may be different from the version that you used for your commit tests. You may pick up a different version of some third-party library that you didn’t intend. Even the configuration of the compiler may change the behavior of the application.

By only building artifacts once, we make sure that the production code is exactly the same as that which has gone through testing. In very small codebases with simple pipelines, it might not make too much of an impact, but it's still best practice to only build once.

---
Tags: artifacts
---


title: Why is Docker-in-Docker considered bad?

text:
# Question
Title: Why is Docker-in-Docker considered bad?
In August 2013 Jérôme Petazzoni created Docker in Docker, `dind` for short, this allowed Docker containers to be created inside of Docker Containers, this functionality proved very popular resulting in Jérôme's GitHub Repository receiving over a thousand stars and three hundred forks.

As of Docker 1.8, released two years later in August 2015, Docker in Docker is directly supported by Docker out of the box. However, the use of Docker in Docker comes with a warning, seemingly related to Jérôme's Post: Using Docker-in-Docker for your CI or testing environment? Think twice. which focuses on the reasons why Docker in Docker is not a great choice for Continuous Integration.

Why is it considered bad to use Docker in Docker? Is it simply a case for avoiding Turtles all the way down? or performance considerations?

# Answer
> 19 votes
# Continuous Integration Concerns

In short: Docker in Docker (dind) doesn't handle concurrency well.

The reason why you shouldn't use dind for CI is because Docker was designed to have exclusive access to the directory it uses for storage (normally `/var/lib/docker`). Dind doesn't respect this as all child processes use this directory concurrently. Every time you rebuild (from CI for example), anything related to your app in this directory could get wiped out and forced to start from zero. How would your users like it if they entered their payment details, clicked "Purchase", and suddenly found themselves back on the login screen as if they'd never done anything? That's just not good UX. Two rebuilds occur at once? That's really going to end badly for everyone involved (including your data integrity).

# Other Concerns

From the link the OP posted, security concerns arise as the system will try to apply security policies in a very "CSS-like" fashion where a lower container could have access to an outer container's resources unless explicitly prohibited. Remember when you could access web server resources by doing something like "mywebsite.com/../another\_folder/private\_resource.txt"? Also, sometimes filesystems just don't play well with each other when they're nested in this way.

# The Fix

Thankfully, the blog post in the OP has a good solution for the issue. Unless your needs are not met by "build/run/push Docker containers from your CI system itself running on Docker", you can use `-v` mode (add a data volume to your container) on the Docker socket (usually `/var/run/docker.sock:/var/run/docker.sock`) to allow the kind of access you need to the "shared" data volume. These containers will be started alongside the parent, instead of underneath, forcing synchronous IO. Now you have the same thing (almost) as dind but without the downsides that come with Docker not being build for concurrency.

Reference (from OP): Using Docker-in-Docker for your CI or testing environment? Think twice.

# Answer
> 0 votes
Docker-in-docker has gotten better over the years and the docker dind image is getting more maturity. However there are still a number of caveats that make its use relatively difficult to get right.

### Storage driver

The main challenge of getting docker-in-docker to work well is the storage driver of the docker installation. On your host you would have filesystems like `ext4`, which docker can run `overlayfs2` on top of. However, within a docker container, docker is unable to use `overlayfs2` as the backing filesystem of another `overlayfs2`. It doesn't work recursively. There are no easy setups to create a recursive docker-in-docker storage driver stack that works well.

The only storage driver that works on anything is `vfs`. However, `vfs` does not have copy-on-write capabilities, meaning that every docker build layer is actually a complete copy of the previous layers. In docker builds with many layers, high amounts of duplication cause performance issues as well as they quickly exhaust disk space. So it's usually not a real solution.

A possible workaround is to use a volume mount for the directory of the inner docker daemon's storage (`/var/lib/docker`). Then the inner docker daemon can use the host filesystem directly and can thus run `overlayfs2` on it.

Another workaround would be to create a loop device formatted as `ext4` which can then be used as the backing filesystem for the inner docker daemon. To avoid pre-allocating space on the disk, you can use a sparse file for the loop device. For example

```
dd if=/dev/zero of="$image_file" bs=1M count=0 seek="$sparse_loop_device_size_mb"
mkfs.ext4 "$image_file"
mount -n -o loop,noatime,nodiratime,noexec,noauto "$image_file" /var/lib/docker

```

(The size of the loop device here is the maximum size, not the initial size)

Note however that docker recommends that write-heavy containers should not use the container's writeable layer due to performance reasons. So it is anyway advisable to use a volume mount for the docker daemon directory.

### Image cache

Another typical issue is the cache of docker builds or of docker pulls within docker-in-docker. If you spin up docker-in-docker daemon for a build every time, it will have zero cache. Or if you spin up a docker-in-docker daemon for an integration test, you will have to pull down everything all over again. This is obvious when you think about it, but it can be a key performance issue if not taken into account. There is currently no easy way to share cache between the host docker daemon and the docker-in-docker.

However, you may be able to mount a volume for the docker dir (`/var/lib/docker`), to use it across multiple runs. Just make sure you never mount it to two docker daemons at the same time as that's an easy way to get unpredictable behavior due to corruption. The docker dir was not designed to be accessed by more than one daemon concurrently.

### Performance

I have seen no evidence in the community of any significant performance impact of using docker-in-docker (aside from the storage driver and cache considerations mentioned above).

### Security

Docker-in-docker requires privileged mode, which can be an additional attack vector. There is ongoing work to support docker-in-docker in rootless mode, including using a fuse variant of `overlayfs`. This is less mature however.

### Workaround: Mounting docker.sock

There is, of course, the possibility of mounting `/var/run/docker.sock` within a container and you can use the docker CLI within that container. This is not a real docker-in-docker, as the container merely uses a client to connect to the host's docker daemon. It is however a popular, hassle-free option for certain use cases. This is far easier to achieve, if your circumstances permit.

---
Tags: docker, docker-in-docker
---


# Question
Title: Need help getting documentation of Jenkins
I need a little help over here:

In every site I seek info to get started using Jenkins (i.e.: devopscube) they say is very easy to understand, and theoretically is really simple once you have look into Jenkinks Architecture, but I'm having troubles to understand some concepts and I need a good site/pdf/free book (I'm from Cuba and my access to internet is a little expensive and no way to pay for courses and books :-( ) to read and fully understand, for example:

* When creating the **Jenkinsfile** for a pipeline (using **groovy**) how is the code structure made? *Where* can I find some examples starting from the easiest to the complex ones?
* What are the actual differences between **stages** and **nodes** on Jenkinks file?
* Where can I learn how to make pipelines from scratch (but not just copy/paste from somewhere, understanding what I'm doing it), i.e.: I want to run a code on the server where I have my Jenkins master every time a fellow coder make a successfully **git push**

I downloaded the Jenkis.io's Jenkins User Handbook but is a bit hard to fully understand the questions I made right up.

Thanks

# Answer
> When creating the Jenkinsfile for a pipeline (using groovy) how is the code structure made? Where can I find some examples starting from the easiest to the complex ones?

The files typically ending in `*.jenkinsfile` can be made in any text editor or IDE. For examples the best place to look is the Pipeline Examples page. There is also a github repo with more examples.

> What are the actual differences between stages and nodes on Jenkinks file?

**Nodes** \- are physical machines or virtual machines that run Jenkins as worker. They can be different architectures.

**Stages** \- These are best thought of as the recipe:

1. Mix the ingredients
2. Add water
3. Mix again
4. Bake

Each stage is an action that Jenkins needs to take.

> Where can I learn how to make pipelines from scratch (but not just copy/paste from somewhere, understanding what I'm doing it), i.e.: I want to run a code on the server where I have my Jenkins master every time a fellow coder make a successfully git push

The first two links I gave are the best way of starting but it's purely subjective. I'd suggest starting simple and then building up from there. In the document you referenced there is also a section on Pipelines which might be the missing step for you.
> 1 votes

---
Tags: jenkins, documentation
---


# Question
Title: WIndows or Ubuntu server when developing on windows?
I am developing a MEAN app a windows laptop. Should I use Ubuntu or windows server on my cloud provider? Are there downsides of choosing one over the other?

# Answer
I dont see downsides of using your Windows laptop. Its a matter of preference. If you know how to deal with Windows Environment variables, paths, EOL characters, you should be good.

Many developers use Mac or Linux because they deal better with \*unix like systems or because similar to the server they will run their apps.

At the end of the day after you build your MEAN solution the output should be the same regardless the OS.

Windows Server or Ubuntu server are just a OS to 'professionally' run your app. Absolutely no need to spend extra money to develop a MEAN app on those OS, I recommend to stick your your computer for development.
> 1 votes

---
Tags: windows, cloud, servers, ubuntu
---


# Question
Title: What is the optimum number of people in a weekly oncall schedule?
What is the optimum number of people in a typical SRE/DevOps weekly on call schedule?

* too few people (less than 4) could lead to burnout
* too many people would lead to members forgetting the ropes of being on call, which isn't desirable either

Is there a reasonable cap on how many could be there in a weekly shift?

# Answer
well, this is not your answer but I hope it helps ...

I lived both situations when I was single and no kids. When the staff was short my boss tried to motivate us by talking about the extra money and giving half day off. (I liked)

When the staff of complete I was getting on-call once a month and the money wasn't good enough to pay for the trouble. (I hated it)

I recommend to assess where your team is, personally and professionally, and put a few suggestion to be voted by them. Then give them the ability to self manage their schedule.
> 1 votes

---
Tags: process
---


# Question
Title: How can I unify my distributed build environments using Jenkins and Docker?
I have recently taken over the CI management for a medium-sized team of developers. Currently, we have one Jenkins server instance as well as six other instances of Jenkins agents to build upon. We have been using labels to determine which builds to run. For example, some instances are setup to run under environments A or B. In the future, we would like to add more instances of A and B, as well as add new environments, e.g., C, D.

My goal now is to unify these build environments and define them in one place to allow for better scale-ability and ease of management of these instances. Docker seems to be a solution to this approach. I know that I can create a Docker image for each of my individual environments. What I don't know is how to distribute the image into containers across my six machine instances. The solution I am looking for involves allowing multiple docker containers to be spun up on a single instance (docker host?), but only if needed (i.e., the scenario when all other hosts are currently hosting a container). Additionally, I need a solution that supports Windows and Linux based hosts.

I know that Docker support in Jenkins seems to be based largely via plugins, so I was hoping for some direction in which plugin to research first.

# Answer
> 1 votes
It sounds like you have a main Jenkins orchestration server and six additional nodes added for build capacity. If you are attempting to improve scalability, there are quite a few options.

If you have containers representing the build environments (A, B, C D), an option would be docker swarm across the existing machines, allowing all of the hosts to be managed as a group and your containers to be dropped on them. https://github.com/jenkinsci/docker-swarm-plugin

Similarly, but more complicated, create a kubernetes cluster (Openshift, Rancher, etc.) with your current six machines and allow the main Jenkins to spin up and use containers on the clustered kubernetes as needed. This would allow you to create, add additional machines to the cluster, if scaling is needed. You could even add your main Jenkins server to the cluster and run Jenkins as a container. https://github.com/jenkinsci/kubernetes-plugin

Another option is to add cloud (GCP, Azure, AWS) capacity as a node(s) to push overflow to pay as you need capacity in the cloud. https://github.com/jenkinsci/google-compute-engine-plugin

Lastly, you could also use your current solution, as is, while developing a cloud native CI/CD pipeline and slowly migrate to a more modular, scaling infrastructure. There are so many options in this space, some not even needing Jenkins, solutions could be tailored to specific needs.

I hope this gives you some ideas and hints!

---
Tags: docker, jenkins
---


# Question
Title: What are the advantages of using AWS nameservers as opposed to servers provided by other registrars?
I am transfering a domain name from namecheap to aws route 53 service. AWS provides you with three options when transfering:

> 1. Continue to use the name servers provided by the current registrar or DNS service.
> 2. Import name servers from a Route 53 hosted zone that has the same name as the domain.
> 3. Specify new name servers to replace the current registrar's name servers (not recommended)

what is the advantage of using option 2 over the others. What is the difference in pros and cons between the options?

# Answer
There are two parts to owning a domain. The registration, and the hosting of the DNS entries. As you are transferring the domain registration, AWS needs to know whose service do you want to use for managing your DNS host names and other settings.

As a sensible default, select option (1) as this will keep everything managed with the current registrar / hosting company and so nothing will break (at this point in time). When your hosting expires with this company, its quite likely they will drop all of your DNS settings as well, in which case you will need to have migrated somewhere else before this point (options 2 and 3)

However, if you are using AWS services you really should be managing the DNS settings in AWS and create a Route 53 hosted zone. This provides better integration with the AWS environment, and allows you to use some of the "AWS aware" aspects of Route53 for your infrastructure. Assuming that you initially selected option 1, you should then set up a parallel zone in Route53, verify all of the DNS records match (except NS settings for the domain) and then update your Domain registration to use option 2. Changing this setting can take 72 hours to roll across the internet. In my experience, usually 90% of the internet knows in under 6 hours.

Option 3 means you (or someone else) is managing the Domain DNS master server for your domain. There may be situations where this makes sense, but they are rare.
> 3 votes

---
Tags: amazon-web-services, dns, servers
---


# Question
Title: Install and remove a program in the same Puppet declaration
The program I want on my VM requires 7-zip as part of the installation process but not after the install. I want to leave the VM as clean as possible by removing 7-zip after the install is done. In puppet it seems like it would be easy enough to declare once to ensure present and again later to ensure absent.

I am currently stuck with a Duplicate declaration error. In my case 7-zip but this could appear for any package.

> Duplicate declaration: Package\[7-Zip 19.00 (x64 edition)\] is already declared

I have the code split into multiple classes but here is what is looks like in a single manifest.

```
$installer_location = 'puppet:///resources/'
$installer_name = "7z1900-${::architecture}.msi"
$install_directory = 'C:/temp/'

#Create directory for installer
file { 'install_dir':
  ensure => directory,
  path   => $install_directory,
}

# Copy installer local
file { 'Copy_7zip_installer':
  ensure => present,
  path   => "${install_directory}${installer_name}",
  source => "${installer_location}${installer_name}",
}

# install 7zip
package { 'Install 7-zip':
  ensure   => present,
  provider => 'windows',
  source   => "${install_directory}${installer_name}",
}

#Use 7-zip to unpack archive and do pipelines agent install
String $package_path = lookup('azure_pipelines_agent::package_source')
String $package_name = lookup('azure_pipelines_agent::package_name')
String $install_path = lookup('azure_pipelines_agent::install_path')

# Create the install directory
file { 'install-path':
ensure => directory,
  path   => $install_path,
}

# Copy Azure pipeline package
# Due to a bug the archive forge module requires 7-zip
# instead of defaulting to powershell for .zip files
archive { 'extract_azure_package':
  source       => $package_path,
  path         => "${install_path}${package_name}",
  extract      => true,
  extract_path => $install_path,
  cleanup      => true,
}

# uninstall 7zip
# The title (or name) of the package must match the value of the package’s DisplayName property in the registry
# https://puppet.com/docs/puppet/6.17/resources_package_windows.html
package { '7-Zip 19.00 (x64 edition)':
  ensure   => absent,
  provider => 'windows',
  source   => "${install_directory}${installer_name}",
}

# ... Configuration and setup of Azure Pipeline Agent

```

Is there a simple means of working around what puppet sees as a duplicate declaration?

# Answer
I think that the simplest solution is to have the lifecycle of the 7-Zip package managed by exec resources rather than as package resources. Imagine an exec resource that installs 7-zip, lets call it **install-7-zip**. There is also a second puppet exec resource that uninstalls 7-zip, lets call it **uninstall-7-zip**.

If all of your other puppet resources that need to use 7-zip then require 'install-7-zip' and notify 'uninstall-7-zip' then they will all be actioned whilst the executable is available.

however its not a very puppet way to solve the problem. It means that even if there are no changes required, you will still be running these exec resources, and puppet will think it has changed the platform.

is repackaging the azure pipeline package a possible alternative? perhaps re-zipping it with a different compression setting? or generating a self extracting 7-zip exe instead?
> 1 votes

---
Tags: infrastructure-as-code, puppet
---


# Question
Title: How to copy library source code from other Git repo to an Android Gradle application GitLab build process?
There is an Android application using Grandle for build. It uses a "library" stated in build.gradle file:

```
dependencies {
    api project(path: ':specialproject.speciallibrary', configuration: 'default')

```

It is configured in settings.gradle:

```
include ':specialproject.speciallibrary'
project(':specialproject.speciallibrary').projectDir = new File('../specialprojectlibrary')

```

Developers currently have the main project and specialprojectlibrary projects side-by-side on their computers. So when they build it, Gradle fetches specialprojectlibrary and builds it when building the main app (APK.

Now, when would like to have GitLab CI/CD for the app. Main app and library app are in separe Git repositories and under separate projects inside a group in GitLab.

After creating CI/CD it occured it throws error:

FAILURE: Build failed with an exception.

```
* What went wrong:
Could not determine the dependencies of task ':something:xyz'.
> Could not resolve all task dependencies for configuration ':something:xyz'.
   > **Could not resolve project :specialproject.speciallibrary.**
     Required by:
         project :mainproject
      > Project :mainproject declares a dependency from configuration 'api' to configuration 'default' which is not declared in the descriptor for project :specialproject.speciallibrary.

```

I assume it is because it canon CI/CD Gradle of the main app does not have access to library code under .//specialprojectlibrary.

So my question is:

How to copy library source code from other Git repo to an Android Gradle application GitLab build process?

# Answer
What you need is not to checkout both projects side by side, but to add the library as a Git submodule in your application repository.

Then set up GitLab CI to checkout submodules before build.

This will not only make your CI work, but as an added benefit each commit from application repository will reference a specific commit from library repository, so nobody will have to remember what commit to checkout in library repository to ensure the application builds correctly.

Edit:

To be clear, I'm not saying that composite builds (which is a proper name for your setup) are always bad, but I believe that they should be used for optional dependencies (like JetBrains did in their kotlin-native repository) or when developers need need to have an option of replacing dependency norally pulled from some Maven repository with locally build version (eg. to test if their app works with some changes in the library before submitting pull/merge request for the library) and not for everyday use.
> 2 votes

---
Tags: continuous-integration, gitlab, continuous-deployment, gitlab-ci-runner, gradle
---


# Question
Title: Terraform: Sanitize a map that may be a map of strings or map of maps or combo of both
I want to take a map such as this that is passed in as a variable

```
cidrs = {
  A = [1.1.1.1]
  B = [2.2.2.2, 3.3.3.3]
  Group1 = {
    C = [4.4.4.4]
    D = [5.5.5.5]
  }
}

```

and then within the locals section i want to sanitize it to the following

```
cidrs_sanitized = {
  A = [1.1.1.1]
  B = [2.2.2.2, 3.3.3.3]
  Group1 = [4.4.4.4, 5.5.5.5]
  C = [4.4.4.4]
  D = [5.5.5.5]
}

```

I have tried various methods of looping

everything i've tried so far has led to me trying to get each set of values as seperate variables and then merge them together example:

```
single = { A = [1.1.1.1], B = [2.2.2.2, 3.3.3.3] }
groups = { Group1 = [4.4.4.4, 5.5.5.5] }
grouped = { C = [4.4.4.4], D = [5.5.5.5] }
cidrs_sanitized = merge(single, groups, grouped)

```

and here's some of the stuff i've tried but failed to succeed with

```
groups = flatten([ for range, value in var.cidrs: tomap({ range = values(value) }) if !can(element(value, 0)) ])
single = flatten([ for range, value in var.cidrs: tomap({ range = value }) if can(element(value, 0)) ])

```

and

```
groups = {
  for range, value in var.ranges :
    range => can(values(value)) ? values(value) : null
}
grouped = flatten([for range in var.ranges : [
  for cidr_key, cidr in range : {
    cidr_key => cidr
  } if length(cidr) > 0
] if can(values(range)) ])

```

The merge always fails as it tells me the output are tuples and i can't figure out how to get terraform to show me the values of the locals grouped, groups and single when calling it as a module

Any help or points in the right direction would be appreciated, thanks.

# Answer
Following the documentation a `map` is a collection of key/values with a constraint:

> "...accepts any element type as long as every element is the same type" source

So the input map you have declared, is already an invalid map for Terraform 0.12 (at least)

Below the example of the error produced:

```
variable "cidrs" {
  description = ""
  type        = map
  default = [
    ["1.1.1.1"],
    ["2.2.2.2", "3.3.3.3"],
    {
      C = ["4.4.4.4"]
      D = ["5.5.5.5"]
    }]
}

```

When validating an error is raised:

```
$> terraform validate

Error: Invalid default value for variable

  on vars.tf line 65, in variable "cidrs":
  65:   default = [
  66:     ["1.1.1.1"],
  67:     ["2.2.2.2", "3.3.3.3"],
  68:     {
  69:       C = ["4.4.4.4"]
  70:       D = ["5.5.5.5"]
  71:     }]

This default value is not compatible with the variable's type constraint: map
of any single type required.

```

Been that said, if we can generalize A and B as "values which are lists" and Group1 as "values which are maps" and the input is a fixed type ordered we can use a *tuple* or an *object*, this can be solve in a more readable (and maintainable) way. If so, let us know so we can extend this answer.
> 1 votes

---
Tags: terraform
---


# Question
Title: Cannot understand jenkins deployment error
I am new to jenkins, and i am trying to make a deployment into a remote serve, but i am getting the following error:

```
    Current dependency:com.vmd positionWatchLoader latest.integration
:: loading settings :: file = /tmp/tmp.uhKXp22892/ivysettings.xml
:: resolving dependencies :: com.org#appli-caller;working
    confs: [default]
    found com.org#appli;300000 in local
    [300000] com.org#appli;latest.integration
:: resolution report :: resolve 82ms :: artifacts dl 0ms
    ---------------------------------------------------------------------
    |                  |            modules            ||   artifacts   |
    |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
    ---------------------------------------------------------------------
    |      default     |   1   |   1   |   0   |   0   ||   0   |   0   |
    ---------------------------------------------------------------------

:: problems summary ::
:::: WARNINGS
        ::::::::::::::::::::::::::::::::::::::::::::::

        ::          UNRESOLVED DEPENDENCIES         ::

        ::::::::::::::::::::::::::::::::::::::::::::::

        :: com.org#appli;300000: configuration not found in com.org#appli;300000: 'runtime'. It was required from com.org#appli-caller;working default

        ::::::::::::::::::::::::::::::::::::::::::::::

:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS

```

And i am not able to interpret the error message, it says: `configuration not found in com.vmd#positionWatchLoader;300000` which configuration ??

```
It was required from com.vmd#positionWatchLoader-caller;working default

```

I have no dependency called: com.vmd#positionWatchLoader-caller !!!!!

I do not know if it can help: i am using Ant to build and Ivy to manage dependencies.

Any help will be much appreciated

# Answer
I resolved the problem publishing the ivy.xml file to nexus too, adding `publishivy="true"` to my publishing task in my build.xml file.
> 1 votes

---
Tags: jenkins, deployment, dependency-management, ant
---


# Question
Title: How to render assets when building a static website with GitLab Pages and Jekyll
I'm using Jekyll to build a static site and deploy it with GitLab Pages. This site is compounded of posts (contained in the `site/_posts` folder) and refer to other HTML and Markdown documents that I placed in an `assets` folder at the project's root, as recommended by Jekyll docs.

When I build the website in the pipeline it runs and deploys it successfully to GitLab Pages. However, if I am able to find my posts, the links towards the files allegedly contained in `assets` return a 404. When I browse the `public` artifacts file, I can't see said files in the assets folder, even though I put them there before building the site.

What am I missing here? It runs fine when serving locally but as soon as it is deployed on GitLab Pages I can't access the assets file.

project architecture:

```
site/
|__> _posts/
|____> entry.md
assets/
|__> v{{ version }}/
|____> yellow-ducky.html
|____> black-ducky.md

```

.gitlab-ci.yml excerpt:

```
pages:
  stage: deploy
  tags: [docker-connect]
  script:
    # entry.md is the blog post I want to publish
    - mv entry.md site/_posts/`date +'%Y-%m-%d'`-entry.md
    # reports come from a previous job's artifacts and contain all files I want to place in assets
    - mkdir -p assets assets/v${TAG}
    - mv reports/* assets/v${TAG}/
    # build website
    - gem install bundler
    - cd site
    - bundle install
    - bundle exec jekyll build -d public
    # move built files to the folder where GitLab expects to find them
    - cd .. && mkdir -p public && mv site/public/* public/
  artifacts:
    paths:
      - public

```

\_config.yml excerpt:

```
title: RubberDucky
email: rubber@duck.io
description: >- # this means to ignore newlines until "baseurl:"
  This is the place where to find all info you need about rubber ducks.
baseurl: "/toys/rubberduck" # the subpath of your site, e.g. /blog
url: "https://gitlab.troubleshoot.io" # the base hostname & protocol for your site, e.g. http://example.com
# Build settings
theme: minima
plugins:
  - jekyll-feed

```

blog post excerpt:

```
<ul style="list-style-type:none">
    {% for duck in ducks %}
        <h2>{{ duck.name }}</h2>
        <li><a href="assets/v{{ version }}/{{ duck.url }}">View</a> | <a href="assets/v{{ version }}/{{ duck.url }}" download>Download</a></li>
    {% endfor %}
</ul>

```

# Answer
> 1 votes
It appears the issue is in the URL used to access the documents in the blog post excerpt. First of all, the URL should begin with a `/` since the `assets` folder is at the root of the site project. However (and most importantly), I also had to add the `baseurl` to the file URL as shown below:

blog post excerpt:

```
<ul style="list-style-type:none">
    {% for duck in ducks %}
        <h2>{{ duck.name }}</h2>
        <li><a href="/{{ baseurl }}/assets/v{{ version }}/{{ duck.url }}">View</a> | <a href="/{{ baseurl }}/assets/v{{ version }}/{{ duck.url }}" download>Download</a></li>
    {% endfor %}
</ul>

```

If I weren't building my post entries from a template the link would actually be `/{{ site.baseurl }}/assets/v{{ version }}/{{ duck.url }}`, but Jinja does not understand the `site.baseurl` object when rendering the template. To bypass this, I read the value of the predefined GitLab `CI_PROJECT_PATH` variable and pass it to the template under the name `baseurl` when rendering.

---
Tags: continuous-integration, gitlab, yaml, jekyll
---


title: TLS handshake failure

text:
# Question
Title: TLS handshake failure
We have an ODBC Driver trying to connect through TLS , but fails. We tested using openssl with the following command and worked fine:

> openssl s\_client -connect xxx.com:31020

the server accepts TLS 1.0 to TLS 1.3 (downgraded just for debugging)

While Trying to connect , here is the wireshark handshake capture flow:

and also while expanding the client hello TLS, this is what we have :

Any idea what's causing the handshake not to start properly?

# Answer
> 1 votes
Since we are using ISTIO as Ingress Gateway with Kubernetes, the issue is related to ISTIO gateway config. We simply had to replace the matching host to '\*' in ISTIO gateway config

---
Tags: security, ssl, istio
---


# Question
Title: Options to assign a static ip to a service running in docker swarm
Situation:

1. docker swarm with 3+ nodes
2. service running in the swarm with replicas=1 and restart=always
3. swarm hosts have multiple interfaces - one for each vlan, with the same name
4. service is deployed as a stack

I want to replace a service running in a VM on a static IP with the same one running docker, but I need to have the same SERVICE\_IP on the ETH\_SERVER interface.

What options do I have for one or more services to have static ips ?

A. added ipv4\_address in docker stack yml - does not seem to be implemented in docker (at least not yet)

B. add a new host with reverse proxies to the docker stack services

C. use keepalived on the docker hosts and map SERVICE\_IP - in this way if a host goes down the floating ip gets reassigned - but not sure if I could expose ports with FLOATING\_SERVICE\_IP:: ...

D. ... ?

Are there any "better" ways of doing this?

---

I tried doing this:

on each docker host:

```
docker network create --config-only --subnet="<CIDR>" --ip-range="<CIDR>" -o parent="INTERFACE" "vlan_name_intermediate"

```

and then a global network for the whole swarm

```
docker network create -d macvlan --scope swarm --config-from "vlan_name_intermediate" "vlan_name"

```

Now if I attach a service to network "vlan\_name" i can ping it from inside that vlan, but the IP is randomly assigned, not static. I tried adding this to the compose yaml file:

```
ipv4_address: "192.168.45.56"

```

where the ip is inside the CIDR ip-range, but the setting is ignored

# Answer
On June 30, 2016 this issue was created on GitHub to request the implementation of static IP on docker-swarm services.

Since then multiple replies with thumbs up and down were added.

At the time of writing on December 20, 2019 this issue is open and the feature has not been implemented.

**Discussion**

After reading all comments I strongly doubt whether static IPs will ever be implemented in docker-swarm. On the other hand it is unclear why this issue has not been closed. If people would like to see that this feature gets implemented, one could create a Pull request or add a thumbs up to create more awareness.
> 4 votes

# Answer
You might be interested in this more flexible Docker Swarm alternative: https://overnode.org/ . You can see it as multi-host docker compose without Swarm, but it is a bit more than this. It has got a great set of features. It also supports static IP addresses. (Disclaimer: I am an author of the project)
> 0 votes

---
Tags: docker-swarm
---


# Question
Title: Is it possible to install bitbucket container on Centos?
I'm trying to run the below composefile on centos7:

```
version: "3.3"
services:

  bitbucket:
    image: atlassian/bitbucket:latest
    ports:
      - 7990:7990
      - 7999:7999
    networks:
      - my-app-net
networks:
  my-app-net:
    driver: bridge

```

But it fails to install it. I took the info from here: https://hub.docker.com/r/atlassian/bitbucket-server/ did someone manage to install it as a container on Centos?

# Answer
> 1 votes
Since I do not currently have access to a server with Centos installed, I tested my soultion on Ubuntu 20.04. That being said, the whole point of using Docker is to make applications distribution-agnostic, so it should work on Centos as well.

You're getting an `image does not exist` error, because the image you specified does not, in fact, exist. There is a typo in one line of the README file provided on Docker Hub and you happened to use that line when writing your Compose file.

That being said, incorect image name is not your only issue. You have not specified a volume for your Bitbucket installation, meaning all of your data will be lost if the container is destroyed. Furthermore, the only way to backup a dockerized Bitbucket installation is to use method described here, which requires using external database.

With that in mind the minimal Compose file for Bitbucket is:

```
version: '3.3'
services:
  bitbucket:
    hostname: bitbucket
    image: atlassian/bitbucket-server:latest
    environment:
     - JDBC_DRIVER=org.postgresql.Driver
     - JDBC_USER=bitbucketuser
     - JDBC_PASSWORD=jellyfish
     - JDBC_URL=jdbc:postgresql://db:5432/bitbucket
    volumes:
      - bitbucket_data:/var/atlassian/application-data/bitbucket
    ports:
      - 7990:7990
      - 7999:7999
    depends_on:
      - db
  db:
    hostname: db
    image: postgres:11.2-alpine
    command: -c 'max_prepared_transactions=64'
    environment:
     - POSTGRES_USER=bitbucketuser
     - POSTGRES_PASSWORD=jellyfish
     - POSTGRES_DB=bitbucket
    volumes:
     - db_data:/var/lib/postgresql/data
volumes:
  bitbucket_data:
  db_data:

```

That can be further extended with httpd or nginx service used as a reverse proxy, which could also be used to enable HTTPS connections to your Bitbucket instance and an Elasticsearch service to replace the embedded one as well as an email server for Bitbucket to use to send various notifications to users.

I would definitely recommend the SSL-enabled reverse proxy and email server for production. Elasticsearch server is only necessary if you want to run Bitbucket cluster rather than a single instance.

---
Tags: docker-compose, containers, bitbucket, centos
---