# Question
Title: kube-controller-manager unable to communicate with etcd
**1. What `kops` version are you running? The command `kops version`, will display this information.** `Version 1.12.2`

**2. What Kubernetes version are you running? `kubectl version` will print the version if a cluster is running or provide the Kubernetes version specified as a `kops` flag.**

```
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.1", GitCommit:"4485c6f18cee9a5d3c3b4e523bd27972b1b53892", GitTreeState:"clean", BuildDate:"2019-07-18T14:25:20Z", GoVersion:"go1.12.7", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.8", GitCommit:"a89f8c11a5f4f132503edbc4918c98518fd504e3", GitTreeState:"clean", BuildDate:"2019-04-23T04:41:47Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"}

```

**3. What cloud provider are you using?** `aws` **4. What commands did you run? What is the simplest way to reproduce this issue?** `kops update cluster` `kops rolling-update` **5. What happened after the commands executed?** kube-controller-manager reports the following error while

```
Unable to sync caches for garbage collector controller
Failed to list <nil>: client: etcd cluster is unavailable or misconfigured; error #0: net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x15\x03\x01\x00\x02\x02"

```

**6. What did you expect to happen?** Upgrade kubernetes

**7. Please provide your cluster manifest. Execute `kops get --name my.example.com -o yaml` to display your cluster manifest. You may want to remove your cluster name and other sensitive information.**

```
apiVersion: kops/v1alpha2
kind: Cluster
metadata:
  creationTimestamp: 2019-05-14T16:15:12Z
  name: k8s.runningenv.cloud
spec:
  api:
    dns: {}
    loadBalancer:
      sslCertificate: arn:aws:acm:ca-central-1:111111111111:certificate/ccxcccss112-ddw-ddwd-ddsds-ddsds3232
      type: Public
  authentication:
    aws: {}
  authorization:
    rbac: {}
  channel: stable
  cloudProvider: aws
  configBase: s3://dummy-data-store/k8s.runningenv.cloud
  etcdClusters:
  - etcdMembers:
    - instanceGroup: master-ca-central-1a
      name: a
    name: main
  - etcdMembers:
    - instanceGroup: master-ca-central-1a
      name: a
    name: events
  iam:
    allowContainerRegistry: true
    legacy: false
  kubelet:
    anonymousAuth: false
  kubernetesApiAccess:
  - 1.1.1.1/32
  - 2.2.2.2/32
  kubernetesVersion: 1.12.8
  masterInternalName: api.internal.k8s.runningenv.cloud
  masterPublicName: api.k8s.runningenv.cloud
  networkCIDR: 172.20.0.0/16
  networking:
    amazonvpc: {}
  nonMasqueradeCIDR: 172.20.0.0/16
  sshAccess:
  - 1.1.1.1/32
  - 2.2.2.2/32
  subnets:
  - cidr: 172.20.32.0/19
    name: ca-central-1a
    type: Public
    zone: ca-central-1a
  - cidr: 172.20.64.0/19
    name: ca-central-1b
    type: Public
    zone: ca-central-1b
  topology:
    dns:
      type: Public
    masters: public
    nodes: public

---

apiVersion: kops/v1alpha2
kind: InstanceGroup
metadata:
  creationTimestamp: 2019-07-19T16:24:11Z
  labels:
    kops.k8s.io/cluster: k8s.runningenv.cloud
  name: large_nodes
spec:
  image: kope.io/k8s-1.12-debian-stretch-amd64-hvm-ebs-2019-05-13
  machineType: t3.xlarge
  maxSize: 2
  minSize: 2
  nodeLabels:
    kops.k8s.io/instancegroup: large_nodes
  role: Node
  rootVolumeSize: 64
  subnets:
  - ca-central-1a
  - ca-central-1b

---

apiVersion: kops/v1alpha2
kind: InstanceGroup
metadata:
  creationTimestamp: 2019-05-14T16:15:13Z
  labels:
    kops.k8s.io/cluster: k8s.runningenv.cloud
  name: master-ca-central-1a
spec:
  image: kope.io/k8s-1.12-debian-stretch-amd64-hvm-ebs-2019-05-13
  machineType: c4.large
  maxSize: 1
  minSize: 1
  nodeLabels:
    kops.k8s.io/instancegroup: master-ca-central-1a
  role: Master
  subnets:
  - ca-central-1a

---

apiVersion: kops/v1alpha2
kind: InstanceGroup
metadata:
  creationTimestamp: 2019-05-14T16:15:13Z
  labels:
    kops.k8s.io/cluster: k8s.runningenv.cloud
  name: nodes
spec:
  image: kope.io/k8s-1.12-debian-stretch-amd64-hvm-ebs-2019-05-13
  machineType: t3.large
  maxSize: 5
  minSize: 5
  nodeLabels:
    kops.k8s.io/instancegroup: nodes
  role: Node
  subnets:
  - ca-central-1a
  - ca-central-1b

```

**8. Please run the commands with most verbose logging by adding the `-v 10` flag. Paste the logs into this report, or in a gist and provide the gist link here.** Logs from kube-controller-manager

```
I0723 17:05:09.115034       1 shared_informer.go:119] stop requested
E0723 17:05:09.115219       1 controller_utils.go:1030] Unable to sync caches for garbage collector controller
E0723 17:05:09.115373       1 garbagecollector.go:233] timed out waiting for dependency graph builder sync during GC sync (attempt 300)
E0723 17:05:09.130184       1 reflector.go:125] k8s.io/kubernetes/pkg/controller/garbagecollector/graph_builder.go:124: Failed to list <nil>: client: etcd cluster is unavailable or misconfigured; error #0: net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x15\x03\x01\x00\x02\x02"
E0723 17:05:09.144553       1 reflector.go:125] k8s.io/kubernetes/pkg/controller/garbagecollector/graph_builder.go:124: Failed to list <nil>: client: etcd cluster is unavailable or misconfigured; error #0: net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x15\x03\x01\x00\x02\x02"
I0723 17:05:09.416248       1 garbagecollector.go:204] syncing garbage collector with updated resources from discovery (attempt 301): added: [{Group:auth.kope.io Version:v1alpha1 Resource:users} {Group:config.auth.kope.io Version:v1alpha1 Resource:authconfigurations} {Group:config.auth.kope.io Version:v1alpha1 Resource:authproviders}], removed: []
I0723 17:05:09.618069       1 controller_utils.go:1027] Waiting for caches to sync for garbage collector controller
E0723 17:05:10.133851       1 reflector.go:125] k8s.io/kubernetes/pkg/controller/garbagecollector/graph_builder.go:124: Failed to list <nil>: client: etcd cluster is unavailable or misconfigured; error #0: net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x15\x03\x01\x00\x02\x02"
E0723 17:05:10.152706       1 reflector.go:125]

```

**9. Anything else do we need to know?** Tried connecting to etcd and this was the response

```
root@ip-172-20-48-95:/tmp/etcd-download-test# ./etcdctl --endpoints https://127.0.0.1:2380 --cert-file=/etc/kubernetes/pki/kube-apiserver/etcd-client.crt --key-file=/etc/kubernetes/pki/kube-apiserver/etcd-client.key  --ca-file=/etc/kubernetes/pki/kube-apiserver/etcd-ca.crt --debug cluster-health
Cluster-Endpoints: https://127.0.0.1:2380
cURL Command: curl -X GET https://127.0.0.1:2380/v2/members
cluster may be unhealthy: failed to list members
Error:  client: etcd cluster is unavailable or misconfigured; error #0: x509: certificate signed by unknown authority

error #0: x509: certificate signed by unknown authority

```

`kubectl get cs` response

```
I0723 13:16:52.688710   95389 request.go:947] Response Body: {"kind":"ComponentStatusList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/componentstatuses"},"items":[{"metadata":{"name":"scheduler","selfLink":"/api/v1/componentstatuses/scheduler","creationTimestamp":null},"conditions":[{"type":"Healthy","status":"True","message":"ok"}]},{"metadata":{"name":"controller-manager","selfLink":"/api/v1/componentstatuses/controller-manager","creationTimestamp":null},"conditions":[{"type":"Healthy","status":"True","message":"ok"}]},{"metadata":{"name":"etcd-1","selfLink":"/api/v1/componentstatuses/etcd-1","creationTimestamp":null},"conditions":[{"type":"Healthy","status":"True","message":"{\"health\": \"true\"}"}]},{"metadata":{"name":"etcd-0","selfLink":"/api/v1/componentstatuses/etcd-0","creationTimestamp":null},"conditions":[{"type":"Healthy","status":"True","message":"{\"health\": \"true\"}"}]}]}
I0723 13:16:52.693112   95389 table_printer.go:43] Unable to decode server response into a Table. Falling back to hardcoded types: attempt to decode non-Table object into a v1beta1.Table
I0723 13:16:52.693491   95389 table_printer.go:43] Unable to decode server response into a Table. Falling back to hardcoded types: attempt to decode non-Table object into a v1beta1.Table
I0723 13:16:52.693534   95389 table_printer.go:43] Unable to decode server response into a Table. Falling back to hardcoded types: attempt to decode non-Table object into a v1beta1.Table
I0723 13:16:52.693556   95389 table_printer.go:43] Unable to decode server response into a Table. Falling back to hardcoded types: attempt to decode non-Table object into a v1beta1.Table
NAME                 STATUS    MESSAGE              ERROR
scheduler            Healthy   ok
controller-manager   Healthy   ok
etcd-1               Healthy   {"health": "true"}
etcd-0               Healthy   {"health": "true"}

```

# Answer
> 1 votes
According to the log there is something wrong with the Etcd certificate:

```
Error:  client: etcd cluster is unavailable or misconfigured; error #0: x509:
certificate signed by unknown authority

error #0: x509: certificate signed by unknown authority

```

Please read the documentation how to create etcd certificates again and try to fix it.

---
Tags: amazon-web-services, kubernetes
---


# Question
Title: Using Code Climate for Test Coverage alongside Jenkins
My company is currently trying to implement Code Climate for test coverage. However, I do not have much idea about it and new to using code climate.

I tried to integrate code climate with Jenkins so that once the tests are successful, I can simply move on with other Jenkins Job.

I consulted the documentation itself, and as per documentation there are things like but there is not much explanation about it.

`./cc-test-reporter after-build -t simplecov --exit-code $? || echo “Skipping Code Climate coverage upload”`

I executed this, but returned there is no such file simplecov. I later found that the -t flag is used to provide the file which will be used for coverage. Below is my Jenkins job.

```
#!/bin/bash
echo "---------------------------------------"
echo "| Downloading Code Climate "
curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 >> ./cc-test-reporter
chmod +x ./cc-test-reporter
echo "| Download Complete "
echo "---------------------------------------"

CODECLIMATE_REPO_TOKEN=${CODECLIMATE_REPO_TOKEN}
echo "Running tests"
./cc-test-reporter before-build.

```

Once I execute this, I get the following output.

```
---------------------------------------
| Downloading Code Climate
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   178  100   178    0     0    933      0 --:--:-- --:--:-- --:--:--   936

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 12.1M  100 12.1M    0     0  10.0M      0  0:00:01  0:00:01 --:--:-- 15.8M
| Download Complete
---------------------------------------
Running tests
Finished: SUCCESS

```

Why am I not getting any code coverage or things like that in the console. How do I properly integrate it in Jenkins.

**`UPDATE`** I have successfully created the job without issue. Now the problem I am facing is

```
$ ./cc-test-reporter format-coverage -t lcov coverage/lcov.info -d -o codeclimate.json
DEBU[0000] coverage path coverage/lcov.info
DEBU[0000] using formatter lcov
DEBU[0000] checking search path coverage/lcov.info for lcov formatter
DEBU[0000] couldn't load branch from ENV, trying git...
DEBU[0000] couldn't load commit sha from ENV, trying git...
DEBU[0000] couldn't load committed at from ENV, trying git...
Error: could not find coverage info for source files

user@host:app$ pwd
app
user@host:app$ ll
coverage
user@host:app$ ll coverage
lcov-report
lcvo.info

```

# Answer
The issue has been resolved. This is for nodejs

Need to install necessary dependencies either using yarn or npm. Once done, run

```
nyc --reporter=lcov yarn unit unit
$ ./cc-test-reporter after-build -t lcov --exit-code $?

```
> 1 votes

---
Tags: jenkins, code-coverage
---


# Question
Title: ansible question to execute and display multiple commands
I have the simple ansible script below:

```
- hosts: myhost
  tasks:
    - name: few commands
      command: "{{ item }}"
      with_items:
        - ls -l
        - df -h
        - cat /tmp/1

```

there will be more commands and files, how I can display the execution of each command ?

# Answer
One could use `-vvv` to get the output of the commands while running the playbook:

```
changed: [localhost] => (item=echo helloworld) => {
    "ansible_loop_var": "item",
    "changed": true,
    "cmd": [
        "echo",
        "helloworld"
    ],
    "delta": "0:00:00.002517",
    "end": "2019-12-20 15:20:09.042020",
    "invocation": {
        "module_args": {
            "_raw_params": "echo helloworld",
            "_uses_shell": false,
            "argv": null,
            "chdir": null,
            "creates": null,
            "executable": null,
            "removes": null,
            "stdin": null,
            "stdin_add_newline": true,
            "strip_empty_ends": true,
            "warn": true
        }
    },
    "item": "echo helloworld",
    "rc": 0,
    "start": "2019-12-20 15:20:09.039503",
    "stderr": "",
    "stderr_lines": [],
    "stdout": "helloworld",
    "stdout_lines": [
        "helloworld"
    ]
}

```
> 1 votes

---
Tags: ansible
---


# Question
Title: Configure Kubernetes with docker container on Ubuntu VM
I am using the `YAML` file to deploy the container on `Kubernetes` with some replication factor on a hosted machine.

# YAML File

```
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mojo-deployment
  labels:
    app: mojo
spec:
  selector:
    matchLabels:
      app: mojo
  replicas: 3
  template:
    metadata:
      labels:
        app: mojo
    spec:
      containers:
        - name: mojo
          image: mojo:1.0.1
          ports:
            - containerPort: 9000

---
#Services Info
apiVersion: v1
kind: Service
metadata:
  name: mojo-services
spec:
  selector:
    app: mojo
  ports:
    - protocol: TCP
      port: 80
      targetPort: 9376

---

#Ingress Configuration
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: mojo-ingress
  annotations:
    kubernetes.io/ingress.class: mojo
spec:
  backend:
    serviceName: mojo-services
    servicePort: 80

```

Steps:

1. Build `Docker` image using \`docker build -t mojo:1.0 .
2. `docker image ls` show me an image id.
3. Skipping `docker build` command to deploy image on container. Do I need to do it? or `kubectl` service will take care of it.
4. Run `kubectl apply -f Prod.yaml`. It shows

> deployment.apps/mojo-deployment created
>
> service/mojo-services created
>
> ingress.networking.k8s.io/mojo-ingress created

5. `kubectl get service` returns
6. `kubectl get pod` returns
7. `kubectl get deployment` returns

Questions?

1. Do I need to build the container before deploying `YAML` file? I tried it but still `kubernetes` not running.
2. Why all pods are showing `Pending` status.
3. Deployment is also showing `pending` status.
4. Though I am trying to access the `Ingress` with :80 and cannot access it.

# Answer
> 1 votes
You have to push your image in a registry in order to get the deployment working. Kubernetes does not deal with images (i.e. building, pushing...).

Try `kubectl describe deployment mojo-deployment`, you'll get more informations about why it's pending (surely because it does not know how to get your Docker image).

# Answer
> 1 votes
Your questions in order:

1. The container not only need to be built, it needs to be in a registry that Kubernetes has access to. If you are using Docker containers (the default), you can push your container to Docker Hub so that Kubernetes can retrieve the image.
2. They are showing as pending because they do not yet have any containers inside of them. You've described in the YAML that you want the container in the pod, but Kubernetes has not yet built a working Pod to your specification.
3. The deployment is pending because Kubernetes has not yet been able to create the 3 desired pods.
4. Services in Kubernetes are independent of Pods. This is why the services are shown as available. So internally, Kubernetes is routing your request to port 80 in the cluster, but no services are yet available on that port.

---
Tags: docker, kubernetes, containers, dockerfile
---


# Question
Title: How to setup Cloudfront for Load balancer for a private staging domain name?
I've got a domain name foobar-staging.com, that I use for the staging environment to get feedback, testing or demo before releasing to production. Recently, decided to deploy the UI static files to s3 and everything else falling back to ALB. The original setup is only accessible privately, so the user has to connect to a VPN to be able to access foobar-staging.com, and as far as I've tested so far Cloudfront content origins need to be public, so it seems that I won't be able to keep the staging server private, unless someone confirms me otherwise?

What I want to access privately is:

```
Domain name > Cloudfront > s3 (/routeX, /routeY) or ALB (other routes /*)

```

The EC2s only have private ips and the ELB security group do not allow public access. I've allowed 80/443 0.0.0.0/16 access to see the Cloudfront work for the setup... I don't necessarily need Cloudfront (other alternative would do, as I can't route to s3 in ALB), but without having Cloudfront in front of s3 and alb, I'd have to access the UI application through the s3 amazon domain, which I'd much prefer to use the foobar-staging.com.

Is it possible to do?

# Answer
> 1 votes
My solution is based on signed cookies (find how to implement it here, https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-signed-cookies.html); I've created a lambda that handles this for me a bit differently, as I need users to access a location to get their cookie in the easiest way possible and have someone control to whom can be granted the auth cookie. An example of a basic implementation of this is here: https://aws.amazon.com/blogs/compute/simply-serverless-using-aws-lambda-to-expose-custom-cookies-with-api-gateway/

Finally, after you have provided your Content origin in Cloudfront, in the Behaviors tab where you setup the Content origin desired behaviour you need to set "Restrict Viewer Access (Use Signed URLs or Signed Cookies)".

Obs: make sure you also forward all cookies and query strings.

---
Tags: amazon-web-services, load-balance, amazon-cloudfront
---


title: AWS ECR start uploaded docker image?

text:
# Question
Title: AWS ECR start uploaded docker image?
I used packer to upload a docker image I created to my AWS ECR repository.

How can I use the AWS cli to automatically locate that image, create a new cluster, set the port rules to allow all inbound and outbound traffic and then spin up the container? I want to automate this process so I can integrate it into packer later, as a post processor.

Can someone explain me how I can do this?

```
{
    "type": "docker-push",
    "ecr_login": true,
    "aws_access_key": "<snip>",
    "aws_secret_key": "<snipe>",
    "login_server": "https://<snip>.dkr.ecr.eu-west-1.amazonaws.com/"
}

```

Currently the last part of my packer script just uploads the image - now I need a way to automatically start this docker image and make sure the network traffic settings are configured correctly.

# Answer
The answer depends on how you want to run the container. There are 3 ways you could accomplish this:

1. Since you mentioned creating a cluster, you could create a new Kubernetes cluster and run your containers in this cluster. AWS does support creating a cluster and managing of it via the CLI.
2. You can choose to run the containers as part of AWS Elastic Container Service (ECS). This supports launching the containers via AWS Fargate (essentially a "serverless" type approach where you pay for CPU and Memory consumed) or on a standard EC2 instance. ECS has its own CLI (separate install). ECS will also allow you to set the ports and set inbound/outbound rules.
3. Finally, you can choose to use the CLI to spin up an EC2 instance and then manually or via a script start the docker containers. However, you'd be much better off choosing one of the first two options.

Finally, AWS does have a tutorial that seems to go over exactly what you are trying to accomplish (configuring a container to run via the CLI).
> 2 votes

# Answer
In my opinion, Packer should only be used for building something like a VM or docker image. If you would like to run a VM then you could subsequently use Vagrant or docker-compose in case of docker.

> now I need a way to automatically start this docker image and make sure the network traffic settings are configured correctly

If one is running k8s then one could update the tag of a docker-image that is defined in the deployment.yaml and do a `kubectl up` again.
> 1 votes

---
Tags: docker, amazon-web-services, deployment
---


# Question
Title: Mitigate false positives due to daylight saving time in CloudWatch anomaly dectector
For monitoring, I set up CloudWatch alerts based on CloudWatch anomaly detection. Overall, they work quite nicely, but they got confused when the clock gets shifted (summer timer to winter time).

We have now recurring false-positives even though the recent time shift in the US and in Europe were weeks ago. I assume CloudWatch will eventually understand it, but then again, the clock shifts have on a regular basis, which will create some noisy.

Unfortunately, I did not find much in the documentation, only that CloudWatch provides to set the `Time Zone Format`. Before I leaved it empty, but now I set it to UTC.

**Questions:**

* What is exactly the effect of setting the time zone? Does it have an impact on the prediction of the anomaly detector, or is it only for visualization and for other configurations such as excluding specified time periods?
* Our user base is all over the world, but mostly in the US and in Europe. As this scenario is not too uncommon, I wonder if asking for one one time zone is even practical. What should be used? UTC? Or pick the time zone with the most users?

I understand that it difficult to avoid false-positives completely, especially, in the first or second week after a clock shift happened. Still, I wonder how to correctly setup up the anomaly detection model, so its prediction will be most accurate.

# Answer
My current understanding is that it is not possible to avoid false positives. At least not if traffic comes from different time zones around the world (as in the context of this question).

AWS operates on UTC times, but this is irrelevant for the anomaly detection. It does play a role if you set timers, but it is not taking into account for the predictions of AWS. In other words, switching to UTC will not make false-positives less likely.

When thinking about it, it is hard to avoid false-positives. If a time-shift in Europe or the US happens, you could take that into account and predict the new traffic. But that requires extra domain knowledge. The AWS predictor only sees a generic time series.

There are other events such as Christmas, which will impact the expected traffic. Again, with extra knowledge you could improve predictions. But the problem is too difficult for the AWS predictor, as it does not have that domain knowledge.

In our concrete case, we still kept the alerts, as they do have value even with the known false-positives. Humans with knowledge about the underlying system, can reject the false-positive. It is not ideal, but it is a trade-off that we are willing to make.

Note that other types of graphs are not susceptible for these types of problems. For example, error rates instead of total number of errors. It is not always possible, but if you can use rates, the system will be more stable in my experience. AWS allows to set alerts on math expressions such as `errors / requests`. The advantage is that normalized rates tend to be agnostic of external events.
> 1 votes

---
Tags: amazon-web-services, amazon-cloudwatch
---


title: Store data in Jenkins job description

text:
# Question
Title: Store data in Jenkins job description
I have a job which I run with a boolean parameter called "BUILD\_CLEAN=true", and it defaults to false. I can store this flag in the job ID `currentBuild.displayName` . Simple enough. But for my purposes I also want to save the commit hash(es) of that build as well, which starts becoming a silly place to store metadata. I'm probably not getting the terminology right, but here is my use case.

My Pipeline declarative job builds a repo, a subrepo and all using shell, and archives the binary artifacts. The code is of mixed types, so Artifactory is not an option for part of this solution either. I want to save just the hashes someplace, to make it easy for subsequent builds made in the same day (24 hours) to inspect/list them, and trigger a separate job with the full list for that day. Do I save these hashes/values into an XML file per build? What is the best way to make extracting small pieces of data easy? *GOAL:* I'm attempting to implement a kind of generalized incremental build job, but am unable to use any of the plugins since I'm building for 5 different targets, some of them across nodes. In my specially marked build (nightly) every 24 hours, I'm .bz2 zipping the objects so I can restore them for any incremental. So it's this archive that I'm wanting to find, check the hashes and retrieve into my workspace.

TAGS: Jenkins,pipeline,tags,incremental (I'm unable to define any tags due to the rep system.)

# Answer
> 1 votes
Not sure if this would work for you, but what I do is store all of the build metadata into a `BuildInfo.json` file. You could just save it as an XML file like you said, but I found it easier to parse info with Groovy if it was a JSON file. Along with the most recent commit hash, we store most of the build information in this file, including the result, duration, user who kicked off the build, etc.

One advantage of this approach is we're able to create other jobs that take action of already built products, such as for deployments. These jobs can just look for the presence of a `BuildInfo.json` file and take action based off the information contained within. Example:

```
import groovy.json.*

def build_info = [
    'Product': 'MyApp',
    'Build Type': 'Full',
    'Build Name': currentBuild.displayName,
    'Version': '1.2.3.4',
    'Source Branch' : params.Branch,
]

def json_output = readJSON text: groovy.json.JsonOutput.toJson(build_info)
writeJSON file: "BuildInfo.json", json: json_output, pretty: 4

```

---
Tags: jenkins, jenkins-pipeline
---


# Question
Title: Legacy build environment in Docker container, incompatible with Jenkins agent
I have a working Dockerfile that creates a build environment for a legacy app, and now I'd like to use that environment from Jenkins, ideally allowing for incremental builds.

The environment is too old to run a recent version of Java, which means it cannot be used as an agent directly.

Is there a way to run a series of build steps inside a container without the container itself running an agent?

# Answer
> 1 votes
According the the Jenkins docs, you could use your custom docker image as the execution environment, assuming the server Jenkins is running on has Docker installed/running.

```
pipeline {
    agent {
        docker { image 'your-custom-image' }
    }
    stages {
        stage('Build') {
            steps {
                sh 'commands to run in your docker image here'
            }
        }
    }
}

```

---
Tags: docker, jenkins
---


# Question
Title: How to disable ping thread on jenkins agent
One of the jenkins slave is going down very often. Found RCA for this issue. It's because of the Ping thread

In the logs I fetch from agent, I have:

```
Terminating the channel channel.
java.util.concurrent.TimeoutException: Ping started at 1576130988405 hasn't completed by 1576131228405
    at hudson.remoting.PingThread.ping(PingThread.java:134)
    at hudson.remoting.PingThread.run(PingThread.java:90)

```

I wanted to disable the ping thread on the agent. In the documentation it's mentioned as `-Dhudson.remoting.Launcher.pingIntervalSec=-1`

But I do not understand how to execute this on remote machine.

I followed this link. but that solution is for JNPL. I am launching the agent on the node via ssh.

# Answer
In the web UI, from the home page, navigate to Manage Jenkins \> Manage Nodes \> Configure. This page has all of the health checks that Jenkins performs to determine if an executor is up. The check you are concerned about is called "Response Time". You can disable the "Response Time" health check on this page.

From the help text on the same page:

> This monitors the round trip network response time from the master to the agent, and if it goes above a threshold repeatedly, it marks the agent offline.
>
> This is useful for detecting unresponsive agents, or other network problems that clog the communication channel. More specifically, the master sends a no-op command to the agent, and checks the time it takes to get back the result of this no-op command.
> 0 votes

# Answer
The executor thread is waiting on the channel to give it more data.

You got a little confused due to the fact that the ping thread has the name of the channel Object in its threads name (which is the same Object that the executor is wating for).
> -1 votes

---
Tags: jenkins
---


# Question
Title: How do I upload my zipped lambda code into AWS Lambda?
So I've read through the docs at: https://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html

I've created the zip file, how / where do I actually upload / publish this zip file into AWS?

# Answer
> 9 votes
It's kind of hidden away and not very intuitive.

Within the AWS Lamba admin: Create Function \> Author from Scratch. Within the `Code entry type` dropdown there is a `Upload a .ZIP file` option.

# Answer
> 5 votes
You can upload using aws-cli.

```
aws configure

aws lambda list-functions

aws lambda update-function-code \
    --function-name MyLambdaFunction \
    --zip-file fileb://index.zip

```

See https://developer.amazon.com/blogs/post/Tx1UE9W1NQ0GYII/Publishing-Your-Skill-Code-to-Lambda-via-the-Command-Line-Interface

# Answer
> 1 votes
Upload to AWS from the Lambda console, by creating a new function here: https://console.aws.amazon.com/lambda/home. You can also build your functions and upload to AWS using a tool like Serverless framework.

---
Tags: amazon-web-services, aws-lambda
---


# Question
Title: Ansible can't recursively delete a directory
So, I am trying to recursively delete a directory with all files/folders inside. I tried:

```
  - name: Cleaning up
    file:
      state: absent
      path: /tmp/myfolder/

```

But the directory did not get deleted. Is there any force/recursive parameter I can add? What am I missing?

# Answer
You may find the following format to be helpful:

```
- name: Recursively change ownership of a directory
file:
  path: /etc/foo
  state: directory
  recurse: yes
  owner: foo
  group: foo

```
> 1 votes

---
Tags: ansible
---


# Question
Title: Re-using Python virtual environments on a build server
Currently, every time we run a build through Jenkins+Ansible, we are re-creating a virtual environment and re-installing all the dependencies listed inside the `requirements.txt` file.

This is very slow and does not scale well. How can we improve and speed up the process? Can we re-use virtual environments?

---

Our latest idea which we have not yet implemented was to build virtual environments outside of a Jenkins workspace, name virtual environments based on a project+branch and keep an MD5 sum of the requirements for every virtual environment. Then, before building an environment the next time, we calculate the MD5 sum of the current requirements in this branch and look up existing virtual environments by this MD5 sum. If an existing environment with this sum was found, just re-use it. We are not sure if this is the best way to solve the problem.

# Answer
> 5 votes
It's generally a best practice to make your builds idempotent. Leaving artifacts behind only opens up opportunities for dependency management issues- and resolving those in a resilient way is exactly why you're using a build server. I suggest you take a look at WHY your Python builds are running so slow- if you discover this is because of an overabundance of packages in each project, this might be optimizable. If you discover this is because of server performance issues, those are easily fixable with more resources. If you find you are blocked by network throughput, consider using an artifact repository to cache packages locally. But building persistent virtualenvs seems like a recipe for problems that could be unpredictable when moving to another server, and extremely frustrating to troubleshoot.

# Answer
> 3 votes
I'm going to add more story to the answer @esoterydactyl gave, in case it goes over too many people's heads.

Instead of a monolithic build process, maybe you want to build intermediate packages that can be versioned and deployed to your workspace without all the intermediate build steps. Docker is one packaging solution which is super popular in this case. You could also use .deb or .rpm or non-Docker tarballs, or make CF "StemCells" or even zip files. Also consider Python has prebuilt "wheel" packages now.

## Problems:

* all of your developers must use the same build-deps package for their sandbox or hair-pulling fun will emerge in the differences between dev sandbox and CI workspace. Thus the package must support dev laptop sandboxes and Jenkins workspace installation.
* the build-deps package must have its own CI build/test/packaging/release pipeline. It, however, can have reduced cycle frequency while nobody is committing any changes to it (or any of its components).
* This is an example of bundling. If you have, for example, a CVE to patch, everything that uses the bundle must be patched. This must be no more difficult to deploy than anything else, so don't stop work when your CI builds are fast. CI needs CD, or you could be delivering production undeployable software.
* If you're making your own prebuilt artifacts, you need infrastructure to host the artifact repository, and a versioning scheme.

The incremental solution would be to solve the minimum number of those problems which yields a significant boost in build cycle speed. This is why it makes sense to maintain a local pypi repository, and add Jenkins jobs to build and push wheel packages to it. I know that sounds like a pain, but imagine if you had to manage internal software development for all of the stuff you can just install via pip?

# Answer
> 3 votes
You may want to take a look at the ShiningPanda Plugin.

It allows running a build step inside a particular python virtual environment, created automatically if it doesn't exist or reused if it already exists. The virtual environment is located outside the workspace, but it can be named and made to appear as a directory with that name inside the workspace.

To address the dependencies you just have to insert a build step performing the desired `pip install -r requirements.txt` variant after pulling the workspace (which will probably bring in your `requirements.txt` file) but before any other build step that would need to use the virtual environment.

The same virtual environment can be used in multiple steps of the same build (by name) and a build can have multiple virtual environments if needed. But AFAIK you can't reuse the same virtual environment in different builds, each build will have its own copy.

The Plugin has a configuration to clear/wipe the environment at any build step using it, if/when needed. Obviously you wouldn't want this always enabled as it would void reusability.

IMHO it works best with `requirements.txt` files with pinned package versions (like those produced by pip-tools).

**Update**: I recently ran into an issue with the plugin on Windows: I had several jobs in different Jenkins folders but with the same **job names** and same plugin configs (notably the non-empty **`Name`** in the `Advanced` section). Turns out all jobs were actually sharing a single environment (and stepping over each-other), I had to switch to unique values for `Name` across the jobs. Apparently the 2 names are used to generate the 2 hashes from the virtual environment path, so same names -\> same path:

`\Jenkins\shiningpanda\jobs\<job_name_hash>\virtualenvs\<venv_name_path_hash>\`

So, if you have the same job names, you actually **can** share the same environment if you really want to :)

---
Tags: jenkins, ansible, python, virtualenv
---


# Question
Title: How to generate a Maven.pom file (not pom.xml)?
This seems not so easy to google because results about the normal ***pom.xml*** are overwhelming.

So, I just have learned there are also Maven artefact related files with .pom extension. This seems to be some special thing about how repositories work - a meta-file for an artefact?

For example, if you check folder contents for an artefact in your local repository, you will find these files in any artefact folder I guess, just like in the path below.

```
\.m2\repository\org\json\json\20090211\json-20090211.pom

```

Contents:

```
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <!-- this POM is released under an Apache 2.0 license-->
  <modelVersion>4.0.0</modelVersion>
  <groupId>org.json</groupId>
  <artifactId>json</artifactId>
  <version>20090211</version>
  <name>JSON (JavaScript Object Notation)</name>
  <description>
    JSON (JavaScript Object Notation) is a lightweight data-interchange format.
    It is easy for humans to read and write. It is easy for machines to parse and generate.
    It is based on a subset of the JavaScript Programming Language, Standard ECMA-262 3rd Edition
    - December 1999. JSON is a text format that is completely language independent but uses
    conventions that are familiar to programmers of the C-family of languages, including C, C++, C#,
    Java, JavaScript, Perl, Python, and many others.
    These properties make JSON an ideal data-interchange language.
  </description>
  <url>http://www.json.org/java/index.html</url>
  <licenses>
    <license>
      <name>provided without support or warranty</name>
      <url>http://www.json.org/license.html</url>
    </license>
  </licenses>

  <organization>
    <name>JSON</name>
    <url>http://json.org/</url>
  </organization>

  <dependencies/>
</project>

```

**Question:** What is exactly this file and is it possible to generate this file as part of a Maven build? Or do we have to create these files manually as part of the codebase ?

(In the context of what I am doing, I use generic Artifactory deployment plugin for Bamboo to deploy artefacts there but this ***.pom*** file is not being created automatically).

UPD I have solved my problem by using a copy of the pom.xml as this pom file and it works but I consider this can not be the correct way.

# Answer
Answer from here:

> Every jar needs to have a pom file describing it, you can just add something simple like this:

Every jar needs to have a pom file describing it, you can just add something simple like this:

http://maven.apache.org/maven-v4\_0\_0.xsd"\> 4.0.0 aGroup aArtifactId aVersion jar a Name
> 0 votes

---
Tags: maven, java
---


# Question
Title: How to configure a pull request quorum making an organizational constraint?
Consider the following situation.

Alice and Anna, Bob and Bart, and finally Eve and Evelyn work on same Git project in three different organizations.

How can they configure a quorum for pull requests, which requires *at least 2 votes but from two different organizations*?

Example:

* Votes from Alice and Anna - no quorum achieved possible as both votes come from the same organization;
* Votes from Alice and Bob - quorum has been achieved because team members from two different organizations have voted.

# Answer
Although I like the idea, based on what I know while working with Gitlab, Bitbucket and Github I do not believe that it is currently possible as I cannot remember that such integrations were available.

Also Googling: `github bitbucket pull request approvals integrated` did not return any positive results.

I think that the users should always be part of the same organization in order to approve Pull Requests. However, when I think about github I can remember that I was able to invite people as a collaborator even when they were not part or part of any organization.

In conclusion, I think that it is possible, but only in a certain git-server providers like GitHub. My suggestion is to try it and otherwise report an issue in the associate Github project.
> 1 votes

---
Tags: git, organization
---


# Question
Title: Git repo setup for multiple dot net project
I have a dot net solution containing a WPF application, MS-SQL database, and 2 dlls. Each is their own product in the sense that each can be shipped independently (dlls go to nuget to be consumed by other applications to talk to the database). They are independant of each other but are developed in conjunction with one another (changes made in one typically results in changes in another). The repos are used by VSTS build and release pipelines.

Should I set up a single git repo that contains all the projects or should i do a repo for each project? Should I use submodules? Are there any alternative solutions?

# Answer
This was solved by having a separate project/solution for the DLL as it is used by multiple projects. It has it's own life cycle and bug backlog. Allowing work to be scoped and tracked for that common library and feature requests to be added to the backlog to extend is shared capability
> 0 votes

# Answer
Only put a library in a separate repo when it is used in two different repos. If that is not the case then put the code in the same repo. Otherwise one could run into the anti-pattern nanoservices.
> 2 votes

---
Tags: git, azure-devops
---


title: Build user for Visual Studio Online

text:
# Question
Title: Build user for Visual Studio Online
Using visual studio online, do I need a separate user for the purpose of ferrying code/builds across machines. I understand that the built in service account builds code and can publish, but with things like Jenkins that require access tokens to retrieve source code (for projects like unity (no we are not able to use unity build)) I find it would be best to have a 'service' account where I can create personal access tokens that isn't tied to any one developer.

Do I need a 'build' user or is there another way to ferry code to Jenkins servers or using personal access tokens for deployment groups?

# Answer
> 0 votes
A separate build user was used so that all tasks that are performed by the build system has an account to preform them with. This way teams can still approve for releases to be deployed, but the build account is the account with the permissions to access the targeted environments.

# Answer
> 1 votes
In my opinion, there should always one key per user:

* Security: If an employee will leave the company then the key could be revoked and the person who left the company cannot trigger builds anymore
* Access-control and responsibilties: developers from team b are allowed to deploy apps to the production environment of team b, while members of team a are responsible for their own app and are restricted to environment a.

---
Tags: jenkins, continuous-deployment, build-pipeline, tfs
---


# Question
Title: nginx for dummies? How to enable Jenkins over https
I'm running Jenkins on an Ubuntu machine bare metal, and as I'm only experimenting and learning how the ops of it all works, I now want to learn how to enable https (with a nominally valid certificate obviously). I want to talk to a git instance and other tools and keep the automation tools secure. I believe the easiest is to use a reverse proxy, NGINX. Which I assume just acts like a forwarder; this is where I get stuck as a newbie.

I am unable to set up or follow the nginx initial steps to obtain a certificate, because the key generation tool (keytool) just does nothing when I give it a URL. It keeps saying my computer's FQDN is not valid.

I guess that's because my ubuntu machine is behind a NAT. I am guessing it's not realistic to create a trusted server on a private LAN which only has egress, but no ingress? Do I really need a real domain? I have one, but since this is work related, I'm loathe to create any kind of link-up between work and play. This feels a little workaround hacky-like, so keen for alternatives to what looks like an overkill configuration. For an internal experimental Jenkins setup, is there a 3 step process that a beginner who knows very little about keys can follow?

# Answer
> 3 votes
Your question is very broad, but I'll try to answer it.

1. You really need to read on what TLS/SSL is and how it works **before** you continue. You can find a bunch of questions in security.stackexchange.com that ask about this and many of them even link to RFCs where you can read the full detailed explanations of how it all works.
2. To configure nginx to work with TLS/SSL certificate you can use https://www.digitalocean.com/community/tools/nginx to generate the template. Just follow the guide and it should create a relatively secured vhost. If you would like to enhance the security even further - you can follow cipherli.st's guide. They update it relatively often so the proposed settings are usually up-to-date.
3. What makes a certificate "valid" or "invalid"? It's the `Certificate Authority`. It is possible to create a private CA certificate and use it to sign a `Certificate Signing Request` (or `CSR`). The `CSR` can then be signed by the `CA` and returned to whoever needs it (in this case the Nginx server). There are multiple tutorials for this available online and you can use `easy-rsa` to do the heavy lifting. However, the down side of doing it this way is that you need to go over all machines in the network and install the `CA` in browsers and all necessary keystores. Otherwise those machine will treat the site's certificates as "untrusted" because it will be coming from an unknown `Certificate Authority`. In some cases you can also create a `Certificate Bundle` for nginx.
4. If you had an ingress available you could have used `LetsEncrypt` to automatically issue certificates and skip complicating your setup by having `CA` and issuing certificates on your own.

Note: Image has been borrowed from ssl.com.

---
Tags: jenkins
---


# Question
Title: Running Automation on Google Cloud Virtual Instances using Ansible
I'm trying to automate google cloud virtual instances remotely only using external ip addresses of virtual machines. I can ssh into the virtual machines using command line with user name `shishir9159_gmail_com` . But If I use any ansible commands like this:

```
ansible -i hosts -u shishir9159_gmail_com --private-key=~/.ssh/google_compute_engine -m ping all

```

and it results in this following error:

```
"msg": "Failed to connect to the host via ssh: shishir9159@35.202.219.6: Permission denied (publickey,gssapi-keyex,gssapi-with-mic)."

```

I've used some parameters in my ansible.cfg:

```
host_key_checking = False
ssh_args = -o ControlMaster=no

```

But I don't think they do much of a help according to this post:
https://serverfault.com/questions/929222/ansible-where-do-preferredauthentications-ssh-settings-come-from

And I tried many methods and recommendations. I have a service account but it doesn't seem to me necessary for this simple ping command.

# Answer
> 0 votes
I solved the problem by adding `ansible_ssh_user` and `ansible_ssh_pass` at the hosts file. This post contain the solution. https://stackoverflow.com/questions/37213551/ansible-ssh-connection-fail?rq=1

# Answer
> 2 votes
You run the command with the user `shishir9159_gmail_com` but the error shows `shishir9159@35.202.219.6: Permission denied` ... seems like the `_gmail_com` portion of the username is truncated. Can you try setting the `ansible_user` config parameter in your inventory file or playbook rather than passing it on via the CLI?

---
Tags: ansible, google-cloud-platform, ssh
---


# Question
Title: Kops: How to delete the multiple SSHPublicKey
```
SSHPublicKey    admin   2b:xx:
SSHPublicKey    admin   2c:yy

```

How to remove one ssh key from the cluster ? when tried to delete via normal way,

```
kops delete secret --name cluster-1.dev.transein.com sshpublickey admin

```

found multiple matching secrets; specify the id of the key

How to remove one , or even the entire sshkey

# Answer
```
kops delete secret --name cluster-1.dev.transein.com sshpublickey admin 2c:yy

```
> 2 votes

---
Tags: kubernetes, kubectl, kops
---


# Question
Title: How to avoid an avalanche of images in Openshift when security patches keep coming in?
Our applications are based on tomcat, strictly speaking on tomcat base s2i image provided by RedHat. This is updated quite frequently, and we need to comply to some regulations and apply the updates. This basically means, that with every single update, we need to replace the base image of each of a few hundred ones deployed. There is about twenty something applications, each in a few different code versions, so cannot be simplified with tagging. I was thinking of extracting tomcat/JDK somehow to another supply chain, but there it gets complicated as well.

# Answer
**Get newest tag**

How I mitigated it was by creating several bots. The first one uses dip to get the latest version of a specific docker image. E.g.:

```
go run main.go -image ubuntu -latest "xenial-\d.*"

```

to get the latest tag of Ubuntu Xenial. If the docker image does not exist in your local docker registry, then it will be pushed.

**Update tag in several Dockerfiles**

The second bot, updates the tags in the Dockerfiles. If the build is successful then it is merged. Currently an approval is always required, but if it works for a couple of months and the team has enough confidence then an auto merge can be done.

**Update orchestration**

The third bot updates the docker-compose.yml files, k8s files or whatever orchestration is used. If an approval is done then it will be merged into master. Again if there is enough confidence then this could be automated as well, but I always prefer some manual intervention if production is applicable. Then the team is aware that there was an update and could revert if needed. On the other hand if the monitoring works well then the team will be notified right a way and if updates are done frequently, i.e. at least once every two weeks then the team is trained and knows how to deal with this.
> 0 votes

---
Tags: security, openshift, tomcat, images
---


# Question
Title: Coding conventions for docker-compose files?
Is there some globally admitted coding conventions for `docker-compose` files? Below is a sample `docker-compose.yml` from a small project we have here:

```
version: 3
networks:
  net:
    driver: 'bridge'

services:
  app:
    build: .
    networks:
    - net
    ports:
    - 3000:3000
  redis:
    image: redis:alpine
    networks:
    - net
  client:
    image: "app/test_clients"
    deploy:
      replicas: '3'

```

The quoting is especially inconsistent. Sometimes quotes are used around text and numerical literals. Sometimes they aren't. Sometimes single quotes are used, sometimes double quotes are used instead. The various example in the official docker-compose documentation seems relatively inconsistent in that area too.

This is valid YAML. And YAML is very permissive about quoting and, as far as I know, quotes could simply be omitted in most of the cases. I know this is probably a matter of internal policy but I wonder if there are some commonly accepted "best practices" in that matter.

# Answer
One could use Composerize to create a consistent docker-compose.yml.

There seems to be two options. Options one is to enter some docker run command, e.g. `docker run app/test_clients` on their website, which will return:

```
version: '3.3'
services:
    test_clients:
        image: app/test_clients

```

Or use the command line: `composerize docker run app/test_clients`.
> 3 votes

# Answer
You could also introduce a YAML linter to your CI to ensure that a consistent docker-compose.yml format will be enforced, like yaml-lint. According the docs it should be possible to define a certain format. If the pipeline fails, then the developer has to apply changes in order to fix the build.
> 0 votes

---
Tags: docker-compose, linting
---


# Question
Title: How to create replicas in docker-compose when `scale` is deprecated?
According to docker-compose issue #5586, `scale` is deprecated. The replacement is `deploy.replicas` but that latter is not handled by `docker-compose`, only by `docker stack` implying you have to run Docker in swarm mode.

As alternate solutions, you can scale services from the command line with docker-compose:

```
docker-compose up --scale app=3

```

I'm not too fond of this solution since in my understanding `docker-compose.yml` is supposed to be self-contained and should not depend on command line options.

Another option is to run docker-compose in `--compatibility` mode:

```
docker-compose --compatibility up

```

In that case, `docker-compose` will translate some `deploy` entries into their *version 2* equivalent. But because of the name of that option, and because of the warnings in the docs, it does not look like a mid- to long-term solution.

What would you suggest to replace the docker-compose *v2* `scale` option?

# Answer
> 3 votes
@decoomanj on github indicated that `scale` has to be replaced by `deploy` in conjunction with `replicas`.

According to mrampant-nist on github one has to use `--compatibility` as well.

Let's test this. First of all all docker-compose.yml is needed:

```
version: '3.3'
services:
    nginx:
      image: nginx
      scale: 6

```

and issuing:

```
docker-compose up

```

results in:

```
ERROR: The Compose file './docker-compose.yml' is invalid because:
Unsupported config option for services.nginx: 'scale'

```

Applying @decoomanj's suggestion, i.e. replacing replicas:

```
version: '3.3'
services:
    nginx:
      image: nginx
      deploy:
        replicas: 6

```

results in:

```
WARNING: Some services (nginx) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.

```

Running `docker-compose --compatibility up` starts 6 services:

```
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
f19aa54634d7        nginx               "nginx -g 'daemon of…"   5 seconds ago       Up 4 seconds        80/tcp              6233_nginx_5
749cff4ac1c3        nginx               "nginx -g 'daemon of…"   5 seconds ago       Up 3 seconds        80/tcp              6233_nginx_3
b86ed7b217fe        nginx               "nginx -g 'daemon of…"   5 seconds ago       Up 1 second         80/tcp              6233_nginx_6
0b92e5ef4a0e        nginx               "nginx -g 'daemon of…"   5 seconds ago       Up 2 seconds        80/tcp              6233_nginx_2
e414bca52b53        nginx               "nginx -g 'daemon of…"   5 seconds ago       Up 4 seconds        80/tcp              6233_nginx_4
e9f33dc634a8        nginx               "nginx -g 'daemon of…"   6 seconds ago       Up 5 seconds        80/tcp              6233_nginx_1

```

In conclusion, `replicas` has to be replaced by `deploy` and `replicas` and `--compatibility` is needed to solve the issue.

---
Tags: docker-compose
---


title: node js deployment on production server

text:
# Question
Title: node js deployment on production server
I wanted to know standard process to deploy nodejs app on production. As of now I am following the process as below

Pull the source code

```
git pull git_url

```

Install npm dependencies

```
npm install

```

Build the code

```
npm run build

```

run the application

```
nohup node server/index.js &

```

But I am thinking of pulling source code to prod is the totally wrong way. Please, someone, guide me the exact way to deploy node app on the production server. Also, any other way to run the app continuously as service. I am aware with pm2 module to run a process in the backend. If there is any other way to manage it properly.

# Answer
You could issue an `npm pack` which will create a tgz and then you could push it to an artifact manager like Nexus3. By doing this one uses the same approach as creating artifacts in Java like JARs and WARs. If you ensure that the artifacts cannot be overwritten, you specify a version and restrict publication to an artifact manager by the CI like jenkins then the artifacts are immutable and you are in control. E.g. if there is a bug in version X you could revert it to the previous version that was working well.

Note: if you are planning to use `npm pack` I would suggest to use an abbreviation like `@angular` does to ensure that there is a clear destinction between artifacts that were created by your company and libraries from npmjs.org. This could be done by adding an abbrevation of your company like `@z` to the package name. A name could look like: `@z/some-package`.
> 1 votes

---
Tags: cicd, github, node.js, artifacts
---


# Question
Title: Kubernetes auto scaling in cloud environment
Anyone knows if there is a solution to provision (non-managed) autoscaling Kubernetes clusters in the cloud?

I am looking for a generic solution that will work in any public cloud and in private data centers.

After quite a bit of research, the known available options do not answer the requirement above -

* `kops` which is AWS specific is using own AMI images.
* `kubespray` (aka kargo) has no notion of autoscaling, executes Ansible on a specific inventory of hosts.
* `kubeadm` expects a known list of hosts and does not take care of OS setup like docker, kubelet, etc...
* `kubernetes autoscaler` controls existing auto scaling groups, does not prepare the images used by those groups.

Is there some generic solution to make an autoscaling group deploy Kubernetes cluster on itself? Possibly a way to build cloud images (like AMI in AWS) using Packer for this.

Looking for a method that is not specific to one public cloud and allows to use an elastic environment that scales automatically based on prepared images.

# Answer
Recently Kubernetes started preview work on https://cluster-api.sigs.k8s.io which is meant to be the standard way to solve the way how High Availability clusters are deployed.

The `kubeadm` tool has also been updated with features to deploy multiple masters since this question was first asked, which help tremendously. With the exception of deploying cloud-specific configuration, the documentation for `kubeadm` configuration YAML file brings you quite the way towards having a properly deployed cluster.

docs for `kubeadm`: https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/
> 1 votes

---
Tags: kubernetes, autoscaling
---


# Question
Title: Docker - Mount a volume from a container to an other (equivalent volumes_from) in docker-compose 3
I've two containers : **nginx** & **angular**. The **angular** container contains the code and is automatically pulled from the registry when there is a new version (with watchtower).

I set up a Shared Volume between **angular** & **nginx** to share the code from **angular** to **nginx**.

```
  ### Angular #########################################
  angular:
    image: registry.gitlab.com/***/***:staging
    networks:
      - frontend
      - backend
    volumes:
      - client:/var/www/client

  ### NGINX Server #########################################
  nginx:
    image: registry.gitlab.com/***/***/***:staging
    volumes:
      - client:/var/www/client
    depends_on:
      - angular
    networks:
      - frontend
      - backend

volumes:
  client:
networks:
  backend:
  frontend:

```

When I build & run for the first time the environment, everything works. The problem is when there is a new version of the **client**, the image is pulled, the container is re-built and the new code version is inside the **angular** container, but in the **nginx** container it still the old code version of the **client**.

The shared volumes does not let me do what i want because we can not specify who is the host, is it possible to mount a volumes from a container to an other?

# Answer
> 1 votes
**The Problem**: Named volumes are initialized on container creation when the volume is empty, and from the first container that starts up using that volume. That initialization includes all files, directories, and file metadata like owners and permissions. If there is content in the named volume, docker will not initialize it again since that would defeat the purpose of persistence.

**Option A**: When you have multiple containers sharing a volume to share updated build time content like you have, my first suggestion is to modify the build to place the relevant data in each image. Depending on what is being shared and between what images, you may be able to use shared image layers to avoid ever needing to copy the data.

**Option B**: When placing the data in each image and avoiding the shared volume is infeasible, then you likely want a process to copy data from the image into the volume as part of your container entrypoint. This has two advantages. First it works when the containers startup in either order, so you can eliminate dependencies between container startup order (though you may find it necessary to include some wait step in the other container entrypoints if they fail when run before the data is copied). And second, your copy can update the volume when existing data is already there.

I've implemented a `load-volume` and `save-volume` pair of scripts in my docker base image that you can reuse in your own images to implement this second option. I typically ran into this issue when using tmpfs volumes to support read-only containers, but the scripts have been made generic enough to support your use case. Note that your scenario likely involves deleting existing data from the volume, and I've implemented that using `rsync` for performance, so that needs to be installed in your image to work.

# Answer
> 0 votes
You could use `named volumes`, see https://docs.docker.com/compose/compose-file/ instead of `bind mounts` like you are doing in the example. The difference is that the `named volume` is a docker volume that could be found by issuing `docker volumes ls` and remove by issuing `docker-compose down -v`. The bind volume starts with a dot, e.g. `./some/mount` and is a directory on the file system.

The named volume could be used to share information between two containers.

---
Tags: docker, docker-compose, volume
---


# Question
Title: Understanding Vagrant virtual machine import
I'm building my Windows server 2012 vm with packer and when I'm doing a `vagrant up` it's failing because the folder name already exists…

Here is my json file for packer

```
{
  "builders": [
    {
      "boot_wait": "2m",
      "communicator": "winrm",
      "disk_size": "{{user `disk_size`}}",
      "floppy_files": [
        "{{user `autounattend`}}",
        "./scripts/disable-winrm.ps1",
        "./scripts/enable-winrm.ps1"
      ],
      "guest_additions_mode": "disable",
      "guest_os_type": "Windows2012_64",
      "headless": "{{user `headless`}}",
      "iso_checksum": "{{user `iso_checksum`}}",
      "iso_checksum_type": "{{user `iso_checksum_type`}}",
      "iso_url": "{{user `iso_url`}}",
      "shutdown_command": "shutdown /s /t 10 /f /d p:4:1 /c \"Packer Shutdown\"",
      "type": "virtualbox-iso",
      "vboxmanage": [
        [
          "modifyvm",
          "{{.Name}}",
          "--memory",
          "2048"
        ],
        [
          "modifyvm",
          "{{.Name}}",
          "--nic1",
          "NAT"
        ],
        [
          "modifyvm",
          "{{.Name}}",
          "--cpus",
          "2"
        ]
      ],
      "vm_name": "windows_2012_r2",
      "winrm_password": "vagrant",
      "winrm_timeout": "{{user `winrm_timeout`}}",
      "winrm_username": "vagrant"
    }
  ],
  "post-processors": [
    {
      "keep_input_artifact": false,
      "output": "vagrantenv/windows_2012_r2_{{.Provider}}.box",
      "type": "vagrant",
      "vagrantfile_template": "vagrantfile-windows_2012_r2.template"
    }
  ],
  "provisioners": [
    {
      "scripts": [
        "./scripts/vm-guest-tools.bat",
        "./scripts/enable-rdp.bat"
      ],
      "type": "windows-shell"
    }
  ],
  "variables": {
    "autounattend": "./answer_files/2012_r2/Autounattend.xml",
    "disk_size": "61440",
    "disk_type_id": "1",
    "headless": "true",
    "iso_checksum": "5b5e08c490ad16b59b1d9fab0def883a",
    "iso_checksum_type": "md5",
    "iso_url": "./iso/windows_2012_r2.iso",
    "winrm_timeout": "30m"
  }
}

```

And my `post-processors` template

```
# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.require_version ">= 1.6.2"

Vagrant.configure("2") do |config|
    config.vm.box = "windows_2012_r2"
    config.vm.communicator = "winrm"

    # Admin user name and password
    config.winrm.username = "vagrant"
    config.winrm.password = "vagrant"

    config.vm.guest = :windows
    config.windows.halt_timeout = 15

    config.vm.network :forwarded_port, guest: 3389, host: 3389, id: "rdp", auto_correct: true

    config.vm.provider :virtualbox do |v, override|
        #v.gui = true
        v.name = "windows_2012_r2"
        v.customize ["modifyvm", :id, "--memory", 2048]
        v.customize ["modifyvm", :id, "--cpus", 2]
        v.customize ["modifyvm", :id, "--vram", 128]
        v.customize ["modifyvm", :id, "--clipboard", "bidirectional"]
        v.customize ["setextradata", "global", "GUI/SuppressMessages", "all" ]
        end
end

```

This steps works porperly and I end up with a file `windows_2012_r2_virtualbox.box`

Then I run `vagrant init -f windows_2012_r2 windows_2012_r2_virtualbox.box`

and finally `vagrant up`

following this I'm getting the error:

```
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Box 'windows_2012_r2' could not be found. Attempting to find and install...
    default: Box Provider: virtualbox
    default: Box Version: >= 0
==> default: Box file was not detected as metadata. Adding it directly...
==> default: Adding box 'windows_2012_r2' (v0) for provider: virtualbox
    default: Unpacking necessary files from: file:///D:/Dev/packer-windows/vagrantenv/windows_2012_r2_virtualbox.box
    default:
==> default: Successfully added box 'windows_2012_r2' (v0) for 'virtualbox'!
==> default: Importing base box 'windows_2012_r2'...
==> default: Matching MAC address for NAT networking...
==> default: Setting the name of the VM: windows_2012_r2
The name of your virtual machine couldn't be set because VirtualBox
is reporting another VM with that name already exists. Most of the
time, this is because of an error with VirtualBox not cleaning up
properly. To fix this, verify that no VMs with that name do exist
(by opening the VirtualBox GUI). If they don't, then look at the
folder in the error message from VirtualBox below and remove it
if there isn't any information you need in there.

VirtualBox error:

VBoxManage.exe: error: Could not rename the directory 'D:\VM\VirtualBox\windows_2012_r2_1556359923004_28925' to 'D:\VM\VirtualBox\windows_2012_r2' to save the settings file (VERR_ALREADY_EXISTS)
VBoxManage.exe: error: Details: code E_FAIL (0x80004005), component SessionMachine, interface IMachine, callee IUnknown
VBoxManage.exe: error: Context: "SaveSettings()" at line 3194 of file VBoxManageModifyVM.cpp

```

I notice that at the stage

* `Successfully added box 'windows_2012_r2' (v0) for 'virtualbox'!` vagrant creates the folder **windows\_2012\_r2** in the virtualbox folder and then at this stage
* `default: Importing base box 'windows_2012_r2'...` vagrant creates a folder **windows\_2012\_r2\_1556359923004\_28925** and try to rename it to **windows\_2012\_r2** but doesn't manage to it because it already exist.
* **windows\_2012\_r2** folder has the *windows\_2012\_r2-disk001\_2.vmdk* file
* **windows\_2012\_r2\_1556359923004\_28925** has windows\_2012\_r2\_1556359923004\_28925.vbox file

I'm confused how vagrant manage the Virtualbox name when importing a packer created box why vagrant is renaming the folder name, and not using the already existing target folder?

# Answer
> 2 votes
It looks like that there is a name conflict:

```
The name of your virtual machine couldn't be set because VirtualBox
is reporting another VM with that name already exists. Most of the
time, this is because of an error with VirtualBox not cleaning up
properly. To fix this, verify that no VMs with that name do exist
(by opening the VirtualBox GUI). If they don't, then look at the
folder in the error message from VirtualBox below and remove it
if there isn't any information you need in there.

```

According to @xofer on Github on has to set the `name`. However in the example you provided in conjunction with the logging, the issue seems to be caused by a VM that was created and still exists in Virtualbox. In other words one has to ensure that the VM has been removed or another name has to be defined to prevent this collision.

---
Tags: vagrant, packer, virtualbox
---


# Question
Title: Trade offs of putting an international master database in us-west-2
I'm hoping this isn't so open ended that it veers towards a "discussion" more than an "answerable" question.

I need to place an RDS Postgres instance in an AWS region that will act as the master DB for writes from all other regions. Those regions will include Australian and European regions, as well as North American.

My concern is latency on writes. I will use replicas for reads, so that is not so much a concern.

I will be using interregion VPC peering for all regions to be able to communicate with the RDS instance.

I'm thinking that placing the RDS server in `us-west-2` as it seems to be in the most "central" spot for all regions of concern.

I understand that the AWS "backbone" is very fast, and perhaps fast enough not for this to be a concern.

Has anyone dealt with this problem and have any metrics or other ways to quantify a decision on where to place this RDS instance?

# Answer
> 1 votes
It's been a while since posting the question, and a comment I made above seems to be helpful despite the fact it's using a quantitative assessment, so I'll post it here as an answer.

**As per comment above:**

@030 I can offer my experiences since asking the question, albeit it's a subjective view as I have no quantitative data. I can offer that a year and a half later, I haven't regretted my decision putting the master into us-west-2. Our users in any region don't seem to be impacted by having the master in us-west-2, at least from a UX perspective. I suppose at some point it could be worth measuring write times -- but to date I haven't had to. There's no business impact from any latencies due to the location of the master, that we can detect at least, so we're happy here

---
Tags: amazon-web-services, databases, amazon-rds
---


# Question
Title: Commit messages from Submodules to Jenkins Change Log
I have a project with 3 submodules that are updated whenever that project is built and the project is built mainly for an installer (which is not in a submodule). Not a lot of changes are made in the installer itself but the commit messages from there are coming through fine.

I'd like to also see the commit messages (changes made) from the submodules to the Jenkins log. Is this possible? If not possible there, would it be possible in the email-blast instead? In the mailer plugin, I have it configured as:

```
${CHANGES, showPaths=true, format="Commit: %r by %a \n %p \n\t Comment: %m \n", pathFormat="\n\t %p"}

```

Also, I kinda inherited Jenkins from an employee that no longer works where I work so I'm very new to this.

# Answer
> 1 votes
After consulting this cheatsheet, I think that you are looking for the subject. If I apply:

```
git log --pretty="%s"

```

to one of my git repositories, the following is returned:

```
learning pointers in golang
receiver struct in golang
embedded structs in golang
multiple spanish words

```

In conclusion, I think you should include `%s` to get the git messages or basically the `commit subjects`.

---
Tags: jenkins, git, jenkins-plugins
---


title: Rundeck ignoring node

text:
# Question
Title: Rundeck ignoring node
I'm trying execute a job on know node by rundeck server, but when execute it was ignored and job run on server local (Rundeck Server). Anyone went through this situation?

* Rundeck 3.1.3-20191204
* Node configured in Project Setting
* It's possible see the host in Node menu of Project
* Rundeck server can access via ssh the node.

# Answer
to solve, after make all setup to Rundeck know and access de node you need go to Project Setting \> Edit Configuration \> Default Node Executor and change to SSH. The problem occured because my project was created like LOCAL default node executor.
> 0 votes

---
Tags: rundeck
---


# Question
Title: What about installing an Antivirus on Windows K8s nodes?
We're building an on-premise Kubernetes cluster with Windows nodes. As a company policy, all Windows Servers needs an antivirus (symantec endpoint security).

The antivirus would be installed on the nodes, and not as a pod.

What do you think about installing an antivirus on Kubernetes nodes? Considering the great chance of false-positives when an antivirus has to deal with Docker.

# Answer
> 3 votes
I don't have much experience with Symantec (I don't really like the product). But searching on Google apparently has some steps to set up, as they don't natively support Kuberentes as an application.

Look if these two articles can help you.

https://support.symantec.com/us/en/article.tech246815.html

https://www.symantec.com/content/dam/symantec/docs/solution-briefs/securing-containers-think-your-containers-are-secure-en.pdf

---
Tags: docker, kubernetes, windows
---


# Question
Title: parameter-store values show '***' instead of actual value
I am using CodeBuild to access the Systems Manager Parameter Store. In buildspec.yml, I have the parameter-store configured to access some values in Systems Manager (I certainly have the /mysite/dev/mongodb\_uri created in parameter store):

```
version: 0.2

env:
  variables:
    MONGO_URI: 'mongodb+srv'
  parameter-store:
    MONGODB_URI: /mysite/dev/mongodb_uri

phases:
  pre_build:
    commands:
      - echo $MONGO_URI
      - echo $MONGODB_URI

```

Unfortunately, while CodeBuild will echo out the value for key in the variables object, it will not for the parameter-store object:

```
[Container] 2019/12/24 03:16:09 Running command echo $MONGO_URI
mongodb+srv

[Container] 2019/12/24 03:16:09 Running command echo $MONGODB_URI
***

```

It just gives me 3 stars. If I use aws cli, I get the right value:

```
$ aws ssm get-parameters --names "/mysite/dev/mongodb_uri"
{
    "Parameters": [
        {
            "Name": "/mysite/dev/mongodb_uri",
            "Type": "String",
            "Value": "mongodb+srv://myuser:mypassword@mysite-cluster-n1qub.mongodb.net/test?retryWrites=true&w=majority",
            "Version": 2
        }
    ],
    "InvalidParameters": []
}

```

My CodeBuild Service Role policies grant me access to Systems Manager:

```
{
        "Effect": "Allow",
        "Action": [
            "ssm:GetParameters",
            "ssm:DescribeParameters",
            "ssm:GetParameter",
            ...
        ],
        "Resource": [
            "arn:aws:logs:us-east-2:880783130023:log-group:/aws/codebuild/mysite-dev",
            "arn:aws:logs:us-east-2:880783130023:log-group:/aws/codebuild/mysite-dev:*",
            "arn:aws:s3:::codepipeline-us-east-2-*",
            "arn:aws:ssm:us-east-2:880783130023:parameter/mysite/dev/*",
            "arn:aws:codebuild:us-east-2:880783130023:report-group/mysite-dev-*",
            "arn:aws:ssm:us-east-2:880783130023:*"
        ]
    }

```

So why does it give me '\***' instead of the actual value when I echo the parameter in buildspec?

# Answer
> 6 votes
The "\***" is the masking of parameter store output in your log file. The value is available in the environment variable $MONGODB\_URI and can be used to pass along to any command or script that needs this value. CodeBuild is merely doing a best-effort masking of log statement which may contain sensitive information.

Also, if you must see the value printed out, you may assign $MONGODB\_URI to another variable within your buildspec and print that out.

---
Tags: amazon-web-services, continuous-delivery
---


# Question
Title: Which problems does a container orchestration solve?
Every company who has anything to do with containers these days, including public cloud providers, is creating their own solution for container orchestration.

These include:

What are the common problems with containers that must be solved by these orchestration solutions? What is the benefit of adding such a complex system in your production?

# Answer
> 5 votes
If you are only running a single container or two containers together you are correct in that an orchestrator may be unnecessary and add unneeded complexity. However, these tools do solve several issues when you running several containers together (especially in production).

> What are the common problems with containers that must be solved by these orchestration solutions

Containers run in an isolated process (usually in it's own namespace). This means that by default the container will not be aware of other containers. Additionally, it will not be aware of the systems files, network interfaces, and processes. While this can greatly help with portability of the software it does not solve several production issues such as microservices, container discovery, scalability, disaster recovery, or upgrades.

> What is the benefit of adding such a complex system in your production?

Adding a container orchestrator can greatly *reduce* the complexity in production as these tools are designed to resolve the issues outlined above. For example, Kubernetes is built to allow containers to be linked together, deploy containers across an entire network, scale and load balance the network based on container resource consumption, and allow upgrades of individual containers with no downtime.

---
Tags: containers, orchestration
---


# Question
Title: Overcommiting CPU Resources in Kubernetes
Let’s assume the following scenario:

I have a cluster with 100 CPU cores and 100 Pods each having a request of 1 CPU.

If I schedule the 101st Pod if a request of 1 CPU this will no longer be schedulable, as per my understanding.

Now, further assume that I as cluster operator have observed that pods on average only consume .3 CPU in reality. It would be beneficial if scheduling further pods instances was possible.

Can I somehow apply a global correction factor that disregards CPU requests to some degree and allows me to overcommit the available resources?

# Answer
You should be able to use the flag `spec.containers[].resources.limits.cpu` to set your desired state on a deployment.

The official docs here and here have some examples of how to specify this in the yaml. You can set a global default along with min/max limits:

```
spec:
  limits:
  - default:
      cpu: 1000m
    defaultRequest:
      cpu: 300m
  - min:
      cpu: 300m
  - max:
      cpu: 1000m

```
> 1 votes

---
Tags: kubernetes
---


# Question
Title: How to develop a docker container for laravel without laradock or docker-compose
I am very new to docker. I want to build a `laravel` development environment with `docker`. I already tried `laradock` but it seemed complicated for me. I was wondering whether it is possible to create a `laravel` development environment without `laradock` or `docker-compose`

# Answer
> 1 votes
Yes. You could create each component of your development environment using containers and manually network them together. This approach would require you to manually start each container and specify the network and volumes they use at runtime.

However, you are adding un-needed complexity by avoiding docker-compose to network your containers together. By using a compose file you can start your environment with a single command: `docker compose up`. In the approach outlined above, you would need to manually re-create the containers *every time* you wished to recreate the environment.

Laradock's website has some simple getting started guides that can get you up and running without a full understanding of Docker. While the learning curve may be steep, it can be beneficial to avoid re-implementing a viable solution.

---
Tags: docker, docker-compose, containers, dockerfile
---


# Question
Title: How to avoid wasting time/resources in a CI/CD workflow?
Ideally, a team should be able to run all the pipelines on every commit and automatically deploy to production.

However, I don't understand how to most effeciently use our resources so that hours are wasted running tests on code which is about to be overwritten by the next commit.

If a build or test takes 20 minutes to go through the pipeline, the system gets "clogged".

What mechanism is used to solve this problem? Do tests/jobs get canceled as new code gets pushed? If so, then how do you deploy multiple times a day? Add to this the need for Pull requests or sign off. What intermediate steps can be taken until it's possible to automate everything?

# Answer
We break ours down into stages, and each stage pulls, rather than pushes.

Our build takes 20 minutes. So if I commit once a minute, I will still only get three builds an hour, but each build will contain 20 commits.

The tests then pull on the latest successful builds and run them, and the same for the deployment scripts.

The main point is that we do NOT build every commit individually. We do NOT test every build individually.

I probably got all this from Continuous Delivery by Jez Humble and David Farley.
> 5 votes

# Answer
The typical CI/CD pipeline is normally triggered for a branch by new versions being created on that branch, which are the **results** of branch merges, either manual or automated by some tools, like PRs (or of direct commits), the traditional CI tools don't need to do any code merging.

Once triggered, the pipeline execution attempts to complete as many steps as possible, provided, of course, that the necessary resources (build servers, testbeds, etc.) are available. If the pipeline is triggered again because of new code changes coming in before the previous execution(s) complete you'd just have new pipeline runs, in parallel with/overlapping the ones in progress. Not a problem. Whichever run is successful will perform a deployment, regardless of how much time passed since the previous one.

The runs already in progress are not waste, they still provide crucial information about the quality of the changes that got integrated into the branch, which is, after all, the primary goal of CI/CD. Stopping a particular pipeline run N because N+1 is triggered would prevent you from immediately identifying the culprit if the N+1 run fails - you can't tell if it's because of the N or the N+1 commit. You'd also unnecessarily delay deployment of the N commit (if its run would have been successful) - you now need to either rollback or fix the problem and do at least one more full pipeline run to deploy it.

The clogging you mentioned only happens if you don't have enough resources to meet the demand. Mostly a planning matter. If your resource deficit is very small your pipeline may still function close to ideal, with just small delays during peaks of activity but catching up during idle/low activity periods.

If the deficit is large and the pipeline indeed doesn't keep up with the rate of the incoming changes you'll be forced to *diverge from the CI way*, with multiple changes being picked up in a run, thus losing the ability to trivially identify the changes causing regressions. The longer it takes to identify the culprits, the longer your branch stays blocked, the slower your overall average pipeline speed gets. Fundamentally this is no different than running your pipeline only nightly, for example, the only difference is just the number of changes picked up in every run.

As for "automate everything", I'm afraid that's just a dream in traditional CI - even if done well it can only indicate when regressions in the branch quality occurred, but human intervention is still required for repairing and unblocking the branch. AFAIK only the gating commit CI systems can be entirely automated, but those are different beasts.
> 1 votes

---
Tags: continuous-integration, continuous-deployment
---


# Question
Title: Puppet directory sync does not delete absent files
Repeating my question from SO.SE here ...

I'm trying to have my configs directory (on Windows server 2016 node) to be synced from puppet master (puppet --version = 5.5.3) modules folder: I have my desired structure repeated under module/files. This:

```
file { "c:\\":
        ensure => directory,
        recurse => remote,
        source => "puppet:///modules/configs"
        }

```

works for files that exist in source dir, but does not delete files that are deleted from source (from docs I understood that it should) also I tried:

```
file { "c:\\":
        ensure => directory,
        recurse => true,
        purge => true,
        source => "puppet:///modules/configs"
        }

```

but then I get an error:

> Error: /Stage\[main\]/Main/Node\[nodename\]/File\[c:\]: Failed to generate additional resources using 'eval\_generate': CreateF ile(c:/Documents and Settings, 20000000000, 1, , 3, 210000000, 0): Access is denied.

I was not able to find even descent source telling possible cause for the error; Access denied makes no sense as puppet agent runs with system privileges and should be able to access every path, also script should have nothing to do with c:/Documents and Settings path to my understanding.

Any help to solve this?

# Answer
> 1 votes
According to https://tickets.puppetlabs.com/browse/PUP-4759 the issue seems to be solved.

---
Tags: windows, puppet
---


# Question
Title: Azure DevOps - Conditionally Build and Deploy
Say we have 6 microservices in a monorepo, each with their own CI build pipeline that is currently only building each project and running unit tests. Is there a way to identify which projects have had code changes since the previous release and only build and deploy those, leaving others untouched? I've not seen anything in the documentation examples that describe this as a possibility but I'm hoping that with all of the conditional and scripting abilities in YAML and possibly templates, never used them, that I could achieve this.

One way forward without this would be to manually comment out the services that I know had no changes in the YAML and commit that, then build from the updated definition. This is open to human error so I'd like to increase the level of automation.

Builds - Get Changes Between Builds Something like this looks interesting but I'm not sure how I'd integrate it with YAML

The goal of all of this is to only version stamp and deploy what has changed. I may be missing some core DevOps concepts here and am happy to read more if you'll guide me.

# Answer
> 2 votes
You can create build pipeline triggers to trigger your builds based on branch along with path triggers. In your release pipeline, you can set triggers and filters to decide which artifacts to be deployed.

---
Tags: continuous-integration, azure-devops, continuous-deployment, azure-pipelines
---


# Question
Title: Running task or role before any playbooks
Is there a way to run a specific task or role at the very beginning of every `ansible-playbook` run?

I'm looking for something that would go into `ansible.cfg` or something like that, so all playbooks using that file would run a specific task at the beginning.

My intention is to use it for doing some verifications: for example, "*is the local host listed on the inventory? If not, perhaps I'm using the wrong inventory*".

I could `import` or `include` such a role in every playbook, but I want it to be automatic, and to work even for playbooks that don't know about that role.

# Answer
What you are looking for is the `meta/main.yml` in the roles of Ansible.

So make your every `playbook` (the ones you want to run after a task/role) as a `role` itself (to make it secondary execution) and what ever task/role you want to run before it should be added to the `meta/main.yml` of all the secondary execution roles. This would make all your roles execute the role mentioned in `meta/main.yml`
> 1 votes

---
Tags: ansible
---


title: How to use ecs service dicovery

text:
# Question
Title: How to use ecs service dicovery
I'm trying to use amazon service discovery to register a Redis server in order to be reachable by other **ECS** services in the same cluster.

I'm using **EC2** to deploy my services not Fargate, so service discovery create a simple **SRV** DNS record, that's mean my others services must request **R53** service to get a real "**A**" record to have access to Redis service.

My question for DevOps/AWS engineer is: what's the correct way to do so? A request to **R53** each time absolutely not a good idea.

My solution is in comments.

# Answer
The goal can be achieved via several solutions. We can use service with **awsvpc** as network mode, so your service can get a real "**A**" **DNS** record and ENI interface, you must pay attention with this solution, ec2 instances has a limited number of ENI that can be attached, for a micro or small I think max 3.

Other solution more flexible is to use an internal **NLB** or custom proxy like **HAproxy** as **ambassador**, you run the proxy as service with awsvpc so he can get his "**A**" **DNS** record eg **proxy.prod**, all other services like Redis, Mongo, Elasticsearch ... can be configured as service with bridge networking mode so they will get an **SRV** record if configured with service discovery.

Finally you have to configure your proxy to redirect all traffic to desired service using port forwarding. **HAproxy** can resolve "**SRV**" records so you don't have to deal with that **HAproxy** will do it for you, you must just configure your app to use **proxy.prod** \+ port, for each service so:

* *Redis* **proxy.prod:6379**
* *Mongo* **proxy.prod:27017**
* And so on...

Using this technique you can do a lot of amazing things, no matter where your proxy can lives, it can run as container in the same node with your apps or on a **Fargate** or in another node ...
> 2 votes

---
Tags: amazon-ec2, amazon-ecs, fargate
---


title: Jenkins Plugin Install Failed

text:
# Question
Title: Jenkins Plugin Install Failed
I am trying to set up new Jenkins. When I download Suggested Plugins OR try to install any plugin I am getting below error. I am using Jenkins Latest **Version 2.204.1**. I have set up this on **VM**. I am getting this error for all the plugins even when I am trying to install suggested plugins during the initial setup, that time also I am getting the same error. I am installing Jenkins on Microsoft Windows Server VM. I have downloaded Jenkins from Browser, extracted zip file and installed it using jenkins.msi.

    at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
    at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
    at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
    at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
    at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
    at java.net.PlainSocketImpl.connect(Unknown Source)
    at java.net.SocksSocketImpl.connect(Unknown Source)
    at java.net.Socket.connect(Unknown Source)
    at sun.net.NetworkClient.doConnect(Unknown Source)
    at sun.net.www.http.HttpClient.openServer(Unknown Source)
    at sun.net.www.http.HttpClient.openServer(Unknown Source)
    at sun.net.www.http.HttpClient.<init>(Unknown Source)
    at sun.net.www.http.HttpClient.New(Unknown Source)
    at sun.net.www.http.HttpClient.New(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.getHeaderField(Unknown Source)
    at java.net.URLConnection.getHeaderFieldLong(Unknown Source)
    at java.net.URLConnection.getContentLengthLong(Unknown Source)
    at java.net.URLConnection.getContentLength(Unknown Source)
    at hudson.model.UpdateCenter$UpdateCenterConfiguration.download(UpdateCenter.java:1194)
Caused: java.net.SocketTimeoutException: connect timed out
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
    at java.lang.reflect.Constructor.newInstance(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection$10.run(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection$10.run(Unknown Source)```[![enter image description here][1]][1]

  [1]: https://i.stack.imgur.com/2TgUP.png

# Answer
> 1 votes
You can try 3 different things-

1. Try pinging https://updates.jenkins-ci.org/download/plugins/ and see if it gets any response. If it doesn't respond then make sure your internet is working fine.
2. You can try installing the plugins by connecting to a VPN.
3. Lastly, you can reinstall Jenkins to see if the error persists.

# Answer
> 0 votes
I have to reinstall the Jenkins 2-3 time and it starts working, but at an initial level still, I am not able to install suggested plugins. I have setup Jenkins first and then installed all the plugins manually.

---
Tags: jenkins, jenkins-plugins, windows
---


# Question
Title: How to pin OS package versions in Docker image
**Scenario**

For obvious reasons I believe that every package installed in a Docker image should have their version nailed down. That's why in our `Dockerfile` we always enforce the OS package manager to install a specific version of a package. For example:

```
FROM node:6.10.0

RUN apt-get update \
      && apt-get -y install nginx=1.6.2-5+deb8u6 \
      && apt-get -y install supervisor=3.0r1-1+deb8u1

```

**Problem**

The upstream repository has changed and the `nginx=1.6.2-5+deb8u6` package is now invalid.

I'm not sure what brings about this change but I think it has something to do with distros going out of their support-cycle or that old packages are removed/moved somewhere else.

**Question**

How do I nail the exact version of OS-leveled packages without running into this problem?

*Note*: I know these 2 options will work but I want to know if there's an easier way out:

1. Make a custom base image
2. Download the binaries and `COPY` them into the Docker image

# Answer
I'm not sure that this qualifies as an "easier" way, but you could host the specific packages on a server (e.g., Artifactory, S3). You would then have the Dockerfile connect to this location and use the packages from there.

This would allow you to avoid creating a custom base image and you would not have to manually download the packages.
> 2 votes

---
Tags: docker, dockerfile
---


# Question
Title: Why is Rails (with Nginx/Passenger via Ansible) unable to see my RAILS_MASTER_KEY environment variable?
I'm writing Ansible playbooks to provision an Ubuntu 18.04 server.

I want to provision the server for a Rails application using Passenger and Nginx. I used geerlingguy.passenger from Ansible Galaxy to configure Passenger and Nginx.

In addition, I used zzet.rbenv to install ruby 2.5.3.

The Ansible playbooks seem to have executed correctly, and when I log into the server via SSH, everything seems to be working (rbenv is on the path, `RAILS_MASTER_KEY` environment variable is defined).

Deployment of my application via Capistrano also appears to run successfully.

However, when I attempt to visit the landing page, I get a Passenger error: "Web application could not be started by the Phusion Passenger application server."

Looking in `/var/log/nginx/error.log`, I see:

```
Missing encryption key to decrypt file with. Ask your team for your master key and write it to /var/www/releases/20191230212823/config/master.key or put it in the ENV['RAILS_MASTER_KEY'].

```

I'm not sure why the above error is happening, because I've already configured a persistent environment variable using the weareinteractive.environment role.

Inspecting /etc/environment confirms that RAILS\_MASTER\_KEY is defined, and I can see if it I log in manually as my 'deploy' user. However, `/etc/nginx/nginx.conf` shows that the default user is `www-data`:

```
user www-data;

```

I think this should be fine, because the environment variables are configured in a way that is not associated with a user account. So, why does Rails fail to see this environment variable even though I see it when I log in via SSH?

I'm able to execute the following command on the server and see the contents of the encrypted credentials file:

```
RAILS_ENV=production EDITOR=vim bundle exec rails credentials:edit

```

So what is going wrong when Nginx/Passenger launch Rails?

Edit: It seems like Passenger/Nginx do not load environment variables from /etc/environment, which mostly explains my problem. I'm trying to figure out the appropriate way to configure environment variables for Passenger using Ansible now.

# Answer
Root cause: environment variables from `/etc/environment` are not passed to Passenger. The standard method seems to be passenger\_env\_var in your nginx config as suggested by user **joshaidan**.
> 0 votes

---
Tags: ansible, nginx, environment-variables, ruby-on-rails
---