title: Cloud service for Django app deployment

text:
# Question
Title: Cloud service for Django app deployment
We are having webapp which is use Django and Mysql which run in Windows using Apache. Now we are planing to move into cloud service like

1) Google cloud

2) AWS

3) Azure

So to deploy existing Django and move MySQL into database into cloud which one is easy tool from above 3 list. Or what is the approach that I can follow move my existing app into cloud

# Answer
The “easiest” approaches to deploying such app is probably using a Platform as a Service (*PaaS*). Here are some of them to get you started, pick any, really:

To me the first three are the easiest, but it all eventually boils down to personal preference.
> 1 votes

# Answer
Asking which service is the "easiest" is very subjective and I'm biased towards AWS since that is the service that I got most of my certification from. In the end, you should play around with each and make your own decision.

With AWS, the best tool would be Elastic Beanstalk. Elastic Beanstalk is great for deploying easily while allowing customization.

Elastic Beanstalk with Django: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create-deploy-python-django.html

For your database, AWS allows migration from an external MYSQL database to their RDS service: https://docs.aws.amazon.com/dms/latest/sbs/CHAP\_MySQL2Aurora.html#CHAP\_MySQL2Aurora.mysqldump
> 0 votes

---
Tags: amazon-web-services, azure, google-cloud-platform
---


# Question
Title: How to check minimum code coverage in pull request changes?
I want to force the developers to write unit tests for all new code they wrote. The code is written in python. I want to do it in the Jenkins pipeline. And here is a question. Is there any available Jenkins plugin that can do that? Or python library? I want to avoid sonar qube because I think, it's a little bit complex solution for the problem. I was thinking about a solution, that will detect changed files and check their code coverage percentage in generating a report.

# Answer
> 1 votes
I found a solution to my problem. There is a library `diff-cover` which does exactly what i need. Here is a link -\> https://diff-cover.readthedocs.io/en/latest/README.html

# Answer
> 0 votes
Many Python packages exist for that purpose, one of them is the popular coverage package.

---
Tags: jenkins, jenkins-pipeline, cicd, python, code-coverage
---


# Question
Title: Best practices for external backups for S3 buckets?
I would like to know how everyone is making external backups of **S3** buckets and best practices/recommendations for this. Regardless of how good **AWS** is it is still a single point of failure.

Especially on a more corporate scale, vast amounts of buckets that are very large.

The **S3** buckets functionality seems to be very limited.

**My resolution was the following**:

> The way I'm looking at doing this now is by downloading the data to an external server (**bare-metal** as **EC2** instance is a lot more expensive) and then pushing that to my backups provider.

# Answer
In terms of S3, for increased durability, consider cross region replication as an option if you want to increase the durability. AWS also provides their AWS Backup service you might investigate. I'm not certain of any benefit to transferring large amounts of data for durability to another provider. You'll pay a lot more for transfer, and it won't be integrated first class with the security IAM provides.

If the concern is about someone maliciously doing something then there are security best practices, access logging, cloudtrail, and other options to provide assurance. If it's more about the possibility (as low as it seems) for a bucket to fail, then the cross-region replication might be a great option for you with minimal configuration impact.
> 1 votes

# Answer
You can take backup of s3 using Amazon, other cloud services like google and local system.

* Local system

> S3 Browser comes with a simple Folder Sync Tool allowing you to upload or download only new and changed files and thereby significantly optimize your bandwidth usage and save you time when performing backups to Amazon S3.

This very good tool if your using window. here is the link http://s3browser.com/amazon-s3-folder-sync.aspx if you using Linux based system then i tired this tool also good one DragonDisk

* cloud-based backup

you can take backup of s3 in google cloud as well.

> The gsutil command-line tool also enables you to transfer data between Cloud Storage and other locations. While you can use gsutil to work with Amazon S3 buckets and transfer data from Amazon S3 to Google Cloud Storage, Storage Transfer Service is recommended for this use case.

* AWS

> As with any environments, the best practice is to have a backup and to put in place safeguards against malicious or accidental users errors. For S3 data, that best practice includes secure access permissions, Cross-Region Replication, versioning and a functioning, regularly tested backup.

* How durable is Amazon S3?

> Amazon S3 Standard and Standard - IA are designed to provide 99.999999999% durability of objects over a given year. This durability level corresponds to an average annual expected loss of 0.000000001% of objects. For example, if you store 10,000 objects with Amazon S3, you can on average expect to incur a loss of a single object once every 10,000,000 years. In addition, Amazon S3 is designed to sustain the concurrent loss of data in two facilities.

https://aws.amazon.com/s3/faqs/
> 0 votes

# Answer
AWS Datasync service will be a great way to sync S3, EFS or FSx data from AWS to another location. You can deploy the agent on a VM in Azure or GCP and then use their storage solution to receive the data from AWS. Obviously you will need to look at charges for outbound data transfer but if you have a strong business case for doing this then the costs should not be a problem.
> 0 votes

---
Tags: amazon-s3, storage, backup, disaster-recovery-plans
---


# Question
Title: Store and analyize logs data from diffenrent application
We have different applications that generate logs intensively, We do a lot of analytics, visualizations, monitoring on these logs.

Currently, we are using azure app insights to do the visualization but it's not enough for the advanced visualization we need, so we decided to move the data to another database that it's suitable for analyzing the data and be able to connect with tableau.

Initially, we are exploring elasticsearch but there is no straight forward connector to tableau so we need to build our connector and also I found some more options like ClickHouse, PrestoDB, InfluxDB, Apache HBase and Tarantool

So my question is, what is the best practice a storage system for logs data which hot active, continuous analysis, ability to monitor and could integrate with tableau?

# Answer
Can you not use Kibana with Elasticsearch or does that not give you the 'advanced' features you need?

Have you looked at this connector for ES and Tableau? https://www.dremio.com/tutorials/unlocking-tableau-on-elasticsearch/
> 1 votes

---
Tags: azure, elastic, storage, logstash
---


# Question
Title: Why can't I connect to Gitea using ssh via command line?
My computer: Windows 10 Pro 1909

I have a Gitea server running in my office, at local IP address 10.100.10.24, with SSH configured on port 22000. Gitea is configured to use port 3000 for HTTP.

I'm able to access the Gitea website through my desktop, which is not the same PC that Gitea is running on, at URL `http://10.100.10.24:3000/.`

Also, GitExtensions is able to connect to Gitea with the following SSH URL:

```
ssh://git@10.100.10.24:22000/afarley/[XXXXXXXX].git

```

I started trying to set up TeamCity on my desktop to build this repository. However, when I configure this repo in TeamCity, I'm unable to connect. To drill down, I've tried to connect via ssh using the command-line.

However, it seems that I'm unable to log in to the Gitea server via SSH; I **don't** get the expected response:

```
Hi there, You've successfully authenticated, but Gitea does not provide shell access.
If this is unexpected, please log in with password and setup Gitea under another user.

```

Rather, the entire terminal just seems to hang.

Things I have checked so far:

* The keyfile I'm pointing to (id\_rsa) is PEM-formatted by exporting OpenSSH format from PuttyGen
* id\_rsa has been set to 400 permissions via 'chmod 400 id\_rsa' (does this make sense? I'm using Windows 10)

The issue feels similar to this: https://github.com/PowerShell/Win32-OpenSSH/issues/1334

But the discussion for that issue seems to go dead around Jul 9, 2019. I can't believe that the openssh client on Windows 10 is this broken; am I doing something else wrong?

---

Here's what I get if I add the `-v` flag:

```
PS C:\Users\alexa\.ssh> ssh -v git@10.100.10.24 -p 22000 -i C:\Users\alexa\.ssh\id_rsa
OpenSSH_for_Windows_7.7p1, LibreSSL 2.6.5
debug1: Connecting to 10.100.10.24 [10.100.10.24] port 22000.
debug1: Connection established.
debug1: key_load_public: No such file or directory
debug1: identity file C:\\Users\\alexa\\.ssh\\id_rsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file C:\\Users\\alexa\\.ssh\\id_rsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_for_Windows_7.7
debug1: Remote protocol version 2.0, remote software version Go
debug1: no match: Go
debug1: Authenticating to 10.100.10.24:22000 as 'git'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256@libssh.org
debug1: kex: host key algorithm: ssh-rsa
debug1: kex: server->client cipher: aes128-ctr MAC: hmac-sha2-256-etm@openssh.com compression: none
debug1: kex: client->server cipher: aes128-ctr MAC: hmac-sha2-256-etm@openssh.com compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ssh-rsa SHA256:KuxUCu3Lh34jm8eqvengtJ10f+TG/mgPSRzrT+oBxGg
debug1: Host '[10.100.10.24]:22000' is known and matches the RSA host key.
debug1: Found key in C:\\Users\\alexa/.ssh/known_hosts:1
debug1: rekey after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 4294967296 blocks
debug1: pubkey_prepare: ssh_get_authentication_socket: No such file or directory
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: C:\\Users\\alexa\\.ssh\\id_rsa
debug1: Authentication succeeded (publickey).
Authenticated to 10.100.10.24 ([10.100.10.24]:22000).
debug1: channel 0: new [client-session]
debug1: Entering interactive session.
debug1: pledge: network
debug1: console supports the ansi parsing

```

---

Edit: About 2 weeks later, I was investigating/making some changes to the apache configuration for this server and SSL started working. I'm not sure what the original issue was, but the short story is that I was able to get a clean integration between a self-hosted Gitea/TeamCity and GitExtensions on my desktop.

There is no issue with the ssh command above (`ssh -T -v git@10.100.10.24 -p 22000`); it gives the expected result now.

The issue *may* have been that the VM running Gitea had gone to sleep; our office machine apparently is pausing the Hyper-V VM when it goes to sleep.

# Answer
> 1 votes
OK, there seems to be two problems here.

---

### SSH hanging after connection

It's probably a problem on the client tty due to limitations of MinGW. In the past I've encountered that `ssh` was unable to `ioctl` the local `tty` because the lack of a control device (`pty`). I've used https://github.com/rprichard/winpty at the time, but I think that newer versions of MinGW/MinGW64 (the Posix layer installed to run bash by Git) have that covered because I didn't see that problem anymore.

Summary:

* Try upgrading git in your Windows client to the latest version (best option).
* Try using `GIT CMD` instead of `GIT BASH`.
* If you're not using the GIT components on Windows, try invoking `ssh` from a Command window.

---

### SSH not using your key

I suspect that this is your problem:

```

debug1: identity file C:\\Users\\alexa\\.ssh\\id_rsa type -1

```

In my PC, I get the following:

```
debug1: Reading configuration data /c/Users/xxxx/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Connecting to xxx [192.168.0.1] port 22.
debug1: Connection established.
debug1: identity file /c/Users/xxxx/.ssh/id_rsa type 0

```

As to why you're getting type `-1`, it *might* be a permissions problem. Make sure that all `C:\Users\alexa`, `C:\Users\alexa\.ssh` *and* `C:\Users\alexa\.ssh\id_rsa` have only permissions for `SYSTEM`, `alexa` and `Administrators`.

Also, please notice that you're getting double backslashes in your log (e.g. `C:\\Users\\alexa\\.ssh\\id_rsa` instead of `C:\Users\alexa\.ssh\id_rsa`). I don't get double backslashes in mine.

# Answer
> 0 votes
Looks like you needs to install it another way. The errors message does give you a hint

> and setup Gitea under another user

The docs say

> Gitea should be run with a dedicated non-root system account on UNIX-type systems. Note: Gitea manages the ~/.ssh/authorized\_keys file. Running Gitea as a regular user could break that user’s ability to log in.

Check here under the system requirements https://docs.gitea.io/en-us/

---
Tags: git, ssh
---


# Question
Title: Is \u the only way to output the user from .bashrc?
I can't get it to work on my Mac (catalina version 10.15.2). `\u` works on ubuntu when I test it but not on my Mac. I want to customize my mac terminal. Colors work fine on my Mac and it's reading the `.bashrc` file when I test it by `source .bashrc` . For example with the following in `.bashrc` in the user home directory.

```
orange=$(tput setaf 166);
PS1="${orange}\u"
export PS1;

```

Outputs my username in orange on ubuntu but on my Mac it just outputs `\u` in orange so for some reason `\u` isn't working on my Mac? Is there an alternative I could try?

# Answer
> 3 votes
With the release of macOS Catalina (10.15) Apple has switched to Zsh as default shell (instead of Bash). Instead of setting *PS1* in Bash you can set *PROMPT* in Zsh: `PROMPT="${orange}%n"`

---
Tags: bash, macos
---


# Question
Title: Which Jenkins file is responsible for Enable cache setting
I want to know how can I modify the below setting from Jenkins server and not UI:

http://XXXXX:YYYYY/configureSecurity/

# Answer
> 1 votes
Add the following lines in `/var/lib/jenkins/config.xml` under `securityRealm` and then restart `jenkins` service to enable cache. Remove the below 4 line if you don't want to enable it.

```
<cache>
  <size>500</size>
  <ttl>3600</ttl>
</cache>

```

Please refer the below image

---
Tags: jenkins, linux
---


# Question
Title: How can I add an additional IP / hostname to my Kubernetes certificate?
I've recently followed this guide to set up Kubernetes on my local Raspberry Pi cluster. I can access it fine on my LAN; however I'd like to be able to access it remotely as well. I've already set up port forwarding on my router, but when I try to connect from outside the LAN, I get a x509 certificate error indicating that the certificate is signed for my local 192.168 address but not for my home's IP. I could bypass this with the `--insecure-skip-tls-verify` options but I'd much rather have the certificate used simply incorporate my IP address. Only... I'm not fully sure how to do that, though I do have some ideas.

When I initially set things up with `kubeadm init`, should I have used the `--apiserver-advertise-address` option to specify my home IP? Would doing so have had any adverse effects (such as precluding my local/192.168 IP?). Ideally I'd like the certificate to be signed for both the internal and external IPs. Or is there some other option I should have given to `kubeadm init`? Secondly, now that it has been initialized, is there any way to regenerate the certificate and swap it out, or will I need to nuke the cluster?

To clarify, I don't have any certificate managers installed. I'm not talking about Ingress. I'm talking about the actual "root" certificate (for lack of a better term) that kubeadm generates when first installing the cluster, i.e. the one that lives at `/etc/kubernetes/pki/apiserver.crt`

# Answer
> 15 votes
To do this, you’ll first need your `kubeadm` configuration file. This creates a file named `kubeadm.yaml`:

`kubectl -n kube-system get configmap kubeadm-config -o jsonpath='{.data.ClusterConfiguration}' > kubeadm.yaml`

Now open the file in an editor, and find the `certSANs` list under the `apiServer` section. If it does not exist, you’ll need to add it; if so, you’ll just add another entry to that list. Example:

```
apiServer:
  certSANs:
  - "172.29.50.162"
  - "k8s.domain.com"
  - "other-k8s.domain.net"
  extraArgs:
    authorization-mode: Node,RBAC
  timeoutForControlPlane: 4m0s

```

Now move the old certificates to another folder, otherwise `kubeadm` will not recreate new ones:

`mv /etc/kubernetes/pki/apiserver.{crt,key} ~`

Use `kubeadm` to generate new apiserver certificates:

`kubeadm init phase certs apiserver --config kubeadm.yaml`

Now restart your *kubeapiserver* container:

1. Run `docker ps | grep kube-apiserver | grep -v pause` to get the container ID for the container running the Kubernetes API server
2. Run `docker kill <containerID>` to kill the container.
3. *The Kubelet will automatically restart the container, which will pick up the new certificate.*

If everything is working as expected, don't forget to update the `kubeadm` ConfigMap stored in the cluster, otherwise, future `kubeadm` upgrade will be lacking your new config:

If using Kubernetes \< v1.15:

`kubeadm config upload from-file --config kubeadm.yaml`

For Kubernetes version \>= v1.15:

`kubeadm init phase upload-config kubeadm --config kubeadm.yaml`

*This article has a more complete guide on how to Adding a Name to the Kubernetes API Server Certificate*

# Answer
> 2 votes
The answer given by @Eduardo Baitello is the correct answer for the vast majority of Kube installs. However I learned that if you've installed the MicroK8S snap install on Ubuntu the instructions are a little bit different. Per this issue, you'll need to update the `/var/snap/microk8s/current/certs/csr.conf.template` file, then run `microk8s.stop` and `microk8s.start` in succession.

---
Tags: kubernetes, ssl
---


# Question
Title: Error from server (Forbidden): error when retrieving current configuration of: Resource: "apps/v1, Resource=deployments",
I integrate the existing kubernetes cluster to the gitlab instance (omnibus). I get the below error in the deployment stage of CI/CD pipeline:

```
Error from server (Forbidden): error when retrieving current configuration of:
 Resource: "apps/v1, Resource=deployments", GroupVersionKind: "apps/v1, Kind=Deployment"
 Name: "test-deployment", Namespace: "default"
 Object: &{map["apiVersion":"apps/v1" "kind":"Deployment" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "name":"test-deployment" "namespace":"default"] "spec":map["replicas":'\x01' "selector":map["matchLabels":map["app":"test"]] "template":map["metadata":map["labels":map["app":"test"]] "spec":map["containers":[map["env":[map["name":"OHH_COMMON_REDEPLOY" "value":"Sun Feb  9 13:55:45 +0330 2020"]] "image":"192.168.10.6:5000/majid/hello-world:v0.01" "name":"test" "ports":[map["containerPort":'P']]]]]]]]}
 from server for: "deployment.yaml": deployments.apps "test-deployment" is forbidden: User "system:serviceaccount:kubetest-2-bina:kubetest-2-bina-service-account" cannot get resource "deployments" in API group "apps" in the namespace "default"
 ERROR: Job failed: exit status 1

```

There is my .gitlab-ci.yaml file:

```
deploy:
  image:
    name: lachlanevenson/k8s-kubectl:latest
    entrypoint: ["/bin/sh", "-c"]
  stage: deploy
  environment:
    name: bina
    url: https://192.168.x.x
  only:
    - master
  script:
    - kubectl version
    - sed -ie "s/THIS_WILL_BE_REPLACED/$(date)/g" deployment.yaml
    - kubectl apply -f deployment.yaml --namespace=default

```

The deployment file look like this:

```
apiVersion: apps/v1
kind: Deployment
metadata:
  name: test-deployment
spec:
  selector:
    matchLabels:
      app: test
  replicas: 1
  template:
    metadata:
      labels:
        app: test
    spec:
      imagePullSecrets:
      - name: regcred
      containers:
      - name: test
        image: 192.168.10.6:5000/majid/hello-world:v0.01
        ports:
        - containerPort: 80
        env:
        - name: OHH_COMMON_REDEPLOY
          value: THIS_WILL_BE_REPLACED

```

I also create ServiceAccount and ClusterRoleBinding:

```
apiVersion: v1
kind: ServiceAccount
metadata:
  name: gitlab-admin
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: gitlab-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: gitlab-admin
  namespace: kube-system

```

clusterRole definition:

```
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: secret-reader
rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "watch", "list"]

```

How can I resolve this issue?

# Answer
Going from this, your ClusterRole isn't configured to allow access to deployments, and the ClusterRole you've listed isn't properly bound to your service account. You could configure it with something like I did below as a troubleshooting measure/to make sure you're able to properly configure permissions and rule out an issue with the serviceaccount's role bindings.

```
apiVersion: v1
kind: ServiceAccount
metadata:
  name: gitlab-admin
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: gitlab-admin-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: deployment-reader
subjects:
- kind: ServiceAccount
  name: gitlab-admin
  namespace: kube-system

```

A new ClusterRole

```
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: deployment-reader
rules:
- apiGroups: ["extensions", "apps"]
  resources: ["deployments"]
  verbs: ["get", "watch", "list"]

```

Then to check to make sure the service account can properly access the resource in the default namespace you can check with the following command

```
kubectl get deployments --as system:serviceaccount:kube-system:gitlab-admin -n default

```
> 3 votes

# Answer
It is possible to get rid of this problem by uncheck the gitlab-managed cluster checkbox.
> 0 votes

---
Tags: kubernetes, gitlab, cicd, gitlab-ci-runner, kubeadm
---


# Question
Title: Does active choice reactive parameter support hide fields option?
I have the below pipeline:  I want that once I choose in the InstllationType the value 'Recreate', some fields will disappear. is it possible to do it with an active choice parameter? if not, is there any other way for doing it?

# Answer
No, this is not possible, because the number and type of parameters for a build are fixed and cannot be modified on-the-fly. The only way to modify which parameters show up in the "build with parameters" UI is to change the job configuration, which alters the parameters permanently.

In my opinion, if you need to do several different tasks which require several different input parameters (such as "build", "test", "install", "deploy", etc.), you should have separate jobs in Jenkins for each task. I personally think it is a good ideal in general to avoid complex Jenkins jobs which require a lot of input parameters and try instead to simplify job configuration in order to reduce the likelihood of subtle, difficult-to-debug errors when executing jobs.
> 1 votes

---
Tags: jenkins-pipeline
---


title: AWS organizations and Billing

text:
# Question
Title: AWS organizations and Billing
I was using an AWS account with my credit card info and my personal account. Since I'm learning my way around AWS services I am using free tier as much as I can but I still have some small charges sent to my credit card at the end of the month. Recently I started a freelance project and they sent me a link and I became a part of the organization. Under My Organization I see the organization with the following note: "Organization features enabled... All features enabled...". My question is how is billing done and if I spin up an instance not in the free tier who will pay for it and how can I manage and define who is paying for what?

# Answer
> 1 votes
Go to the Billing Dashboard link in your console

If you can see details of your CC then it's you.

If you see something like this message:

Then it's managed by a different account that you do not even have permissions to see.

---
Tags: amazon-web-services, amazon-ecs
---


# Question
Title: How to decrypt Jenkins passwords from credentials.xml?
I've taken over the project where a lot of Jenkins credentials has passwords or passphrase strings which I need to know in order to progress with the project, unfortunately these weren't documented anywhere.

I've checked the `credentials.xml` file where these credentials are stored, but they're in not plain text, e.g.:

```
<passphrase>{AAAAAAAAAAAANzxft/rDzyt8mhxpn3O72dxvVqZksL5vBJ4jNKvAjAA=}</passphrase>

```

Note: I've changed it slightly for privacy reasons.

How can I decrypt its original password based on the string above?

# Answer
Luckily there is a `hudson.util.Secret.decrypt()` function which can be used for this, so:

1. In Jenkins, go to: `/script` page.
2. Run the following command:

```
println(hudson.util.Secret.decrypt("{XXX=}"))

```

or:

```
println(hudson.util.Secret.fromString("{XXX=}").getPlainText())

```

where `{XXX=}` is your encrypted password. This will print the plain password.

To do opposite, run:

```
println(hudson.util.Secret.fromString("some_text").getEncryptedValue())

```

Source: gist at `tuxfight3r/jenkins-decrypt.groovy`.

---

Alternatively check the following scripts: `tweksteen/jenkins-decrypt`, `menski/jenkins-decrypt.py`.

---

For more details, check: Credentials storage in Jenkins.
> 122 votes

# Answer
Here is a short snippet you can just run from the jenkins script console, to dump all of your credentials to plain text.

```
com.cloudbees.plugins.credentials.SystemCredentialsProvider.getInstance().getCredentials().forEach{
  it.properties.each { prop, val ->
    if (prop == "secretBytes") {
      println(prop + "=>\n" + new String(com.cloudbees.plugins.credentials.SecretBytes.fromString("${val}").getPlainData()) + "\n")
    } else {
      println(prop + ' = "' + val + '"')
    }
  }
  println("-----------------------")
}

```

A more complicated version that lists for non-system credential providers:

```
import com.cloudbees.plugins.credentials.CredentialsProvider
import com.cloudbees.plugins.credentials.Credentials
import com.cloudbees.plugins.credentials.domains.Domain
import jenkins.model.Jenkins
def indent = { String text, int indentationCount ->
  def replacement = "\t" * indentationCount
  text.replaceAll("(?m)^", replacement)
}

Jenkins.get().allItems().collectMany{ CredentialsProvider.lookupStores(it).toList()}.unique().forEach { store ->
  Map<Domain, List<Credentials>> domainCreds = [:]
  store.domains.each { domainCreds.put(it, store.getCredentials(it))}
  if (domainCreds.collectMany{ it.value}.empty) {
    return
  }
  def shortenedClassName = store.getClass().name.substring(store.getClass().name.lastIndexOf(".") + 1)
  println "Credentials for store context: ${store.contextDisplayName}, of type $shortenedClassName"
  domainCreds.forEach { domain , creds ->
    println indent("Domain: ${domain.name}", 1)
    creds.each { cred ->
      cred.properties.each { prop, val ->
        println indent("$prop = \"$val\"", 2)
      }
      println indent("-----------------------", 2)
    }
  }
}

```
> 36 votes

# Answer
Based on Magnus' answer but as a simple one-liner with still readable output:

```
com.cloudbees.plugins.credentials.SystemCredentialsProvider.getInstance().getCredentials().forEach{println it.dump().replace(' ', '\n')}

```
> 10 votes

# Answer
@kenorb example with `hudson.util.Secret` is good. Also answers listing all credentials are very useful (thanks @ymajoros for one-liner).

These do not handle secret files though where `secretBytes` are used and still an encrypted string is shown. In such case the `SecretBytes` class needs to be used. Assuming the file is UTF-8, one can do:

```
secret = "{....}"
new String(com.cloudbees.plugins.credentials.SecretBytes.fromString(secret).getPlainData())

```

HTH

**Update:** This is the full solution from my gits:

```
com.cloudbees.plugins.credentials.SystemCredentialsProvider.getInstance().getCredentials().forEach{
  it.properties.each { prop, val ->
    if (prop == "secretBytes") {
      println(prop + "=>\n" + new String(com.cloudbees.plugins.credentials.SecretBytes.fromString("${val}").getPlainData()) + "\n")
    } else {
      println(prop + ' = "' + val + '"')
    }
  }
  println("-----------------------")
}

```
> 5 votes

# Answer
For the record, The following snippet to be pasted into the console also does the job :

```
def creds = com.cloudbees.plugins.credentials.CredentialsProvider.lookupCredentials(
    com.cloudbees.plugins.credentials.common.StandardUsernameCredentials.class,
    Jenkins.instance,
    null,
    null
)

for(c in creds) {
  if(c instanceof com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey){
    println(String.format("id=%s  desc=%s key=%s\n", c.id, c.description, c.privateKeySource.getPrivateKeys()))
  }
  if (c instanceof com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl){
    println(String.format("id=%s  desc=%s user=%s pass=%s\n", c.id, c.description, c.username, c.password))
  }
}

```
> 4 votes

---
Tags: jenkins, encryption
---


# Question
Title: How to copy files from a different linux server folder to Kubernetes pod
I have multiple files in a different linux server folder and I want to create a cron job and run daily in kubernetes that should copy the files to a pod network folder.

I am researching a lot on this, but non of them gave a me clue to do this experiment. Is it we need to create Persistent volume claim against linux folder? Please help to accomplish this.

# Answer
> I have multiple files in a different linux server folder and I want to create a cron job and run daily in kubernetes that should copy the files to a pod network folder.

I am unaware how to perform the task as you have stated.

How to solve your problem? With the belief your "POD network folder" is a SMB, NFS, other type mounted resource;

I would implement one of the following, either will work the same in the end:

1. Establish a connection directly from the Linux Host (that presently has the file(s) to be copied) to the Networked Folder. Use SCP or Rsync to perform the copy.
2. Using SCP or Rsync copy the file(s) directly from the Linux Host homing the files to the IP Address of the POD. Given the POD has a standing connection to the Hosted Folder. e.g. NFS, SMB, other mount.
> 1 votes

---
Tags: kubernetes
---


# Question
Title: Cloud agnostic infrastructure automation better than terraform?
Are there any infrastructure automation technologies which do a better job at being portable than terraform?

We heavily leverage terraform as it is a cloud-agnostic technology (vs, say, ARM templates in azure, or cloud formation in AWS).

I was very depressed a year ago when we deployed an app from azure to AWS to see that a basic scale-set + load balancer basically required a complete rewrite between Azure and AWS though. Having used ansible equivalents, the same seems to be true there. Both basically use cloud-specific extensions to create cloud resources meaning that, while you understand the tool still, any project you code has to be re-coded for deployment in a new cloud.

I understand that a generic tool would have to understand numerous differences like:

* Azure uses resource groups to encapsulate all resources.
* AWS has security groups to define network boundaries.
* AWS load balancers require separate target groups/listeners vs Azure's simpler backend pool concept.
* IAM differences, and so on.

I'm surprised there aren't any companies or OSS projects trying to simplify/automate the translations here with a higher level DSL or something though. So, I'm thinking I might be missing something.

Note: We are heavy kubernetes users, so I do understand that kubernetes mitigates most of these issues **once you've brought up a cluster** (and if you have ported all your apps to it).

# Answer
> 3 votes
There are no "agnostic" products due to the dramatic differences of the design of the Cloud Services provided by different companies.

My best recommendation is to write your code to support the different cloud providers. Whether you choose Ansible or Terraform this can be done with Ansible Roles or Terraform Modules that are specific to the cloud provider. Just the same as you would write Ansible roles specific to Operating Systems or different scenarios.

---
Tags: amazon-web-services, terraform, azure, automation
---


# Question
Title: how to checkout only specific folder from git repo((monorepo)) and build
Hey folks I need some help on the jenkinsfile. Below is my usecase

This is the strcuture of my GIT repo:

```
root
  |->app1
  |   |->jenkinsfile
  |   |->dockerfile1
  |->app2
      |->jenkinsfile
      |->dockerfile2

```

I am having a monorepo, app1 and app2 in the root folder and when there is a change in app1 folder, only app1 should build and same for app2. I have defined the jenkinsfile in jenkins but when its build. its looking for dockerfile1 in root folder not inside app1.

P.S: I can see some thread where people asking the same question but the answer is in detail.

jenkinsfile:

```
pipeline {
    agent any

    environment {
        PIPENV_VENV_IN_PROJECT = true
        DEVPI_USER = '\'jenkins_user\''
        DEVPI_PASSWORD = '\'V$5_Z%Bf-:mJ\''
        WORKSPACE="${WORKSPACE}/app1"

    }
    stages {
        stage('Notify Bitbucket') {
            steps {
                bitbucketStatusNotify(buildState: 'INPROGRESS')
            }
        }

        stage('Build Environment') {
            steps {
                sh 'docker build -t app-builder .'
            }
        }

        stage('Test') {

            steps{
                sh 'docker run --rm app-builder pytest'
            }
        }

```

# Answer
You can check out specific files or folders from a repo using `git archive`. I've done this from a Jenkinsfile using a shell command like so:

```
sh("git archive --remote=${repo_url} ${ref} ${path} | tar x")

```

You will, of course, need to set the variables `repo_url` (the url of the remote repository), `ref` (the ref/branch/commit/etc. to check out), and `path` (the path of the file or folder within the repo to check out).
> 2 votes

---
Tags: jenkins, jenkins-pipeline, jenkinsfile
---


# Question
Title: AWS launch template create-before-destory?
In terraform, to reduce downtime when creating a new auto-scaling-group behind an existing load balancer, you can use a launch configuration and set create-before-destroy.

Launch templates are the recommended/new replacement for launch configs... but I can't seem to figure out how to achieve the same thing.

How can you mimmic this behavior in launch templates?

# Answer
> 4 votes
It seems you're misunderstanding what the create-before-destroy lifecycle parameter of the launch configuration does.
It's about replacing the launch configuration within an autoscaling group, not replacing the autoscaling group.

Launch templates have versions which makes this parameter useless as you'll create a new version of the launch template before updating the ASG to use it.

Terraform gives an example in the ASG documentation.

The underlying idea is to be able to roll back more easily than with launch-configurations.

---
Tags: amazon-web-services, terraform, asg
---


# Question
Title: per-merge request GitLab CI pipeline customization
We have a gitlab CI pipeline for building&testing our product. Our product can be, fairly clearly, divided into two distinct parts - let's say we call them A and B. The most time-consuming part of our pipeline are end-to-end (e2e) tests that take \> 40 minutes to execute, and we want to cut this time down. They are currently executed in 1 job.

Because our product consists of A and B, for each merge request we want to run only a subset of our e2e tests (only tests for A or only tests for B or all) - we want to be able to specify this manually for each merge request.

We were looking for some idiomatic way to do this in GitLab CI, but didn't find one. What is the cleanest, least obtrusive way to achieve this? We ofc also want to be able to run all e2e tests - this would be the case of master branch.

Our ideas included encoding this information in commit messages or merge request title and extracting this from env variables - https://docs.gitlab.com/ee/ci/variables/predefined\_variables.html but both seem like a hack.

Thank you :)

# Answer
> 1 votes
I'd go with what gitlab already offers, the only/except keyword.

Something like:

```
test:e2e:A:
  script:
    - whatever you do to test A part
  only:
   refs:
      - master
      - /^A-.*/

test:e2e:B:
  script:
    - whatever you do to test B part
  only:
   refs:
      - master
      - /^B-.*/

```

The only requirement is that work on A or B is done in branches prefixed with A- or B-.

the `refs` keyword accept regexes to match, more details on gitlab-ci docs here

If your gitlab is pretty recent (\>12.3) the rules method would be prefered and the example above for A would become:

```
test:e2e:A:
  script:
    - whatever you do to test A part
  rules:
    - if: $CI_COMMIT_REF_NAME =~ /^A-.*$/
    when: always
    - if: $CI_COMMIT_REF_NAME =~ /^master$/
    when: always

```

Or with a single regex:

```
test:e2e:A:
  script:
    - whatever you do to test A part
  rules:
    - if: $CI_COMMIT_REF_NAME =~ /^(A-.*|master)$/
    when: always

```

I didn't test those entries above, let me know if one fail

---
Tags: gitlab
---


# Question
Title: Unable to start step function after account move
I recently moved my development account away from my prod one, everything seemed fine except I have a service that runs a lambda to start a step function - not ideal but used to work.

My issue comes in that I am now able to run everything up to and including the execute step function from the npm package provided by aws. I have checked and the account ID is updated in the arn of the step function/machine. I see in my cloudwatch logs that it is timing out after 30 seconds (had a 6 but changed to test).

What are some possible causes of a step function not executing after an account change with no code changes other than the account ID update.

# Answer
> 1 votes
Issue is resolved by updating the subnet to a private subnet which uses a nat gateway instead of a public subnet with an internet gateway. This SO post lead my in the right direction.

---
Tags: amazon-web-services, node.js, step-functions
---


# Question
Title: Jenkinsfile handling an expected job failure
I've got two types of failures to account for within a Jenkins job.

1. The job genuinely failed
2. The job failed because of a certain bit of info hasn't been published. Nothing is actually wrong here.

How do I handle the 2nd case so it shows the job did no complete, but nothing is actually wrong?

# Answer
> 1 votes
One possibility would be to place the checks for all the necessary pre-conditions (i.e. including those bits of info that would need to be published) in a separate build step to be executed prior to the actual build step.

If the checks in this preliminary step fail the build could be marked as aborted instead of failed. The aborted build would, hopefully, be a better indication that "nothing is actually wrong" (debatable, I know).

Marking the build as aborted could maybe be done in a manner similar to these SO answers:

If those checks cannot be pulled in a step separated from the actual build step things are a bit more complicated:

* the actual build step would have to somehow be configured to not fail the job. I know it's possible for steps performed in virtual environments using the ShiningPanda plugin, but other types of steps do not have such capability.
* an additional step would have to be added after the actual build step. Its job would be to somehow determine if the actual build step failed and, if so, if failure was causing by one of those unfulfilled pre-requisite conditions (maybe by parsing the build logs, for example?), in which case it would abort the build. Otherwise it would pass or fail, depending on the actual build step outcome, to properly indicate the overall run status in Jenkins.

---
Tags: jenkins, jenkins-pipeline, jenkinsfile
---


# Question
Title: Stop Multiple Windows Service Simultaneously
I have multiple windows service and there are no dependencies between them. I want to stop all the windows services at the same time. I don't want to stop all the services one by one. How can I achieve this?

# Answer
The `-name` parameter of the `Stop-Service` command is an \[array strings\]\[1\]. You can pass multiple services to it.

```
Stop-Service -name service1,service2,service3

```

**Edit**

```
[string[]]$services = @('service1', 'service2')
Stop-Service -Name $services -NoWait
while ((Get-Service -Name $services | Where-Object { $_.status -ne 'Stopped'}).Count -ne 0)
{
    Start-Sleep -Seconds 5
}

```
> 1 votes

---
Tags: powershell
---


# Question
Title: How to create Golang Linux binaries using a Windows host
When `go build` is issued on a Windows host a .exe binary is created. How to ensure that a Linux binary is created on a Windows host?

The following was tried, but did not solve the issue:

```
GOOS=linux;go build hello-world.go

```

# Answer
> 4 votes
You can easily set the target operating system and processor architecture using the environment variables **GOOS** and **GOARCH** respectively. So, as you want to build it for linux operating system, following command with above environment variables will do,

```
$ GOOS=linux GOARCH=amd64 go build -o hello main.go

```

Here is the list of all the supported operating system with which you can easily do cross compilation using Go lang.

# Answer
> 3 votes
It seems that `docker build` opens a new shell and `export` is needed. The following command created a Linux binary:

```
export GOOS=linux; go build hello-world.go

```

# Answer
> 0 votes
Windows also has a one-liner solution, just a bit uglier than the linux version. Any of the below lines will do the job:

```
cmd /C "SET GOOS=linux&& SET GOARCH=amd64&& go build main.go"
cmd /C "SET "GOOS=linux" && SET "GOARCH=amd64" && go build main.go"

```

Be careful, if you just write `...SET GOOS=linux && SET ...` then you will get an error message

```
cmd/go: unsupported GOOS/GOARCH pair linux /amd64

```

because there will be an extra space after the value `linux`. In order to avoid this, use either the condensed form `linux&&` or the other version with quotation marks.

---
Tags: docker, linux, windows
---


# Question
Title: Continuous deployment of package to virtualenv
My colleagues don't want to create their own virtualenv and deploy my tools from git on their own as part of their automation. Instead they want the tools pre installed on a shared server.

So I was thinking of making a /opt directory, putting the virtualenv in there and then pulling from git every hour to force an update of the python package. I am not version tagging the tools currently and instead would just ask pip to force the install every time.

The problem is the race condition. If the tool is called by their automation during the pip upgrade, the tool can fail due to the install not being atomic - I believe pip removes the entire package first.

I've thought of various work arounds (forcing the use of flock, using symlink to atomically switch the virtualenv, wrapping the tool in a script that makes the virtualenv in a temp directory for every use...)

Is there a best practice here I'm not aware of?

# Answer
> 2 votes
Using a shared virtualenv for automation while also expecting the virtualenv's content to be (automatically) updated without the automation's knowledge/control is not a good practice - opening up for all kinds of obscure/intermittent/hard to reproduce failures.

So I wouldn't spend too much time trying to figure out a solid way of updating your package in the shared virtualenv without hitting the automation jobs - sooner or later they'll be hit by other problems stemming from the same root cause - using a shared virtualenv. Think back at the very reason(s) for using a virtualenv in the first place.

For consistent results automation should ensure that the virtualenv content is updated as needed as a preliminary step and does not (unexpectedly) change while the job is performed. Which isn't really possible with a virtualenv shared across multiple **independent** automation jobs. It *might* be possible with coordinated jobs, all sharing a common step that performs the virtualenv updates in a controlled manner to ensure virtualenv's consistency during jobs, but IMHO it's just not worthy to build such complex system, it's much simpler to not share the virtualenv and keep the jobs independent.

If, regardless of these considerations, the request for a shared virtualenv still stands then don't update the virtualenv outside (well defined and announced) maintenance windows during which execution of the automation jobs should be suspended.

As a side note it's also not a good practice to not version your packages, unless maybe if you have a very clear and easily accessible indication of the exact code version being used in each particular case (the commit SHA for example) - when problems occur you'll most likely need that info when attempting to reproduce the problem, debug it and/or verify fixes with some degree of certainty.

---
Tags: continuous-deployment, python, virtualenv
---


title: Jenkins deploy static web with Apache2

text:
# Question
Title: Jenkins deploy static web with Apache2
I have a static website that I would like to deploy to my Apache2 server using Jenkins. I think of 2 solutions:

* Should I change the workspace dir of Jenkins to /var/www/?
* Or should I copy all files in workspace to /var/www in post build?

# Answer
> 1 votes
As mentioned, there are many ways to accomplish the final goal of exposing a site built from a Jenkins job to public traffic.

Both ideas could work, but here are some tweaks.

* Instead of changing the job workspace directory, build the site to a sub directory in the workspace and symlink the directory with root directory of your site. This allows control over what is part of live site.
* To put a boundary between the build and deploy, rsync would be more efficient and easier to manage than copying. Follow the same pattern of creating a sub directory and rsync to the site directory.

The decision depends on the requirements of the site. If it is static with no promise of up-time, a symlink will work fine, however if the Jenkins job breaks and breaks the workspace version of the site, the actual site will be down. I would recommend using rsync and adding a validation step between the build and rsync to ensure the site stays up.

Final note, using git or other scm provides backups which can be tagged. If the current build breaks, rolling back would be easy. However, if there is no scm (not recommended), keep some segregation between the build and live site to ensure it is not broken for an extended amount of time.

---
Tags: jenkins, apache-httpd
---


# Question
Title: StatefulSet or Deployment for Jira on Kubernetes
We are looking into moving our Jira instance on a Kubernetes cluster and one of the concerns that were recently risen is whether we should go with a StatefulSet or a Deployment. So, are there any suggestions or lessons learned by using any of the above controllers? Any additional tips, source of information and comments by people who went through the process could be very valuable to us.

# Answer
> 1 votes
Deployments are primarily based on stateless services, where data is externalized to be stored on some other storage block or database. Using deployments for stateful services is not recommended as you will encounter scaling problems, since multiple replicas will read from the same persistent volume.

Instead, StatefulSets are the best approach to stateful services. Each replica of the StatefulSet has its own state mounted on a dedicated persistent volume, thus scaling a StatefulSet will create a new persistent volume and attach the new replica to it.

---
Tags: kubernetes, jira
---


# Question
Title: Project pipeline not able to access NuGet package from Artifact Feed in different Project
> Updated 20th Feb & 21st Feb

**Summary**

Not able to find NuGet package from our private feed under Artifacts.

**Our Azure DevOps environment**

In Azure, we have our company "space": dev.azure.com/OurCompany

Under that, we have multiple Projects, so for example, imagine these three:

* Green: this holds applications of a certain type
* Blue: this holds applications of a different type
* Red: this holds our "common" helper code

**RED**

RED holds all our common helper code are DLLs consumed by applications in GREEN and BLUE. These are made available as NuGet packages.

Under RED's Artifacts, we see an "organisational scoped feed" called "OurCompany". We understood that we couldn't use this because these NuGet packages need to be available to everyone in OurCompany, which means not only on Azure, but also on our DeskTop machines in our Corporate network (where we run Visual Studio). The "organisational scoped feed" I don't think is exposed to our Corporate Network.

We therefore created a new Feed under RED's Artifacts. When I view Permissions, I see the following entries:

* {me} : (Owner)
* \[OurCompany\]\Project Collection Adminstrators : (Owner)
* \[RED\]\Project Administrators : (Owner)
* Project Collection Build Service (OurCompany) : (Contributor)
* \[RED\]\Contributors (Contributor)
* RED Build Service (OurCompany) : (Contributor)
* BLUE Build Service (OurCompany) : (Contributor)

**Corporate**

So, back in Corporate, we add the NuGet package feed to our Visual Studio and it can then pull the NuGet packages down from Azure.

**GREEN and BLUE**

In here we have our products. We create a build Pipeline and we add a NuGet restore step:

```
- task: NuGetCommand@2
  displayName: 'Restore for Our Application'
  inputs:
    command: 'restore'
    restoreSolution: '**/*.sln'
    feedsToUse: 'select'
    vstsFeed: '{Guid1}/{Guid2}'

```

The two Guids were filled in when we picket the Azure Artifacts feed from the drop-down.

**GREEN or BLUE NuGet Restore Failure**

So this is where we run into our issue. When the NuGet restore step executes, it fails with the following message:

```
Unable to find version 'nnnn.n.n' of package 'TheNuGetPackage'.

```

The log output is vast, but the salient lines appear to be:

```
2020-02-18T18:36:35.2838118Z [command]C:\hostedtoolcache\windows\NuGet\5.4.0\x64\nuget.exe sources Add -NonInteractive -Name {Guid2} -Source https://pkgs.dev.azure.com/OurCompany/{Guid1}/_packaging/{Guid2}/nuget/v3/index.json
-ConfigFile d:\a\1\Nuget\tempNuGet_356.config 2020-02-18T18:36:35.7513495Z Package source with Name: {Guid2} added successfully. 2020-02-18T18:36:41.1671856Z     [CredentialProvider.183638]Command-line v0.1.20+a5d37185a1a95acb4f0g997b9c23cab99508: "d:\a\_tasks\NuGetCommand_333b11bd-d341-40d9-bdbd-b32d5ce6f23b\2.164.0\CredentialProviderV2\plugins\netfx\CredentialProvider.Microsoft\CredentialProvider.Microsoft.exe"
-Plugin 2020-02-18T18:36:41.1673979Z     [CredentialProvider]Handling 'Request' 'GetAuthenticationCredentials'. Time elapsed in ms: 3 - Payload: {"Uri":"https://pkgs.dev.azure.com/OurCompany/{Guid1}/_packaging/{Guid2}/nuget/v3/index.json","IsRetry":false,"IsNonInteractive":true,"CanShowDialog":true} 2020-02-18T18:36:41.1674299Z     [CredentialProvider]Handling auth request, Uri: https://pkgs.dev.azure.com/OurCompany/{Guid1}/_packaging/{Guid2}/nuget/v3/index.json, IsRetry: False, IsNonInteractive: True, CanShowDialog: True 2020-02-18T18:36:41.1674465Z     [CredentialProvider]URI: https://pkgs.dev.azure.com/OurCompany/{Guid1}/_packaging/{Guid2}/nuget/v3/index.json 2020-02-18T18:36:41.1674818Z     [CredentialProvider]Skipping NuGetCredentialProvider.CredentialProviders.VstsBuildTaskServiceEndpoint.VstsBuildTaskServiceEndpointCredentialProvider, cannot provide credentials for https://pkgs.dev.azure.com/OurCompany/{Guid1}/_packaging/{Guid2}/nuget/v3/index.json 2020-02-18T18:36:41.1675261Z     [CredentialProvider]VstsBuildTaskCredentialProvider - https://dev.azure.com/OurCompany/ 2020-02-18T18:36:41.1675425Z     [CredentialProvider]VstsBuildTaskCredentialProvider - https://pkgs.dev.azure.com/OurCompany/ 2020-02-18T18:36:41.1675722Z    [CredentialProvider]VstsBuildTaskCredentialProvider - https://pkgs.dev.azure.com/OurCompany/ 2020-02-18T18:36:41.1676021Z    [CredentialProvider]VstsBuildTaskCredentialProvider - https://pkgs.dev.azure.com/OurCompany/ 2020-02-18T18:36:41.1676161Z    [CredentialProvider]VstsBuildTaskCredentialProvider - Matched prefix: https://pkgs.dev.azure.com/OurCompany/ 2020-02-18T18:36:41.1676330Z    [CredentialProvider]VstsBuildTaskCredentialProvider - Found credentials for endpoint https://pkgs.dev.azure.com/OurCompany/{Guid1}/_packaging/{Guid2}/nuget/v3/index.json 2020-02-18T18:36:41.1680820Z   https://pkgs.dev.azure.com/OurCompany/{Guid1}/_packaging/{Guid2}/nuget/v3/index.json: Unable to load the service index for source https://pkgs.dev.azure.com/OurCompany/{Guid1}/_packaging/{Guid2}/nuget/v3/index.json. 2020-02-18T18:36:41.1680996Z   Response status code does not indicate success: 404 (Not Found - VS800075: The project with id 'vstfs:///Classification/TeamProject/{Guid1}' does not exist, or you do not have permission to access it. (DevOps Activity ID: 0628EDE7-6E7E-465D-A6AE-7B6B0AA065D2)).

```

This looks like a permissions thing to me. But what seems odd is that the Artifact Feed in RED was discoverable when creating the Pipeline Build YAML when we then don't have permissions to use it.

I did try adding permissions to the Feed in RED. Clicking on the "Add users/groups". I found the following and added them as either Reader or Contributor, but with no success.

1. \[GREEN\]\ Green Team
2. GREEN Build Services

So:

1. Am I going about this the wrong way? If so, what ought I be doing (I thought I'd followed the Azure DevOps instuctions fairly closely)
2. Am I going about this the right way? If so, what have I missed?

**Update #1**

Verbose error message from Restore step

> https://api.nuget.org/v3/index.json: Package 'Package1.nnnn.n.n' is not found on source 'https://api.nuget.org/v3/index.json'. https://pkgs.dev.azure.com/OurCompany/{Guid1}/\_packaging/{Guid2}/nuget/v3/index.json: Unable to load the service index for source https://pkgs.dev.azure.com/OurCompany/{Guid1}/\_packaging/{Guid2}/nuget/v3/index.json. Response status code does not indicate success: 404 (Not Found - VS800075: The project with id 'vstfs:///Classification/TeamProject/{Guid1}' does not exist, or you do not have permission to access it. (DevOps Activity ID: {Guid})).

\** Update #2\**

Added `BLUE Build Service (OurCompany) : (Contributor)` to the permissions above.

# Answer
Very few people were viewing this thread, so afterwards I posted a duplicate question on Stack Overflow, and now contains the answer. Below is a copy of it ...

Finally, I found the problem. I wonder if this is a bug, or just poorly documented....(or maybe I just didn't interpret it correctly).

As per Leo Lui-MSFT's suggestion above (documented by Microsoft), I'd added the permissions to the NuGet feed, and this didn't work. However, I was intrigued by his sentence:

> If you've changed your builds to run at project scope

Personally, I hadn't, so I got with our System Administrator and we went through all the possible settings. Eventually, we found it - it wasn't something I had the ability to change.

So in my example above, BLUE and GREEN are the two projects that are trying to use the NuGet package in RED's Artifact feed.

So picking BLUE (or GREEN), in the bottom left hand corner of Azure DevOps you'll see a link "Project Settings". Under "Pipelines \> Settings" there are four settings. We had to change the following to be:

> Limit job authorization scope to current project = false

If the option is greyed out in the project, that's probably because it's enforced at organization level. In that case, you have to first change it in the organization, and *then change it in the project as well*.

Hope that helps!
> 6 votes

---
Tags: azure-devops, azure-pipelines, nuget, azure-artifacts
---


# Question
Title: Where do you store your configurations? Dockerfiles, gitlab-ci, application.properties, deploy scripts
Let´s suppose that our repository contents is something that will be shared. Is strange that we include the deploy scripts there, explaining how to deploy in our machines

# Answer
> 1 votes
If by "Shared" you mean that your team has access to the files then it should be stored in Source Control so that everyone on the team is using the same scripts. You can consider a secrets manager so that team members do not share sensitive information (e.g., Docker Secrets, AWS Secrets Manager, Azure Key Vault).

If this will be publicly available you can still use a secrets manager to publish these files with the sensitive data removed.

---
Tags: configuration-management
---


# Question
Title: How to avoid OOMKilled with azcopy on k8s
When running `azcopy copy` with huge repos, or when running `azcopy bench`, the pods are killed with the error `OOMKilled`. (azcopy v10)

The pod has to following resource limitations:

```
limits: { cpu: 1, memory: 2Gi }

```

# Answer
> 1 votes
`azcopy` (v10) tries to use all available memory to buffer the blobs on transfer. It seems that the calculated memory size is incorrect and exceeds the pod memory limit.

This can be controlled with the option `AZCOPY_BUFFER_GB`, as described in the `azcopy env` output below.

> INFO: Name: AZCOPY\_BUFFER\_GB
>
> Description: Max number of GB that AzCopy should use for buffering data between network and disk. May include decimal point, e.g. 0.5. The default is based on machine size.

So for a pod limit of 2GiB, you can set the buffer a bit lower:

```
export AZCOPY_BUFFER_GB=1.5
azcopy bench ...

```

---
Tags: kubernetes, azure-devops, azure
---


# Question
Title: What is the difference between a jail environment in linux and docker container?
I am curious about difference between a jail environment in linux and Docker. What separates these two technologies?

# Answer
While both Docker and jails have some similar functionality, they are built for different purposes.

The term "jail" comes from the FreeBSD distro. A lot of people will also refer to a chroot as a jail. At its core these jails virtually change the root directory for a process and its children. This means that these processes are contained (jailed) within this new directory and should not have access to folder outside of it. Jails are a great way to run a process that you want isolated from the rest of the system.

Docker Linux is an entire platform for running containers. At its core it does this by using Linux namespaces and Linux Control Groups (source). Docker is built around packaging these containers and allowing them to easily be shared with other systems.

Ultimately, both technologies are isolating processes. However, Docker does this with the intent of making the isolated processes portable and repeatable.
> 4 votes

---
Tags: docker, containers, linux, openshift, centos
---


# Question
Title: CodeReady Container for OpenShift won't start on Windows 10
Two different things might happen:

1 - On the output, get the following messages

```
ERRO unknown error occurred
ERRO unknown error occurred
ERRO unknown error occurred
ERRO unknown error occurred

```

2 - On `~.crc/crc.log`, see the following errors:

```
Error dialing TCP: dial tcp [removed]:22: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or es
tablished connection failed because connected host has failed to respond.

```

# Answer
It took me some time to figure these out, so sharing
> 1 votes

---
Tags: openshift
---


title: AWS API Gateway as an http proxy to ALB

text:
# Question
Title: AWS API Gateway as an http proxy to ALB
I have a setup of API Gateway proxying all the requests to my EC2 instances behind load balancer (ALB). But I want api endpoint to have a nice domain `my-api.mydomain.com`. I'm trying to accomplish that via custom domains.

Then I register `Target Domain Name` generated in Route53 like this

and my http proxy setup is just a passthrough to the ALB which manages requests to underlying EC2 instances providing business logic.

This setup doesn't work, so I'm wondering what's wrong with this configuration and what critical conceptual thing I have missed. Doing `curl -SL https://my-api.mydomain.com/v1/records/` (or any api endpoint) throws `{"message": "Internal server error"}`. CloudWatch logs shows

> Execution failed due to configuration error: Host name 'my-api-lb-111222333.us-east-1.elb.amazonaws.com' does not match the certificate subject provided by the peer (CN=\*.mydomain.com)

I'd be grateful for any suggestions.

# Answer
> 2 votes
Try directly accessing `https://my-api-lb-111222333.us-east-1.elb.amazonaws.com` from your web browser and you'll get a similar error. The SSL certificate is not just used for encryption -- it also verifies that the host is not an impostor, by being consulted in a check of whether the hostname used to access the target matches the hostname of the certificate that's offered by the target

Because amazonaws.com is not your domain, you can't get a certificate for that.

You need to assign a domain name from your domain to the ALB in DNS, e.g. `origin.example.com`, and point API Gateway to that name, not to the system-assigned hostname of the ALB.

---
Tags: amazon-web-services, amazon-ec2, load-balance, api, aws-elb
---


# Question
Title: Why not run a stateless Hashicorp Vault in production?
I'm contemplating whether to provision Hashicorp Vault from code, from scratch, and seed it with keys to be shared between a few, predefined consumers (some humans, some services). The provisioning step has secure access to keys already and can use them to initialize Vault. My reasoning is that this would make it operationally simpler to run -- no backups or risk of configuration drift -- and can act as an on-ramp to using Vault in more places.

Is this feasible? In Hashicorp's documentation for Vault, the in-memory storage option is not recommended for production use because the data will be lost on shutdown. Is there data that is lost that can't be re-provisioned every time?

# Answer
> 1 votes
I think you're misunderstanding what is considered bad practice.

Running vault with an in memory backend and being able to spin it up and provision it is *fine*. What is *Not Fine* is seeding the secrets in Vault from an insecure source.

If you are just using Vault as a way to simplify operations, as opposed to a long term persistent state store - then go for it.

One caveat - if you use the capabilities in vault to rotate keys for other integrated services then if you lose your vault cluster then you have just lost the keys to those services, so hopefully anything accessed using a Vault managed, regularly rotated, key is also easy for you to reprovision

---
Tags: hashicorp-vault
---


# Question
Title: Should I chunk my docker images in many small layers instead of one large?
When I build docker image for existing applications I try to use as few layers as possible and clean up any unwanted files. For example building an image for moodle:

```
# Dockerfile for moodle instance.
# Forked from Jonathan Hardison's <jmh@jonathanhardison.com> docker version. https://github.com/jmhardison/docker-moodle
#Original Maintainer Jon Auer <jda@coldshore.com>

FROM  php:7.2-apache

# Replace for later version
ARG VERSION=37
ARG DB_TYPE="all"

VOLUME ["/var/moodledata"]
EXPOSE 80

ENV MOODLE_DB_TYPE="${DB_TYPE}"
# Let the container know that there is no tty
ENV DEBIAN_FRONTEND noninteractive \
    MOODLE_URL http://0.0.0.0 \
    MOODLE_ADMIN admin \
    MOODLE_ADMIN_PASSWORD Admin~1234 \
    MOODLE_ADMIN_EMAIL admin@example.com \
    MOODLE_DB_HOST '' \
    MOODLE_DB_PASSWORD '' \
    MOODLE_DB_USER '' \
    MOODLE_DB_NAME '' \
    MOODLE_DB_PORT '3306'

COPY ./scripts/entrypoint.sh /usr/local/bin/entrypoint.sh

RUN echo "Build moodle version ${VERSION}" &&\
    chmod +x /usr/local/bin/entrypoint.sh &&\
    apt-get update && \
    if [ $DB_TYPE = 'mysqli' ] || [ $DB_TYPE = 'all' ]; then echo "Setup mysql and mariadb support" && docker-php-ext-install pdo mysqli pdo_mysql; fi &&\
    if [ $DB_TYPE = 'pgsql' ] || [ $DB_TYPE = 'all' ]; then echo "Setup postgresql support" &&\
        apt-get install -y --no-install-recommends libghc-postgresql-simple-dev &&\
        docker-php-ext-configure pgsql -with-pgsql=/usr/local/pgsql &&\
        docker-php-ext-install pdo pgsql pdo_pgsql; \
    fi &&\
    apt-get -f -y install --no-install-recommends rsync unzip netcat libxmlrpc-c++8-dev libxml2-dev libpng-dev libicu-dev libmcrypt-dev libzip-dev &&\
    docker-php-ext-install xmlrpc && \
    docker-php-ext-install mbstring && \
    whereis libzip &&\
    docker-php-ext-configure zip --with-libzip=/usr/lib/x86_64-linux-gnu/libzip.so &&\
    docker-php-ext-install zip && \
    docker-php-ext-install xml && \
    docker-php-ext-install intl && \
    docker-php-ext-install soap && \
    docker-php-ext-install gd && \
    docker-php-ext-install opcache && \
    echo "Installing moodle" && \
    curl https://download.moodle.org/download.php/direct/stable${VERSION}/moodle-latest-${VERSION}.zip -o /tmp/moodle-latest.zip  && \
    rm -rf /var/www/html/index.html && \
    cd /tmp &&  unzip /tmp/moodle-latest.zip && cd / \
    mkdir -p /usr/src/moodle && \
    mv /tmp/moodle /usr/src/ && \
    chown www-data:www-data -R /usr/src/moodle && \
    apt-get purge -y unzip &&\
    apt-get autopurge -y &&\
    apt-get autoremove -y &&\
    apt-get autoclean &&\
    rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* cache/* /var/lib/log/*

COPY ./scripts/moodle-config.php /usr/src/moodle/config.php
COPY ./scripts/detect_mariadb.php /opt/detect_mariadb.php

ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
CMD ["/usr/sbin/apache2ctl", "-D", "FOREGROUND"]

```

Source

But when I build images for an another (production project) I noticed that once I pull newer images for existing containers only the changed layers are actually being downloaded.

So I wonder wouldn't be better to have a layer for each purpose for example a single layer for the php itself and an another layer for the application itself? So as a result during deployment only the changed parts will be downloaded instead of a HUGE single layer.

But as a downside splitting my builds on smaller layers may require to re-download and delete many required packages for building php extentions. Do you think on large applications may be a problem or does it worth to have longer builds for smaller download during deployment or I am talking nonsense?

On the other hand I understand using a single layer makes the overall image size smaller. But as a result it will need to actually download the whole layer itself. So if my image is 100MB it will need to re-download the whole 100Mb instead of few kilobytes that a changed layer will be.

# Answer
> 5 votes
Generally speaking, you want to:

* Combine multiple **related actions/files** into a single layer so you don't have tons of layers.
* Separate out unrelated actions/files which are likely to change independently.
* Order layers such that those least likely to change occur first in the file.
* Use multi stage builds to clean up if you have lots of garbage in your image/layers that does not need to be in the final image.

If you have to make a change to anything in the docker file, everything below that point has to be redone; caching is useless at that point. You want to avoid this.

So, rather than trying to combine everything into one layer or something odd like that, you should focus on understanding what is likely to change and how you can reduce the amount of layers you have to rebuild if and when that change happens.

The best explained example I've seen of this is here for Spring Boot.

Details are below... but to summarize; they determined that a huge layer/size was the application dependencies from Maven. They figured most app changes don't require new maven dependencies, so caching the maven dependencies in their own layer would generally make builds and image pulls faster by avoiding re-pulling dependencies. Then when you change the application code itself, it just pulls the later/smaller layers for the app itself which is very efficient.

---

# A Better Dockerfile

A Spring Boot fat jar naturally has "layers" because of the way that the jar itself is packaged. If we unpack it first it will already be divided into external and internal dependencies.

## Dockerfile

```
FROM openjdk:8-jdk-alpine
VOLUME /tmp
ARG DEPENDENCY=target/dependency
COPY ${DEPENDENCY}/BOOT-INF/lib /app/lib
COPY ${DEPENDENCY}/META-INF /app/META-INF
COPY ${DEPENDENCY}/BOOT-INF/classes /app
ENTRYPOINT ["java","-cp","app:app/lib/*","hello.Application"]

```

There are now 3 layers, with all the application resources in the later 2 layers. If the application dependencies don’t change, then the first layer (from BOOT-INF/lib) will not change, so the build will be faster, and so will the startup of the container at runtime as long as the base layers are already cached.

---

There really isn't a magic answer to this. It really just depends on your application and its dependencies and what is likely to change in your development/deployment environment over time.

---
Tags: docker, dockerfile
---


# Question
Title: How to create credentials for Docker Hub?
I was asked to use Docker Hub as part of a build/deploy pipeline. I am supposed to provide some authentication credentials to a third-party tool, so that this tool can push container images to private repos.

Now, I don't see any place where I can generate secrets/API key that are safe to give to that third-party. Am I missing something? In Amazon for example I can create roles with IAM.

Note: The platform I tried to use is banzaicloud.com . Trying to setup a basic Nodejs app following their instructions. As far as I'm concerned, the problem of authentication should not be on the third-party tools side. The same way I can generate API keys for any kind of service and use them in whatever client I want. In the platform I refer to, it seems only the "full docker hub account" credentials can be used instead of some kind of dedicated third party mechanism (like API keys or IAM, etc).

# Answer
> 1 votes
You can generate an access token for docker hub by going here:

https://hub.docker.com/settings/security

And creating a new access token.

It's essentially the same process for organisational access tokens or for personal access tokens IIRC.

Once you have generated the access token you can literally just use that token instead of the password for most docker hub operations (ie. `docker login --username $user --password $token`)

# Answer
> 0 votes
If you are using Jenkins the answer is in this document:

https://getintodevops.com/blog/building-your-first-docker-image-with-jenkins-2-guide-for-developers

---
Tags: dockerhub
---


title: How to calculate burn rate for SLOs?

text:
# Question
Title: How to calculate burn rate for SLOs?
I've read the Google SRE book a few times but I need some clarifications on exactly how to set up the burn rate and understanding how long it'll take to trigger an alert.

Most of my questions are specifically from this section on the book: https://landing.google.com/sre/workbook/chapters/alerting-on-slos/#4-alert-on-burn-rate

1. In Table 5-4, the error rate for 99.9% SLO for burn rate of 1 is equal to 0.1%. I just want to confirm - this 0.1% is coming from 100% - 99.9% where 99.9% is the SLO. Does this mean if some service has a ridiculously lower SLO, (say 90% SLO), then their error rate for burn rate of 1 would be 10%. Am I interpreting this properly?
2. Couple sentences below, the book says

> For burn rate-based alerts, the time taken for an alert to fire is (1-SLO/error ratio) * alerting window size * burn rate.

So if my SLO is 95%, and my error rate is 1 (let's assume all the requests coming up in last 1 hour are errors). Let's assume my burn rate is 1. If I plug these values into the formula, I am getting,

```
(100-95/1) * 1 hour * 1 = 5.

```

This is where my confusion is. Is this 5 hours? Do you convert 1 hour into 60 minutes to get the minutes? Is that how long it'll take Prometheus to fire the first alert?

Furthermore, isn't the detection time too late if it will take 5 hours to get an alert? Maybe some concrete examples on how you use this formula to calculate some real values would be really helpful.

3. The next formula,

> The error budget consumed by the time the alert fires is: (burn rate * alerting window size)/period.

Just want to clarify this - if my burn rate is 1 and my alerting window size is 1 hour, this means, within 5 hours, I would have consumed, (1 * 1)/5 = 20% of my error budget.

Is this right?

4. The table 5-8 has recommendations to send an alert if the long window is 1 hour and the short window is 5 minutes and if the burn rate is 14.4 and the error budget consumed is 2%.

In the graph above, it shows that the error rate for 5 minutes (10%) is higher than the error rate for 1 hour. Why would their error rates be different if the burn rate is same = 14.4? I'm having difficulty understanding this bit.

It also says it will take 5 minutes to alert based on this information. Will this be true for any SLO? Or is it only true for 99.9% SLO?

5. And lastly, just wanted to clarify: The examples in the chapter use this following recording rule, which is for ratio of errors per request.

> job:slo\_errors\_per\_request:ratio\_rate1h{job="myjob"} \> (14.4\*0.001)

If I want to something similar for identifying latency - i.e. ratio of all requests which have a latency of more than 2 seconds, then this would be the following PromQL:

```
(sum by (job, le) (rate(latency_quantile{job="myjob", le="2"}[1h]))
/
sum by (job, le) (rate(request_count{job="myjob"}[1h]))) > (14.4 * 0.001)

```

Does this look right?

Bottom line, I wish this particular chapter, which the authors acknowledge that it has some complex implementations, had some more concrete examples especially around the formulae and the burn rate. Some examples specified in the tables and chart make sense. But some need a little more clarification to understand the nuances (e.g. specifying that a burn rate of 14.4 translates into 2% of the error budget and this is because when you divide by 30/14.4, you get 50 hours of error budget, and 1 hour is 2% of those 50 hours.)

# Answer
> 8 votes
1. Yes, you are interpreting that correctly.
2. When you express your SLO as a percentage, you should express your error rate as a percentage too. So, in you example your error rate of 1 is 100%. That would make the equation:

```
(100-95/100) * 1 hour * 1 = 0.05 hours = 3 minutes

```
3. The period in that equation is the reporting period that you have chosen for your SLO. You have not mentioned what time period you chose. The examples in the workbook are 30 days. If you use a 30 day (720 hour) period in your example, you would consume approximately 0.14% of your error budget in the 3 minutes it takes for the alert to fire:

```
( 1 * 1 hour ) / 720 hours = 0.14%

```
4. The rates are different because they are measured over different time periods. (Note that the graph is using a logarithmic scale, so the actual error rate and the peak of the 5m measurement is 15%, not 10%.)

* Consider what you would measure one minute after the errors started:
        + The 5m measurement would see 4 minutes of no errors plus one minute of 15% errors, which gives an overall error rate of 3%.
        + The 60m measurement would see 59 minutes of no errors plus one minute of 15% errors, which is just an overall error rate of 0.25%.
    * Then 5 minutes after the errors started:
        + The 5m measurement would see 5 minutes of 15% errors, so the overall rate would be 15%.
        + The 60m measurement would see 55 minutes of no errors plus 5 minutes of 15% errors, so the overall rate would be 1.25%. This is actually still too low to trigger the alert, so it will actually take more than 5 minutes to alert.
    * Using the equation from before, the actual time taken to alert is 5.76 minutes:

    ```
    5.76 = (0.001/0.15) * 60 * 14.4

    ```
5. Your PromQL seems to be looking at the good events instead of the bad events, as the le="2" selector is for the bucket counting latencies *less than or equal to* (le) 2 seconds. You may want to create a separate "good\_latency\_rate1h" recording rule for the good events, and then use something like "(1-good\_latency\_rate1h) \> (14.4\*0.001)" for your alerts.

---
Tags: prometheus, sre
---


# Question
Title: GitLab runner not able to fetch AWS ECR images in the current AWS account
GitLab and GitLab runners are running on the same host (I know how bad it is). There is an IAM Role attached to the EC2 on which GitLab is running.

This IAM Role gives the permission to perform some actions on multi-account ECR's.

Runners use docker as executor and assume role perfectly to push,pull images. But, if images need to be pulled/pushed to the account on which GitLab is running, it doesn't work. I first need to pull images on the GitLab host so they are accessible within the runners.

I did a small test: I manually started a container directly on the GitLab host and ran `aws ecr get-login...` and I was able to login on the account.

I've installed amazon-ecr-credential-helper and configured like stated in the doc.

I don't understand why runners cannot pull images from the account...

# Answer
> 1 votes
I use this script in gitlab-ci.yml

```
image: docker:19.03
services:
- docker:19.03.0-dind
before_script:
- $(aws ecr get-login --no-include-email --region eu-central-1)

```

... and provide AWS\_ACCESS\_KEY\_ID, AWS\_SECRET\_ACCESS\_KEY variables into Settings/CI-CD section. Ensure your region with images is the same where are you logging in.

# Answer
> 0 votes
This script can be used as a bootstrap script to register private Gitlab runner and configure runner to access AWS ECR. You can use Ubuntu 18 LTS AMI and launch AWS EC2 instance to host the Gitlab runner.

Attach AWS IAM role to grant EC2 instance to access AWS ECR and push-pull docker images. Also, awscli AWS configure command can be used or export AWS\_ACCESS\_KEY\_ID and AWS\_SECRET\_ACCESS\_KEY as environment variable.

Need to export GITLAB\_RUNNER\_TOKEN and AWS\_REGION as variable to run the script successfully.

```
#!/bin/bash
set -e
apt update -y && apt install awscli make -y
curl -LJO https://gitlab-runner-downloads.s3.amazonaws.com/latest/deb/gitlab-runner_amd64.deb
dpkg -i gitlab-runner_amd64.deb
sed -i -e '/volumes/s/\["\/cache\"]/\["\/cache","\/var\/run\/docker.sock:\/var\/run\/docker.sock"]/' /etc/gitlab-runner/config.toml
curl -fsSL https://get.docker.com -o get-docker.sh && sh get-docker.sh && usermod -aG docker gitlab-runner
gitlab-runner register \
  --non-interactive \
  --url "https://gitlab.com/" \
  --registration-token "${GITLAB_RUNNER_TOKEN}" \
  --executor "docker" \
  --docker-image "docker:latest" \
  --description "gitlab-runner" \
  --tag-list "gitlab,runner,aws" \
  --run-untagged="true" \
  --locked="false" \
  --access-level="not_protected"
echo -e "export AWS_SDK_LOAD_CONFIG=true \nexport AWS_REGION=${AWS_REGION}" >> ~/.bashrc
source ~/.bashrc && mkdir ~/.docker
cat <<EOF >>~/.docker/config.json
{
  "credsStore": "ecr-login"
}
EOF
git clone https://github.com/awslabs/amazon-ecr-credential-helper.git
cd amazon-ecr-credential-helper/ && make docker
cp bin/local/docker-credential-ecr-login /usr/local/bin/
systemctl restart docker  && gitlab-runner restart
export AWS_REGION=${AWS_REGION} && docker-credential-ecr-login list

```

---
Tags: docker, gitlab-ci-runner, aws-ecr
---


# Question
Title: How To Determine Min/Max Resources For Kubernetes Pods?
Say I have 2 nodes for a kubernetes cluster. Both has

```
2 CPU → 4 Total
2G RAM → 4 Total

```

I have 4 pods to shcedule-- no replicas yet, 1 major pod that'll use 80% of resources, other use 5% each. What resource limits should I specify for these pods?

**CURENT USAGE STATS**

```
# kubectl top pod --all-namespaces
NAMESPACE       NAME                        CPU(cores)   MEMORY(bytes)
default         pod1-64c455bcf8-mzxqc           1m          122Mi
default         pod2-6f7b44478b-pn4tt           1m          33Mi
default         pod3-867c956697-mf6zw           0m          48Mi
default         pod4-844bd559dd-z96fh           0m          35Mi

```

# Answer
> 1 votes
The DevOps toolkit 2.3 book by Victor Farcic goes into some detail on this.

# What You Should Understand

* You as a developer specify whatever requests/limits you like initially.
* These values have no baring on reality.
* In order to get "real" values, you must monitor your application's real resource usage.
* This is best done through a monitoring tool like prometheus, instana, datadog etc.
* E.g. in instana, I can see that I have 512MB set up for a pod's requests, 2G for its limits, but the pod only has one container currently taking 57MB.
* But I can't just use 60MB as a value now... I need to load test my application and see how it behaves under real/stressed circumstances.
* Then you review these values and you can adjust the requests/limits accordingly.

# Determine Usage Without a Monitoring Tool

You can use some kubectl commands to get these values as a one off (it's annoying at scale). Here is an example of a different app I have running.

Notice that requests/limits are set to 1/4-2 CPUs and 4G-8G memory. But the container is really using almost no CPU and 1.198G memory. If I load tested this and those values did not move up, I would need to change them. But I have load tested it and know that they would go up towards these realistic requests/limits, so it's okay.

```
$ kubectl describe pod -n gazelle gazelle-ws-0-0-1-6d67cffcf-lvlhk

...
    Limits:
      cpu:     2
      memory:  8Gi
    Requests:
      cpu:      250m
      memory:   4Gi

$ kubectl top pod gazelle-ws-0-0-1-6d67cffcf-lvlhk -n gazelle --containers

POD                                NAME         CPU(cores)   MEMORY(bytes)
gazelle-ws-0-0-1-6d67cffcf-lvlhk   gazelle-ws   5m           1198Mi

```

---
Tags: kubernetes
---


# Question
Title: Jenkins view is getting duplicate every container restart
I'm using Jenkins container and facing an issue that each container restart, the RunningJobs getting duplicate:

Any idea what is the root cause and how can I avoid it?

# Answer
> 1 votes
The solution was to remove the directory `init.groovy.d` from Jenkins home folder (inside the container)

---
Tags: jenkins
---


# Question
Title: Active directory integration issue with Jenkins
The active Directory login is not working for us on DMZ server since the request is going to multiple domain controllers servers (which cause us slowness in login- almost 10 minutes). how can we limit it to only one DC server?

Jenkins version: 2.176.1

# Answer
We had to mention the specific DC on the Site field
> 1 votes

---
Tags: jenkins, active-directory
---


# Question
Title: Prometheus - join series based on the key part of a label
In 1.16 k8s deprecated the label kubernetes.io/role: worker I had previously been using as a filter to target graphs to specific node role using this in my grafana variables declaration

```
label_values(kube_node_labels, label_kubernetes_io_role)
#join series based on the the label value
  sum(kube_pod_container_resource_requests_cpu_cores
  * on (node) group_left (label_kubernetes_io_role) kube_node_labels{label_kubernetes_io_role="$node_role"}

```

In this scary new world of using the label `node-role.kubernetes.io/worker: ""` this no longer works, nor can I necessarily regex it when the part I care about is in the key.

Is there a new way anybody knows of to filter queries by arbitrary labels on the node

# Answer
> 2 votes
```
label_values(kube_node_role, role)

sum(kube_pod_container_resource_requests_cpu_cores
  * on (node) group_left (role) kube_node_role{role="$node_role"})

```

kube\_node\_role out of kube-state-metrics can give you a relationship to join

---
Tags: prometheus
---


# Question
Title: Does Docker images works without client application
I recently installed Docker in Ubuntu 18.04 and with that, I have downloaded the **mariadb** image like this

```
docker run --name mariadb \
-p 3306:3306 \
-v /opt/mariadb_data:/var/lib/mysql \
-e MYSQL_ROOT_PASSWORD=lopaz \
--restart always \
-d mariadb:10.1

```

this is pretty cool method that I used to download from the official **mariadb** image and it's working well. What my question is that I want to use to login to the MariaDB ? when I type

`mysql -h <myhost> -u root -p`

MySQL or MariaDB not found on my local Ubuntu machine. What happened ? how my pc knows that I installed **mariadb** using docker? then I installed **mysql client app** then only I can use `mysql` commands on my local pc! please give me a clarification about why it happens when install using docker?

# Answer
> 2 votes
When you download mariadb image, mysql client tool is only available inside container. It is not getting magically installed on your local - that is why you had to install mysql client app.

Alternative way to connect to db without installing mysql app is to use docker exec and through this get to mysql tool available on container. So you would do something like:

```
docker exec -it your_maria_db_container_id mysql -h localhost -u root -p

```

Hope this helps.

---
Tags: docker, dockerfile, images
---


# Question
Title: upstream connect error or disconnect/reset before headers. reset reason: connection failure
I am deploying a virtual service and gateway but when I run it on browser get the below error:

**upstream connect error or disconnect/reset before headers. reset reason: connection failure**

I deployed with helm chart and the result of dry run is below for virtual service and gateway:

```
# Source: beta-optimus/templates/gateway.yaml
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: beta-api-dev-gateway
spec:
  selector:
      istio: ingressgateway
  servers:
    - port:
         number: 443
         name: https
         protocol: HTTP
      hosts:
         - "*.myapp.xyz"
---
# Source: beta-optimus/templates/virtualService.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: beta-api-dev
spec:
  hosts:
    - "beta-api-dev.myapp.xyz"
  gateways:
    - beta-api-dev-gateway
  http:
    - route:
         - destination:
               host: beta-optimus

```

the beta-optimus is running on 2 pods fine. I am using AWS so I made entry into Route53 for the proposed URL with istio. My other deployments work fine, but getting the error only in this one.

# Answer
I encountered the same error. The Service port needed a name added to it. https://github.com/istio/istio/issues/19966. Make sure your *beta-optimus* service port has a name.

https://istio.io/docs/ops/deployment/requirements/

```
ports:
    - name: https # Use http or https
       protocol: TCP
       port: 8080
       targetPort: 8080

```
> 1 votes

---
Tags: kubernetes, istio
---


# Question
Title: create entry-point script to get all env variables starting with VUE_APP_ and take it's value
I would like to Dockerize my existing vuejs app for staging , production and I used multi staged approach to create my app container and I'm using Ubuntu 18.04 and this is my,

### Dockerfile

```
# build stage
FROM node:lts-alpine as build-stage

LABEL maintainer="servers@xpl.com"

WORKDIR /app

COPY package*.json ./

RUN npm install

COPY . ./

RUN npm run build

# production stage
FROM nginx:stable-alpine

LABEL maintainer="servers@xpl.com"

COPY --from=build-stage /app/dist/ /app/dist/

COPY nginx/nginx.conf /etc/nginx/

EXPOSE 80

```

I don't want to create `docker-compose.yml` file to assign anything and my question would be like , Create `.env` file with hardcoded values.

**i.e**

```
VUE_APP_API_ENDPOINT_DOMAIN=VUE_APP_API_ENDPOINT_DOMAIN

```

1 .When running npm run build it'll put VUE\_APP\_API\_ENDPOINT\_DOMAIN in the dest files.

2 .We need to know which file. Probably index.html I believe.

3 .We create entry-point.sh script will get all env variables starting with `VUE_APP_` and take it's value. And then using **`sed`** command we find and replace those hard coded variable placeholders in index.html. And finally start Nginx.

**I was stuck on step 3.**

# Answer
> 1 votes
While it's not clear what your trying to do (the script you have so far for the bit you are having trouble with would probably be handy), I *think* you are essentially just trying to parse your environment and manipulate the variables at the start of your script?

I don't see why add something like this to the first line of your script wouldn't work:

```
eval $(typeset -p | grep -E 'declare -x VUE_APP_' | sed -e 's#declare -x VUE_APP_#export VUE_APP_#g')

```

But then I also don't know what shell you're running or anything else about the container, so I'm not sure..

---
Tags: docker, dockerfile, shell-script, bash
---


title: ansible filter debug output

text:
# Question
Title: ansible filter debug output
I have the following output resulted from a shell command which is run on ansible on every host from my inventory:

```
TASK [debug]
ok: [10.240.22.44] => {
    "msg": {
        "changed": true,
        "msg": "All items completed",
        "results": [
            {
                "_ansible_ignore_errors": true,
                "_ansible_item_result": true,
                "_ansible_no_log": false,
                "_ansible_parsed": true,
                "changed": true,
                "cmd": "/opt/confluent/bin/nodefirmware smm1",
                "delta": "0:00:00.128325",
                "end": "2020-02-05 11:22:19.435049",
                "failed": false,
                "invocation": {
                    "module_args": {
                        "_raw_params": "/opt/confluent/bin/nodefirmware smm1",
                        "_uses_shell": true,
                        "chdir": null,
                        "creates": null,
                        "executable": null,
                        "removes": null,
                        "stdin": null,
                        "warn": true
                    }
                },
                "item": "10.240.18.20",
                "rc": 0,
                "start": "2020-02-05 11:22:19.306724",
                "stderr": "",
                "stderr_lines": [],
                "stdout": "smm1: SMM: 1.10 (TESM14F)\nsmm1: PSOC: 0.7",
                "stdout_lines": [
                    "smm1: SMM: 1.10 (TESM14F)",
                    "smm1: PSOC: 0.7"
                ]
            },
            {
                "_ansible_ignore_errors": true,
                "_ansible_item_result": true,
                "_ansible_no_log": false,
                "_ansible_parsed": true,
                "changed": true,
                "cmd": "/opt/confluent/bin/nodefirmware smm1",
                "delta": "0:00:00.096292",
                "end": "2020-02-05 11:22:22.847949",
                "failed": false,
                "invocation": {
                    "module_args": {
                        "_raw_params": "/opt/confluent/bin/nodefirmware smm1",
                        "_uses_shell": true,
                        "chdir": null,
                        "creates": null,
                        "executable": null,
                        "removes": null,
                        "stdin": null,
                        "warn": true
                    }
                },
                "item": "10.240.19.21",
                "rc": 0,
                "start": "2020-02-05 11:22:22.751657",
                "stderr": "",
                "stderr_lines": [],
                "stdout": "smm1: SMM: 1.10 (TESM14F)\nsmm1: PSOC: 0.7",
                "stdout_lines": [
                    "smm1: SMM: 1.10 (TESM14F)",
                    "smm1: PSOC: 0.7"
                ]
            }
        ]
    }
}

```

The information I need to extract and display on screen is this (for every host):

```
"stdout_lines": [
    "smm1: SMM: 1.10 (TESM14F)",
    "smm1: PSOC: 0.7"

```

The previous command with the full output is registered as a variable using ansible register module:

```
- name: Get smm version
  shell: some command
  register: smm_output

```

Then I'm trying to display it using (which displays the full output):

```
- debug:
    msg: "{{ smm_output }}"

```

If I'm using msg: "{{ smm\_output.results.stdout\_lines }}" it returns error

I also tried with **json\_query** filter but with no results

So how do I parse that output?

# Answer
The registered variable `smm_output` is a dict object and you can correctly dive into it. But you cannot use `smm_output.results.stdout_lines` because `smm_output.results` is a list. You need to iterate over it or filter the first element of results. So, if you know its only one element in the list, you should use `smm_output.results.0.stdout_lines`.
> 2 votes

# Answer
So there's something wild going on there with how that command is being called. It's not just being executed on each host (as one night expect) but it looks sus to me because the `ok` is from `10.240.22.44` but the contents of the variable indicates that the commands actually ran on `10.240.18.21` and `10.240.19.20`.

Which I can only imagine happening if you were to run the task something like this:

```
- name: run command
  shell: /opt/confluent/bin/nodefirmware smm1
  loop:
    - 10.240.18.21
    - 10.240.19.20
  register: smm_output

```

I assume that's not what you're doing because the loop there is pointless, but if you were intending to run the command on each host then not looping and just using the variable would work.

In the likely scenario that you need to run the command on each host individually 1 at a time then you should be able to delegate the facts to those hosts and look it up in the `hostvars`

Alternatively if the command really does need to run from that one host, registering results when looks are involved is a known PITA that I believe the ansible docs cover what the result is and it's pretty much universally going to be a pain to parse later.

However I suspect that there's something else going on here and this is an XY Problem type thing and you are trying to achieve something specific but are now Yak Shaving other problems.
> 0 votes

---
Tags: ansible
---


title: Provisioning raspbery pi sdcard

text:
# Question
Title: Provisioning raspbery pi sdcard
I was searching for a tool suitable for provisioning sdcard for raspberry pi.

I've successfully managed to implement most of it using Ansible playbook. Need only some time for last polishing.

Process looks like this:

1. Format sdcard
2. Mount sdcard in temporary directory
3. Debootstrap sid
4. Chroot
5. Finish configuration

It seems to work well for now, but I'm a bit worried that it is not in the scope of Ansible. Is there any more suitable tool for this task?

# Answer
> 6 votes
If you're previsioning your raspberry pi SD cards then you would probably be best using something like packer to build the actual image (you can use ansible as the provisioner for packer if you are really keen) then once you have your pi image, burn it onto the SD card.

If you need to make subsequent configuration changes them that would be where ansible fits most neatly in managing the ongoing configuration of the pi's

---
Tags: ansible, provisioning
---


# Question
Title: How do I add my IP address to the start of my terminal using the bashrc file?
I want to customise my terminal on an Amazon linux device. I have colours using `$(tput setaf 33);`, user name with `/u` but the IP address not there.

```
#blue=$(tput setaf 33);
PS1="\[${blue}]\u$(tput sgr0)"
PS1="${blue}[\u@\w $(tput sgr0)"
PS1+=""
export PS1;

```

# Answer
> 2 votes
Well... you would need to know what IP address you actually want. Is it a public IP address? Is it a private subnet address? Do you only have 1 IP address? A few things you would need to figure out.

But, if we just assume that we want to have whatever ip address your VM thinks it's address is then:

Add the following to your .bashrc

```
PS1="\[$(tput setaf 33)\][\u@$(ip -4 -o addr show scope global $(ip -4 route | grep default | head -n1 | awk '{print$5}') | awk '{gsub(/\/.*/, "",$4); print $4}' | head -n 1) \w ]\[$(tput sgr0)\] \[$(tput setaf 34)\]\\$\[$(tput sgr0)\] "

```

To throw an IP address into your prompt.

Now, because that's not really human friendly, let's break it down:

Make the prompt blue (honestly I can't remember why I always wrap with `\[ \]` I just remember someone telling me I needed to do that, so I guess see what works for you:

```
\[$(tput setaf 33)\]

```

This bit you already had figured out:

```
[\u@

```

This insanity to find the default IP address, it's a bit hard to break down without just reading through the `ip` man pages but basically we are searching for the interface marked as default then grabbing the IP address and stripping all the extra stuff out with awk/grep/head:

```
$(ip -4 -o addr show scope global $(ip -4 route | grep default | head -n1 | awk '{print$5}') | awk '{gsub(/\/.*/, "",$4); print $4}' | head -n 1)

```

I tend to want to know my ${PWD}:

```
\w ]

```

Reset our colour scheme just to be stafe (I've found weird stuff happens when you keep layering colours without resting:

```
\[$(tput sgr0)\]

```

Set some other colour:

```
\[$(tput setaf 34)\]

```

Set the `$` or `#` prompt which is handy if you want it to be obvious if you are logged in as root:

```
\\$

```

Then reset colour again:

```
\[$(tput sgr0)\]

```

---
Tags: bash
---