# Question
Title: AWS Credentials - What are multi region user/role management best practices?
In the past, I've usually had separate accounts for each region that I've worked in. These accounts had different IAM roles/etc, so there was very little chance I'd inadvertently mess up region A when working on region B and vice versa.

Recently, we deployed to another region *within the same account*. This means, without extra controls, just setting the default region differently on your CLI could have fairly disastrous consequences.

While IAM is global, is it generally recommended to have distinct uses per region / restrict which resources they can hit with policies, or is that overkill? Are there any good tools for helping you "not mess up" when running anything on the CLI? (including things like terraform)? Do you know any other "best practices" or good resources to learn about this particular area?

# Answer
Start by reading this: https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html

Those are best-practices and do not take into account business policies.

I don't think having one account per region is more secure than having policies scoped by regions. In cli, you will have to specify the region in the command anyway (not sure if AWS cli uses a default region if you don't specify one in your credentials file).
> 2 votes

---
Tags: amazon-web-services, credentials
---


# Question
Title: What approach should I take to store environment variables for a repo?
I am setting up a new infrastructure as code repository for a small to medium size project. The problem I am currently trying to solve is how to store my environment variables such that they can be read and used by all services and scripts within the repo. Some of the tools being used are Ansible, Powershell, Bash...

My current solution is a simple text file, manually created per environment. Each line has a variable that the services and scripts can look up. However, this seems clunky, as I have to create the logic to read this file in every script that is made.

So my question is, what approach should I be taking to provide all my services with environment variables? Am I on the right track, or should I be using something entirely different for this?

I'm not sure what I can use to provide this to all the different tools. For details sake, I am not talking about secrets management. I mean variables such as: DNS address of the environment's DC, log server address, etc... There are significant constraints on addition of new software/tools due to the sensitivity of the environment. For example, I was considering using JSON format for the environment file, but Bash does not parse that natively. Meaning I would have to install a third party tool like 'jq' to use.

Any feedback welcome. Thank you!

# Answer
> 4 votes
## General Notes

I don't really understand this one:

> However, this seems clunky, as I have to create the logic to read this file in every script that is made.

You can source all variables from a bash file in one line (source file-name.sh). If they are "exported", you can pick them up in applications in any language you like (Java, python, or whatever), and most have built in syntax for doing it (like spring boot's SPEL syntax which lets you define your config to directly populate from environment variables automatically on start-up).

You can also make this export part of your ~/.bashrc file or whatever so it's auto sourced in your shell contexts, etc.

## Repositories + CI/CD

This may depend on your repository and CI implementation more than the installed code. For example, GitLab CI and GitHub Actions both support environment variables, as does Jenkins, etc.

In GitLab you can even go to your group level (above a bunch of related projects), and set per-environment environment variables there which are pushed into your deployment so you can use them in your installed code. Then, you don't even need to store them in any extra files.

References:

## Docker

Whatever your code is, you can probably bundle it into docker and pass the variables to the container at run time.

---
Tags: git, infrastructure-as-code, environments, environment-variables, repository
---


# Question
Title: Is it possible to set role in FreeIPA to use for applications as Jenkins, Github?
LDAP in FreeIPA, it can create groups, roles and set permissions. Applications such as Jenkins, Github also has roles feature.

2 questions:

1. Can roles feature be used in every application? (At least Jenkins and Github may use groups feature in LDAP)
2. If want to control several users can access Jenkins, several users can access Github, is it possible to control it in LDAP side? If deploy multiple LDAP server for different applications will realize it, but that's complex.

# Answer
> 1 votes
Based on your title, no.

1. It's impossible to quantify if every application has this capability. Many do.
2. Users/Groups in Jenkins OSS map back to LDAP but you control the permissions via Jenkins.

As you can see above Users/Groups have specific permissions that are specified/controlled in Jenkins. If this group exists in LDAP (xxx) the users in this group get the permission you setup in Jenkins. This makes it easier then managing individuals.

---
Tags: jenkins, github, role-based-access-control
---


# Question
Title: Using Jenkins declarative when conditions in scripted pipelines
Jenkins declarative pipelines offer very convenient **when** directive with conditions like **changeset**.

Is is possible the this changeset and other when conditions in a scripted pipeline?

# Answer
> 1 votes
It would seem not - in the scripted pipeline, groovy needs to be used - see the docs

```
node {
    stage('Example') {
        if (env.BRANCH_NAME == 'master') {
            echo 'I only execute on the master branch'
        } else {
            echo 'I execute elsewhere'
        }
    }
}

```

(taken straight from the docs)

This would be the same as `when { branch 'master' }`

---
Tags: jenkins, jenkins-pipeline, jenkinsfile
---


# Question
Title: On Premise CI / CD recommendations for Docker images
We have TFS building Docker images, and loading them into a Private Registry on one network, our Development Network. Everything works well, and we are generally happy with the CI process in that environment. We have an Air-gapped Integration Network that we need to export these Docker images to, and then have something that finds them, puts them into a registry there, and then restarts a Docker swarm.

I am looking for a solution that can:

1. Watch a folder for new files
2. When a new Docker Tar file is found, load it into Docker Registry.
3. Move the Docker image that was loaded to an "archive" folder
4. Restart Docker swarm to use new images in the Registry

It seems like step 2 might not be possible directly, I might need to load it into a local Docker instance, tag it, and then push it, but if so, that's fine.

Is this something I should Jenkins for? I could just write a script in Bash or Python, but I would like to use a proper CI tool, if I can.

# Answer
## Jenkins Option

If you're comfortable with Jenkins and want to use a CI tool, it is a decent option.

It has multiple plugins for watching for files arriving in a file system, and at the end of the day, you can even just wrap your bash scripts into it so you have a central GUI and recorded logs. So, you're not limited in any way by it.

## Inotify Option

I personally might just have a CI tool like Jenkins or GitLab-CI **deploy** a bash script though. Then you could use **inotify** to properly watch a folder and execute your script exactly when things arrive. See: https://www.howtogeek.com/405468/how-to-perform-a-task-when-a-new-file-is-added-to-a-directory-in-linux/ for an example.

## Why iNotify?

In many tools, I believe you generally have to "poll", which means run over and over just to see if you have anything to do. Inotify lets you genuinely "watch" and be pushed events only when things happen. This is cleaner and more efficient as you won't have hundreds or thousands of tasks running a day that literally do nothing.

## Monitoring

Note: One plus side to the pure Jenkins approachis built in monitoring. Every job run that fails can easily email/alert you. You can pull this off with the iNotify script as well, but if you don't have the right tooling for monitoring/etc in your environment, you may favor the Jenkins approach.
> 1 votes

---
Tags: docker, continuous-integration, continuous-delivery
---


# Question
Title: Is it possible to create a module that either creates a resource or sources a data source, based on a boolean value?
Basically, what I'd like to do is create a module that does this pseudocode:

```
if <boolean> then
  resource "<resource type>" "name" {
    # ... set variables ...
  } else {
  data "<data source>" "name" {
    # grab data
  }

output "resource_type.name" {
  if <boolean> then
    value = resource_type.name
  else
    value = data_source.name
}

```

I hope I'm being clear. Happy to add more info to clarify what I'm hoping is possible.

# Answer
You can use the count meta-parameter to achieve the effects of an if-else statement. It is talked about in detail in the link below and I also pulled one relevant example out.

I'm not 100% sure if you can give both the data source and resource the same name without a conflict; but I assume it would work. It may be dependent on the resource you're creating though, not sure.

https://blog.gruntwork.io/terraform-tips-tricks-loops-if-statements-and-gotchas-f739bbae55f9

## Excerpt:

```
# This is just pseudo code. It won't actually work in Terraform.
if var.give_neo_cloudwatch_full_access {
  resource "aws_iam_user_policy_attachment" "neo_cloudwatch_full" {
    user       = aws_iam_user.example[0].name
    policy_arn = aws_iam_policy.cloudwatch_full_access.arn
  }
} else {
  resource "aws_iam_user_policy_attachment" "neo_cloudwatch_read" {
    user       = aws_iam_user.example[0].name
    policy_arn = aws_iam_policy.cloudwatch_read_only.arn
  }
}

```

To do this in Terraform, you can use the count parameter and a conditional expression on each of the resources:

```
resource "aws_iam_user_policy_attachment" "neo_cloudwatch_full" {
  count = var.give_neo_cloudwatch_full_access ? 1 : 0
  user       = aws_iam_user.example[0].name
  policy_arn = aws_iam_policy.cloudwatch_full_access.arn
}
resource "aws_iam_user_policy_attachment" "neo_cloudwatch_read" {
  count = var.give_neo_cloudwatch_full_access ? 0 : 1
  user       = aws_iam_user.example[0].name
  policy_arn = aws_iam_policy.cloudwatch_read_only.arn
}

```
> 2 votes

---
Tags: terraform, infrastructure-as-code
---


# Question
Title: Nginx ingress controller redirects the url to /login
I am trying to access a url example.com/service/test1 but it redircts me to example.com/login.

Below is my YAMl file:

```
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: test-ing
  namespace: test
  annotations:
     nginx.ingress.kubernetes.io/ssl-redirect: "false"
     nginx-ingress.kubernetes.io/use-regex: true

spec:
  rules:
  - host: example.com
    http:
      paths:
      - backend:
          serviceName: svc-test
          servicePort: 8080
        path: /service/test
  tls:
  - secretName: tls-secret

```

# Answer
# It just works

Your configuration is correct.

Provided `YAML` file generates `nginx` config like this:

```
...
server {
 listen       80;
 server_name  example.com;
 ...
 location ~* "^/service/test" {
   root    /var/www/example.com/htdocs;
 }
}

```

Testing this config via this cool tester shows that location filter works:

* `http://example.com/service/test1` -\> `Location: ^/service/test`
* `http://example.com/service/test` -\> `Location: ^/service/test`
* `http://example.com/login` -\> `Errors: No locations matched`

For more details, see this explanation of Ingress Path Matching and examples from NGINX Docs:

> * *Examples* show how to use advanced NGINX features in Ingress resources with annotations.
> * *Examples of Custom Resources* show how to use VirtualServer and VirtualServerResources for a few use cases.

# Check your app

Check your app, maybe it forward non-authorized requests to `/login` url. You may use `curl -v` or `wget`, e.g:

```
curl -v -L http://example.com/service/test1 | egrep "^> (Host:|GET)"

```

```
wget  http://example.com/service/test1 2>&1 | grep Location

```

# Howto debug `nginx ingress`

There is an `Nginx` debug mode for debugging `nginx` `locations` and redirects. To enable it, use `--v=5` in `kubectl edit deploy`.

See: k8s debug logging:

> Using the flag `--v=XX` it is possible to increase the level of logging. This is performed by editing the deployment.
>
> ```
> $ kubectl get deploy -n <namespace-of-ingress-controller>
> NAME                       DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
> default-http-backend       1         1         1            1           35m
> nginx-ingress-controller   1         1         1            1           35m
>
> $ kubectl edit deploy -n <namespace-of-ingress-controller> nginx-ingress-controller
> # Add --v=X to "- args", where X is an integer
>
> ```
>
> > * `--v=2` shows details using `diff` about the changes in the configuration in nginx
> > * `--v=3` shows details about the service, Ingress rule, endpoint changes and it dumps the nginx configuration in JSON format
> > * `--v=5` configures NGINX in debug mode
> 3 votes

---
Tags: nginx, ingress, azure-kubernetes-service
---


# Question
Title: Blacklist network ranges Docker is allowed to use for its containers (swarm mode)
Basically I need to blacklist an IP range (and maybe multiple ranges) so that Docker can use any other IPs for its containers BUT IPs in this range.

Assume I have my docker nodes in a swarm of n nodes. They are in IP range `10.1.1.0/24`. There are multiple other VMs in this IP range like a VPN Server, a firewall, etc.

Can I restrict Docker from assigning IPs from this range to containers it starts?

# Answer
Docker should avoid using a network if there's a route to that network (see `ip r`). You can whitelist networks that docker is allowed to use with configurations in several places:

* You can adjust `bip` in `/etc/docker/daemon.json` which controls the docker bridge network named bridge. This needs to be set to a gateway ip with CIDR notation.
* You can adjust `default-address-pools` in `/etc/docker/daemon.json` which is a list of CIDR blocks for all other bridge networks.
* You can pass `--default-addr-pool` when initializing a swarm to a CIDR block for overlay networks.

Make sure the two pools are large enough to create multiple networks, e.g. a `/16` subnet to allow creation of 256 `/24` networks.

More details of this are available in my DockerCon presentation (press `P` for presenter notes).
> 4 votes

---
Tags: docker, networking, docker-daemon
---


# Question
Title: Creating ec2 instances and installing mongoDB using automation
I am trying to create a 3 node MongoDB replica set using automation (ansible or terraform) in AWS. I am able to do this in the management console of AWS, but I'm not sure how to approach it.

Essentially, the script must create the instances and then set up a 3 node MongoDB replica set on them. I want to add an EBS disk on it and then maybe secure it.

All of this via Ansible (or Terrafor) so I can automate it fully.

Is this possible?

Thanks!!

# Answer
> 1 votes
It's possible. I would use Terraform for creating the infrastructure on AWS and Ansible for provisioning. You will have to write .tf files and ansible-playbooks in order to do this.

Just to let you know, if all 3 nodes are equal you should do the installation and provisioning once and then create an AMI to set up the rest. Maybe an autoscaling group is a possibility too.

Keep in mind that AWS provides documentdb as document-oriented database as a service. It's not mongoDB but it's pretty similar (there are some functions mongo provides and docdb doesn't).

---
Tags: ansible, terraform
---


# Question
Title: get value of default_statistics_target on postrgesql
I want to see the value of `default_statistics_target` on `postgresql` before running

```
SET default_statistics_target=1000

```

Its most likely the default (100) but would like to see it.

Where it can be found ?

# Answer
Actually, there are two questions:

* Where default(100) value is declared?
* How can I see `Stats target` value before applying `SET default_statistics_target=1000`

## Where default(100) value is declared?

You can see it in the source code

> ```
> {
>       {"default_statistics_target", PGC_USERSET, QUERY_TUNING_OTHER,
>           gettext_noop("Sets the default statistics target."),
>           gettext_noop("This applies to table columns that have not had a "
>                        "column-specific target set via ALTER TABLE SET STATISTICS.")
>       },
>       &default_statistics_target,
>       100, 1, 10000,
>       NULL, NULL, NULL
>   },
>
> ```

## How can I see `Stats target` value before applying `SET default_statistics_target=1000` ?

Let's suppose that column `bar` of table `foo` has `attstattarget=500`:

```
alter table foo alter column bar set statistics 500;

```

So, there are two ways to show `attstattarget`

* SQL\_query:

```
SELECT attrelid::regclass, attname, attstattarget FROM pg_attribute WHERE attstattarget > 0 order by attstattarget desc;

 attrelid | attname | attstattarget
----------+---------+---------------
 foo      | bar     |           500

```
* `\d+` command:

```
\d+ foo

                                                            Table "public.foo"
          Column           |            Type             |                  Modifiers                   | Storage  | Stats target | Description
---------------------------+-----------------------------+----------------------------------------------+----------+--------------+-------------
 bar                       | varchar                     | not null                                     | plain    | 500          |

```

\** Note\**: `Postgres` doesn't show `stats target` until it distinct from default value.

## `300` \- yet another magic number

Thx to this comment, you can read a paper about another magic number and check it in the source code:

> ```
> /*
>    * Determine which standard statistics algorithm to use
>    */
>   if (OidIsValid(eqopr) && OidIsValid(ltopr))
>   {
>       /* Seems to be a scalar datatype */
>       stats->compute_stats = compute_scalar_stats;
>       /*--------------------
>        * The following choice of minrows is based on the paper
>        * "Random sampling for histogram construction: how much is enough?"
>        * by Surajit Chaudhuri, Rajeev Motwani and Vivek Narasayya, in
>        * Proceedings of ACM SIGMOD International Conference on Management
>        * of Data, 1998, Pages 436-447.  Their Corollary 1 to Theorem 5
>        * says that for table size n, histogram size k, maximum relative
>        * error in bin size f, and error probability gamma, the minimum
>        * random sample size is
>        *      r = 4 * k * ln(2*n/gamma) / f^2
>        * Taking f = 0.5, gamma = 0.01, n = 10^6 rows, we obtain
>        *      r = 305.82 * k
>        * Note that because of the log function, the dependence on n is
>        * quite weak; even at n = 10^12, a 300*k sample gives <= 0.66
>        * bin size error with probability 0.99.  So there's no real need to
>        * scale for n, which is a good thing because we don't necessarily
>        * know it at this point.
>        *--------------------
>        */
>       stats->minrows = 300 * attr->attstattarget;
>   }
>   else if (OidIsValid(eqopr))
>   {
>       /* We can still recognize distinct values */
>       stats->compute_stats = compute_distinct_stats;
>       /* Might as well use the same minrows as above */
>       stats->minrows = 300 * attr->attstattarget;
>   }
>   else
>   {
>       /* Can't do much but the trivial stuff */
>       stats->compute_stats = compute_trivial_stats;
>       /* Might as well use the same minrows as above */
>       stats->minrows = 300 * attr->attstattarget;
>   }
>
> ```
> 2 votes

---
Tags: postgresql, performance
---


# Question
Title: When is it ethically acceptable to fork someone else github code and continue with that?
https://github.com/kamaln7/ansible-swapfile

Seems not to be maintained. For example, an issue was created in May and no reaction. A new issue has just been created to ask whether this repo is still maintained.

When is it allowed to fork someone else repo and to publish it to Ansible galaxy for example?

# Answer
It is always ethically acceptable to fork any code on Github (or Sourceforge or wherever you got it from) and do whatever you want with it, within the terms allowed by its license! This is what open source and permissive licenses are all about. The original author wanted the world to contribute to this piece of software, and you will be doing exactly that.

Fork it, respect the license requirements, and go on with it. It does not matter whether it is abandoned (like your example, since the last change was 3 years ago) or current.

If you do feel that you did some good improvements, send pull requests back to your upstream, as usual. If the original author never comes back, then so be it. If he does, he can integrate them (or not if he so chooses). But this is strictly optional, you are neither legally nor ethically forced to do so.

About uploading it to some other locations (Ansible Galaxy or whatever - say Docker Hub for docker images and so on): the issue is probably mostly to avoid irritations due to the name being "replaced" (but I'd assume if the original author already has uploaded it to A.G., you can't just replace it, anyway). So, in the latter case, just upload it with another name, which is still easily identifyable as related to the original - say, append some suffix which is not too obnoxious.

If the author, in your case, should later re-appear, and you had uploaded the software to the Ansible Galaxy, and he wants to take over, then you two will simply communicate and resolve the issue somehow (i.e., merge the two forks again and decide who keeps updating Ansible Galaxy). Chances are that he will be more than glad to leave the work to you.

Oh, and to be sure; this piece of software is under the MIT license:

> Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, **copy**, modify, merge, **publish**, **distribute**, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

There you go.
> 13 votes

# Answer
As others have pointed out, this should be ethically OK. However, I would find it polite (and ethically correct) to add a note to a prominent place in your readme file stating that this is a forked repository and provide a link to the original.
> 1 votes

---
Tags: culture
---


# Question
Title: How to tell kubectl to use different api groups
I am trying to list down resources which is in different api group. I am running the below command

```
[jk@ome-esrs-adapter-master csadmin]# kubectl get issuers -n default
Error from server (NotFound): Unable to list "certmanager.k8s.io/v1alpha1, Resource=issuers": the server could not find the requested resource (get issuers.certmanager.k8s.io)

```

As you can see it is not listing any issuer which is in default space because it is searching in this api group "certmanager.k8s.io/v1alpha1" and acutal resource is in "cert-manager.io/v1alpha2" this API group so, I have specify api group in rest call and it is listing the issuer

```
[jk@ome-esrs-adapter-master csadmin]# curl http://localhost:8080/apis/cert-manager.io/v1alpha2/issuers | jq .
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   975  100   975    0     0  47140      0 --:--:-- --:--:-- --:--:-- 48750
{
  "apiVersion": "cert-manager.io/v1alpha2",
  "items": [
    {
      "apiVersion": "cert-manager.io/v1alpha2",
      "kind": "Issuer",
      "metadata": {
        "annotations": {
          "kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"cert-manager.io/v1alpha2\",\"kind\":\"Issuer\",\"metadata\":{\"annotations\":{},\"name\":\"ca-issuer\",\"namespace\":\"default\"},\"spec\":{\"ca\":{\"secretName\":\"nginxsecret\"}}}\n"
        },
        "creationTimestamp": "2020-04-02T13:05:56Z",
        "generation": 1,
        "name": "ca-issuer",
        "namespace": "default",
        "resourceVersion": "67806",
        "selfLink": "/apis/cert-manager.io/v1alpha2/namespaces/default/issuers/ca-issuer",
        "uid": "2970a3e5-6d0c-4f2b-964d-4598be9beaa3"
      },
      "spec": {
        "ca": {
          "secretName": "nginxsecret"
        }
      },
      "status": {
        "conditions": [
          {
            "lastTransitionTime": "2020-04-02T13:05:53Z",
            "message": "Signing CA verified",
            "reason": "KeyPairVerified",
            "status": "True",
            "type": "Ready"
          }
        ]
      }
    }
  ],
  "kind": "IssuerList",
  "metadata": {
    "continue": "",
    "resourceVersion": "284959",
    "selfLink": "/apis/cert-manager.io/v1alpha2/issuers"
  }
}

```

can anyone please help how to tell kubectl to look into diffrent API groups? Because I am sure that resource is there as rest api is working and when I reapply yaml file it says Issuer unchanged.

# Answer
> 1 votes
It works in dotted notation (Object.apiVersion.APIGroup), for e.g. deployments this way: `kubectl get deployments.v1.apps` In your case just try:

```
kubectl get issuer.v1alpha2.cert-manager.io -n=default

```

---
Tags: kubernetes, cert-manager
---


# Question
Title: Terraform and Kubernetes' StorageClass with node affinity
Im able to make a number of kubernetes constructs in terraform and even have done so for my ephemeral stateful data that requires low latency disk performance. I can terraform up a kubernetes\_persistent\_volume but trying to do so with node\_affinity has got me a bit perplexed as it requires a lists of lists (nodeSelectorTerms and matchExpressions), where the inside list has key/value pairs, and just not sure how to terraform all that. Heres the working yaml version:

```
apiVersion: v1
kind: PersistentVolume
metadata: # PVs do not have namespaces
  name: nodeconfig
spec:
  capacity:
    storage: 25Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  storageClassName: nodeconfig
  local:
    path: /mnt/nodeconfig
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - myhost

```

Has anone dealt with this?

# Answer
> 2 votes
Well of course its always the same - as soon as you actually do ask for help you figure it out...

```
resource "kubernetes_persistent_volume" "nodeconfig" {
  metadata {
    name = "nodeconfig"
    labels = {
      a = "a"
    }
  }
  spec {
    capacity = {
      storage = "25Gi"
    }
    storage_class_name               = "nodeconfig"
    persistent_volume_reclaim_policy = "Retain"
    access_modes                     = ["ReadOnlyMany"]
    persistent_volume_source {
      local {
        path = "/mnt/nodeconfig"
      }
    }
    node_affinity {
      required {
        node_selector_term {
          match_expressions {
            key = "kubernetes.io/hostname"
            operator = "In"
            values = [var.config.RGs.Primary.clustervmhost]  # must mnanually put this one in
          }
        }
      }
    }
  }
}

```

So the docs were a bit off of the lists of lists, it was a map of maps of lists.

---
Tags: kubernetes, terraform, kubernetes-volume
---


# Question
Title: Blue Green Deployment DB transactions management
Imagining I have 2 sets of environment(Blue and Green)

```
           +
           |
           |
           |
           v
   +-------+---------+
   |     Proxy/LB    |
   |                 |
   +--+-------------++
      |             |
+-----++          +-+----+
|      |          |      |
|APP v1|          |APP v2|
|Blue  |          |Green |
+--+---+          +--+---+
   |                 |
+--+---+          +--+---+
| DB   |          | DB   |
| Blue | DB Sync  | Green|
|      <---------->      |
+------+          +------+

```

The version Blue is currently running and green is deployed and all DB replication completed. Now I want switch traffic to green environment. In this case, what is the best way to manage the DB transactions?(I have one thought below)

* Forward the new traffic to green and wait for existing blue's environment's transactions complete. Once all are done, switch traffic completely to green(**UPDATE: Can I achieve this with Nginx?**)

If above is true, I'd have to do little extra work to forward only new traffic to green.

Or what is the best way of doing Blue Green deployment in this case?

**PS:**

* We are using Nginx proxy
* All servers are bare metal, no cloud!
* 2 standalone DB setups(Postgresql with master-slave)

# Answer
> 6 votes
As per Martin Flowers' definition of Blue-Green deployment

> The blue-green deployment approach does this by ensuring you have two production environments, as identical as possible. At any time one of them, let's say blue for the example, is live. As you prepare a new release of your software you do your final stage of testing in the green environment. Once the software is working in the green environment, you switch the router so that all incoming requests go to the green environment - the blue one is now idle.

# Steps

You've said that DBs are synced using `master-slave` topology. So, to switch traffic from one instance to another, we have to make steps:

* Switch web-traffic from `APP v1` to `APP v2`
* Shutdown deprecated `DB Blue`
* Promote `DB Green` to `master` mode
* Start new `DB Blue v2` in `slave` mode (you may ask - why shutdown `DB Blue` instead of using `master-master` replication or other ways to sync `DB Blue v1` with `DB Green` in `master` mode. See explanation in `DB replication`)

## Graceful switch web-traffic

To switch web-traffic from `APP v1` `APP v2` \- just use `nginx reload` \- it will graceful end all working connection and move traffic from `App v1` to `App v2` There are many scripting solutions for switch `blue/green` using shell/python scripts

E.g.:

The logic is simple: add `green` upstream in `nginx` config, use it as default and send `reload` to `nginx`:

Sample `nginx.conf` for `reverse proxy`/`load balancer`

```
http {

upstream appv1 {
    zone appv1 64K;
    server 10.10.0.1:80;
}

upstream appv2 {
    zone appv2 64K;
    server 10.10.0.3:80;
}

server {
...
location / {
    proxy_pass http://$appv1;
}
}

}

```

Replace `proxy_pass http://$appv1;` with `proxy_pass http://$appvv;`

And then

```
nginx -s reload

```

## Stop `DB Blue`

`systemctl stop postgresql`

## Promote `DB Green` to `master` mode

* Trigger the promotion using your defined trigger in `$PGDATA/recovery.conf`

```
touch $PGDATA/failover

```
* Remove `failover` trigger from `Green DB`

```
cd $PRIMARY_DATA
rm -f recovery.* failover

```
* Make sure that `hot stanby` mode is `on` on `Green DB` in `master` mode

```
cat postgresql.conf | grep '#hot_standby = on'

```

# DB replication

## Make a backup of master Green DB

```
psql -c "SELECT pg_start_backup('Streaming Replication', true)" postgresql://postgres@GreenDB/postgres

```

## Send a backup to Blue DB

```
rsync -a $PG_DATA_Green_DB/ $BlueDB_IP:$PG_DATA_Blue_DB/ --exclude postmaster.pid --exclude postmaster.opts

```

## See also

# DB Migration (More complex but more robust database migration scheme)

You may use more complex scheme for migration using `Ansible` and \`Pglupgrade tool:

See for details:

# P.s some definitions.

Just to...

> ... sort the buyers from the spyers, the needy from the greedy...

And `blue/green deployment` vs `canary deployment` vs `A/B test`

# Definitions, kinds of deployments and tests

(see details in these answers )

* **Blue-Green Deployment** \- When deploying a new version of an application, a second environment is created. Once the new environment is tested, it takes over from the old version. The old environment can then be turned off.
* **A/B Testing** \- Two versions of an application are running at the same time. A portion of requests go to each. Developers can then compare the versions.
* **Canary Release** \- A new version of a microservice is started along with the old versions. That new version can then take a portion of the requests and the team can test how this new version interacts with the overall system.
* **Feature flagging** \- The action of "configuring" (cold, or even hot) which functionality is (not)available for which (group) of users. If you also do something like "feature flagging" you can deploy first, measure soundness of your release in backwards compatibility/bug perspective, and release new functionality gradually to different users, or vice versa (scale down or even rollback functionality and/or binaries). Feature flagging allows for splitting availability of functionality from deployment of binaries, and gives much more fine-grained decision making then only "deploy/rollback"

# P.P.S `Blue/Green` vs `Canary`

source

* Both blue-green and canary releases solve the same purpose
* Although both of these terms look quite close to each other, they have subtle differences. One put confidence in your functionality release and the other put confidence the way you release.

## Blue-Green Deployment

> When deploying a new version of an application, a second environment is created. Once the new environment is tested, it takes over from the old version. The old environment can then be turned off.

`1. It is more about the predictable release with zero downtime deployment.`

`2. Easy rollbacks in case of failure.`

`3. Completely automated deployment process`

`4. In cloud environment where it is easier to script & recreate infrastructure, blue/green deployment is preferred as it allows the infrastructure to be in sync with the automation`

## Canary Release

> A new version of a microservice is started along with the old versions. That new version can then take a portion of the requests and the team can test how this new version interacts with the overall system.

`1. The canary release is a technique to reduce the risk of introducing a new software version in production by slowly rolling out the change to a small subset of users before rolling it out to the entire infrastructure.`

`2. It is about to get an idea of how new version will perform (integrate with other apps, CPU, memory, disk usage, etc).`

See also illustration (source):

Blue/Green:

Canary:

# Answer
> 1 votes
As promised, here is a solution using haproxy with sticky cookies. Note, that this approach requires support on the application side (as only your application layer may know once transactions are completed).

To achieve this, first - start with haproxy configuration of the following kind:

```
    global
        daemon
        maxconn 256
    defaults
        mode http
        timeout connect 5000ms
        timeout client 50000ms
        timeout server 50000ms

    frontend http-in
        bind *:80
        default_backend servers

    backend servers
        balance roundrobin
        cookie SID insert indirect preserve
        server server1 blue-server-hostname:8000 cookie sblue

```

Note that this configuration is pretty generic and taken from haproxy documentation - from here: https://cbonte.github.io/haproxy-dconv/2.2/configuration.html#2.5

The part that is used to achieve stickiness is cookie instruction that is added in the end, detailed documentation here: https://cbonte.github.io/haproxy-dconv/2.2/configuration.html#4-cookie

So, with this line:

```
cookie SID insert indirect preserve

```

we tell haproxy to insert cookie called **SID** which may only have value **sblue** for now (this value is defined in last line of configuration). And we also say that if this cookie is supplied by the back-end server, we don't want haproxy to overwrite it (achieved by **insert** and **preserve** keywords).

The next part is to add some application level logic to set **SID** cookie to client and make its value configurable on the application side. You want to set **SID** cookie to **sblue** for your old code and to **sgreen** for your new code.

Now, when you do deployment, at first you have only blue instance, all your **SID** cookies are set to **sblue** and haproxy directs all your traffic to the only blue instance.

Next, you add your green instance, and after that you add it to haproxy by adding the following line to the end of configuration above:

```
server server2 green-server-hostname:8000 cookie sgreen

```

Now any request which gets to your new green instance would have its cookie set to sgreen and would stick to the new instance.

Finally, you want to engage your application-level configuration on the old blue instance to set cookie to **sgreen** to clients once their transaction is over (basically, at the point where application knows it's safe to do so). Then any new requests from such clients would be directed to green instance. At some point all your requests would be transitioned to green - and you can detach blue instance at that point.

This kind of approach (or its variations) would be the most graceful and controlled by your application-layer transaction logic.

---
Tags: deployment, nginx, postgresql, bluegreen
---


# Question
Title: Which cloud region could have the fastest time for a docker image push?
**TLDR** How to know what region in the cloud nearest to docker hub?

**Full details** From my localhost my network is super slow to push the docker image onto docker hub.

My workaround is to use a cloud machine to do that `docker image push` hopefully that the internet there would be faster.

So my question is which region on the cloud is nearest to docker hub for me to choose?

My google search results little helpful

p.s. I also asked on stackoverflow here

# Answer
### TLDR

From cloud instance, just go push to docker hub; you will be routed to the best location automatically.

### Full details

DockerHub is a massive scale and very critical global service. So, you almost definitely don't have to think about this at all.

There are techniques on cloud hosts (AWS, Azure, GCP) to replicate & host content/services in regions across the globe and route traffic to the closest entry point automatically. The cloud hosts make this quite easy to do with their managed services (while without them it would be very difficult).

Also, there are many third party providers of technologies like CDNs which do similar things at scale (and they are really good at it). They are **extremely** frequently used in these cases.

So, again - just go to docker hub and don't think about it. You will be routed to the best location automatically.
> 3 votes

---
Tags: docker, cloud
---


# Question
Title: Tool or way to support handling dependencies of modules in various platforms (for machine learning in Python)
I am Data Scientist and I have a problem with setting of versions modules like Keras, Tensorflow PyTorch, etc. Although I am using tools like **virtuaenv** or **conda**, my code isn't cross-platform, version of modules from mac don't fit at windows and I spending a lot of time to the handling of infinity number of problems. **Docker** isn't helping every time, because sometimes I have to send code to my client, which is non-technical(very weak at CS at all).

The main question is: **is the way/tool/software/anything** which supports handling dependencies of modules on **various** platforms?

# Answer
> 1 votes
Python is notoriously painful when it comes to dependency and version management.

I think its your job to make a portable application, and there are usually methods to do so (even if they boil down to ugly branching logic based on the host OS type/etc). Most of the time, you can avoid OS specific things with some effort though.

In terms of deploying code to clients, I would recommend looking at tools that bundle your code into a stand-alone application (pyinstaller, py2app, etc). If you can build a standalone app for your clients for each OS, you won't have to worry about what is on their system (python versions, docker, etc).

---
Tags: docker, python
---


# Question
Title: JAVA home give me wrong value on Jenkins node
I have Jenkins node with below configuration for JavaPath:

```
/usr/java/jdk1.8.0_131/bin/java

```

I wrote a simple Jenkins job which does the following:

```
#!/bin/ksh
echo "JAVA_HOME=${JAVA_HOME}"
echo $PATH

```

for some reason, the output I'm getting is:

```
JAVA_HOME=/usr/java/jdk1.8.0_31

```

why it doesn't use the value which used to load node?

# Answer
> 1 votes
Got response from SO forum: Is there a `JAVA_HOME` configured under `Mange Jenkins -> Global Tool Configuration -> JDK`? You may have to click on `JDK installations` to be able to see what's configured there.

---
Tags: jenkins
---


# Question
Title: How to provide a user to access only a particular bucket in AWS S3?
I have a list of buckets in AWS S3. I have created an IAM user. I have an option to provide S3 full or read only access for a user using groups. Is there any options to provide access only to a particular bucket?

# Answer
Amazon's IAM roles generally grant a role access to a particular ARN (Amazon Resource Name). Amazon notes on their pages that for S3 a resource

> ...can be a bucket-name or a bucket-name/object-key.

They also provide a helpful example for doing just this which appears as follows:

```
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:ListBucket"],
      "Resource": ["arn:aws:s3:::test"]
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject"
      ],
      "Resource": ["arn:aws:s3:::test/*"]
    }
  ]
}

```
> 10 votes

# Answer
Attach below policy to that user:

```
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                        "s3:GetBucketLocation",
                        "s3:ListAllMyBuckets"
                      ],
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::YOUR-BUCKET",
                "arn:aws:s3:::YOUR-BUCKET/*"
            ]
        }
    ]
}

```

https://www.serverkaka.com/2018/05/grant-access-to-only-one-s3-bucket-to-aws-user.html
> 2 votes

# Answer
To provide access for specific bucket, you can define the following policy for that user or group:

```
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation",
                "s3:ListBucketMultipartUploads"
            ],
            "Resource": [
                "arn:aws:s3:::my-bucket"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:AbortMultipartUpload",
                "s3:DeleteObject",
                "s3:DeleteObjectVersion",
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:GetObjectVersion",
                "s3:GetObjectVersionAcl",
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:PutObjectVersionAcl"
            ],
            "Resource": [
                "arn:aws:s3:::my-bucket/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets"
            ],
            "Resource": "arn:aws:s3:::*"
        }
    ]
}

```

Where `my-bucket` is your name of your bucket.

Then send them the Console URL for that bucket, e.g.

* `https://s3.console.aws.amazon.com/s3/buckets/BUCKET_NAME/`

Related:
> 1 votes

# Answer
Since you are looking for programmatic role creation you might consider IAM Roles for EC2. Those can be applied to EC2 instances without human intervention, and they also avoid the need to store many credentials/keys, which is always nice when using automation.
> 0 votes

# Answer
Add the following policy if you want to access from FTP software like **WinSCP**, **CyberDuck**, etc.

```
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetBucketLocation",
        "s3:ListAllMyBuckets"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": ["s3:ListBucket"],
      "Resource": [
            "arn:aws:s3:::test1",
            "arn:aws:s3:::test2"
        ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject"
      ],
      "Resource": [
            "arn:aws:s3:::test1/*",
            "arn:aws:s3:::test2/*"
        ]
    }
  ]
}

```

I learnt from Writing IAM Policies: How to Grant Access to an Amazon S3 Bucket
> 0 votes

---
Tags: amazon-web-services, amazon-s3, aws-iam
---


title: What can be considered as an artifact

text:
# Question
Title: What can be considered as an artifact
Recently, I have had a lecture at my university about advanced testing and CI. To understand all concepts, I was searching what artifact represents, specifically in DevOps sphere. I found many contradictory statements, definitions, and articles.

Therefore, I would like to ask, if there is a **standard** (or at least some unwritten rules) of what everything could be considered as an artifact. What are the most important parts of artifact which should always be included?

Many sources say that artifact should not be source code, but deliverable, and JDK is often used as an example. In that case, I would like to know what is the equivalent when working with scripting languages like JavaScript or Python.

Thank you for all your explanations!

# Answer
Artifact is an abstraction. It could be anything that's produced by somebody else. In Bulgaria for example we're making artifacts for the winter time ( it's called 'compot') . Basically it's an artifact, it was produced by my grandmother and could be used by me :)

Same goes into the IT world, but with one small difference.

The Java artifacts are a little bit more special, since Java is a static language and before the run phase you would have those compiled artifacts - \*.jar files. After that you could reference them during the 'run' phase. This is the reason why they are always pointed as the most straightforward artifact example, the java compiler will produce them.

One major distinguishment between my grandmother's artifacts and Java's ones is that it's a good practice to have a version for each artifact, that way you could easily switch them and see what works and what does not.

JavaScript and Python are dynamic languages, they do not have any compiled artifacts like Java, you could just refer to the '.py' file.

As a summary an artifact could be anything: from tar.gz to a jar file, but it must always be versioned :) Of course the versioning might come during a later stage, like a docker image tag, but the principle still remains.
> 2 votes

---
Tags: continuous-integration, terminology, artifacts
---


# Question
Title: Halt execution of Bamboo build or deploy, without throwing an exception
This seems so trivial, I am almost shy to ask... Depending on certain conditions, which I am able to check by looking at Bamboo variables, I sometimes want to just stop the execution of a build and/or it's deploy. The possibility to conditionally execute steps seems very limited in Bamboo, and I am having trouble solving this. I found a workaround, which is running a script that exits with an error, like a powershell script simple as this

```
exit 1

```

But that obviously causes the build to appear as failed, which really isn't what I want. I just want to prevent a deploy in certain conditions.

Does anyone know an elegant way to accomplish a full stop in a Bamboo plan?

# Answer
> 1 votes
You can manually stop a build in progress. From Stopping an active build:

> To stop an active plan build:
>
> 1. Click Dashboard and then the All Plans tab.
> 2. Click the 'Stop' icon next to the active plan you want to stop.

It also appears possible to stop the build/deploy programmatically, using the REST API, more specifically making a `DELETE` request to the appropriate url:

> /queue/{projectKey}-{buildKey}-{buildNumber : (\[0-9\]+)}?stage&executeAllStages ... DELETE Stop build execution.
>
> available response representations:
>
> 401 Returned when user does not have sufficient rights to view or execute build for specified plan
>
> 204 On success (success is also when build was already completed - so nothing to stop)
>
> 404 Returned when specified job does not exist

# Answer
> 1 votes
I know that is not the clean solution everybody looks for and is a nasty workaround, but what worked for me was to squash all my deployment logic into a single Script Task (shell script) and then put the necessary conditions there, such as:

```
if [ <put condition for terminate here> ] ; then
   exit 0
fi
<Continue with other execution logic>

```

In that way, you can exit with 0 (thus build shows as successful) , but of course that prevents you from reusing all of the out-of-the-box Bamboo Task types that are readily available and questions the benefits of using the bamboo framework as a wrapper.

---
Tags: continuous-integration, bamboo
---


# Question
Title: Cross account access from ec2 to other accounts
Is it possible to run aws cli from ec2 to create some resources in another account without using aws\_secret\_key and aws\_access\_key?

# Answer
Supposing the scenario with two accounts A & B the explanatory steps should be:

1. In **account A**, I created a **role** (e.g `RoleForB`) to trust **account B**, and attach to the before created role a **IAM policy** to allow it to perform some read operations in **account A**. `e.g ReadOnlyAccess`
2. In **account B**, I created a **role** (e.g `AssumeRoleInA`) and attach a **policy** to allow it to assume the role that is created in **account A**.
3. In **account B** Associate to your EC2 instance `ec2-profile` the IAM role (`AssumeRoleInA`) created in step 2.
4. In **account B** login into this EC2 instance to **assume the role** in **Account A** using the command `aws sts assume-role --role-arn "arn:aws:iam::Account_A_ID:role/RoleForB" --role-session-name "EC2FromB"`.
5. In **account B** EC2 terminal when the command is **step 4.** finished, you can see the **access key ID, secret access key, and session token** from wherever you've routed it, in our case `stdout` either manually or by using a script. You can then assign these values to environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`)

So Let’s check the configurations mentioned above step by step but with some mode detail:

1. As before presented in **account A**, it builds the trust to **account B** by creating the role named `RoleForB` and attaching `ReadOnlyAccess` permission to it.

```
{
    "Version": "2012-10-17",
    "Statement": {
        "Effect": "Allow",
        "Principal": {"AWS": "arn:aws:iam::Account_B_ID:root"},
        "Action": "sts:AssumeRole"
    }
}

```
2. In **account B**, create a **role** named `AssumeRoleInA` then attach the corresponding `policy` to allow it to assume the role named `RoleForB` in **account A**.

```
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "sts:AssumeRole",
      "Resource": [
        "arn:aws:iam::Account_A_ID:role/RoleForB"
      ]
    }
  ]
}

```
3. In **account B**, create a new **EC2 instance** (if it does not exists yet), and associate it's **ec2-profile** with the **IAM role** named `AssumeRoleInA`.

```
{
    "Version": "2012-10-17",
    "Statement": {
        "Effect": "Allow",
        "Principal": {"Service": "ec2.amazonaws.com"},
        "Action": "sts:AssumeRole"
    }
}

```
4. In **account B** login into this EC2 instance to **assume the role** in **Account A** using the command:

```
aws sts assume-role --role-arn "arn:aws:iam::Account_A_ID:role/RoleForB" --role-session-name "EC2FromB"`

```

eg:

```
jenkins@bb-jenkins-vault:~$ aws sts assume-role --role-arn arn:aws:iam::521111111144:role/DeployMaster --role-session-name "project-dev-jenkins-deploy"
{
    "AssumedRoleUser": {
        "AssumedRoleId": "AROAJBXGEHOQBXGEHOQ:project-dev-jenkins-deploy",
        "Arn": "arn:aws:sts::521111111144:assumed-role/DeployMaster/project-dev-jenkins-deploy"
    },
    "Credentials": {
        "SecretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
        "SessionToken": "FQoGZXIvYXCUm8iG6/zLdQ7foognvCDpxKP7cRJiZgc...CUm8iG6/zLdQ7foognvCDpxKP7c+OQF",
        "Expiration": "2019-03-29T15:41:02Z",
        "AccessKeyId": "AKIAI44QH8DHBEXAMPLE"
    }
}

```
5. In **account B** EC2 terminal when the command is **step 4.** finished, you can see the **access key ID, secret access key, and session token** from wherever you've routed it, in our case `stdout` either manually or by using a script. You can then assign these values to environment variables

```
$ export AWS_ACCESS_KEY_ID=AKIAI44QH8DHBEXAMPLE
$ export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
$ export AWS_SESSION_TOKEN=FQoGZXIvYXCUm8iG6/zLdQ...<remainder of security token>
$ aws ec2 describe-instances --region us-east-1

```

**credit:** https://stackoverflow.com/a/55420220/6657158
> 3 votes

# Answer
Sure, if the EC2 role allows `STS::AssumeRole` you can assume a role in the other account and get temporary credentials to do whatever you need to do in the other account.

See for example here: **Cross Account Access with AWS-CLI**
> 2 votes

---
Tags: amazon-ec2, aws-cli, aws-iam
---


# Question
Title: How to fix a "heartbeat failure" in Docker Swarm?
My cluster is currently located in a single data center. I've been trying to change that by adding a single worker node from another data center, but so far it hasn't worked.

I'm able to make this node join the swarm and get listed by the managers, but it is always shown as "Down". Here is what "docker inspect" shows me about this node:

```
    "Status": {
        "State": "down",
        "Message": "heartbeat failure",
        "Addr": "xxx.xxx.xxx.xxx"
    }

```

I've opened the following ports in both sides:

```
2377 tcp
7946 tcp+udp
4789 udp

```

How do I troubleshoot and fix this?

# Answer
> 2 votes
This might not be the answer to your specific cross data-center-ip setup.

I occasionally run into one or more swarm-nodes being status: `Down` and Availability: `Active`. Having the `Status.Message`: `"heartbeat failure"`. This can happen after a reboot.

What helped was to stop the docker daemon, remove `/var/lib/docker/swarm/worker/tasks.db` and start the docker daemon again.

from: https://github.com/moby/moby/issues/34827#issuecomment-457678500

sometimes it fixes itself: https://stackoverflow.com/a/54126180/2087704

---
Tags: docker, docker-swarm, networking
---


# Question
Title: Permission denied within mounted volume inside Podman container
I am starting to learn about containers using `podman` that came with RHEL8.1 (which AFAIK can be used in place of `docker`), and have the following baby `Dockerfile` as a learning exercise:

```
# Use Alpine Linux base image
FROM alpine:latest

# Install pacakges
RUN apk --no-cache add bash gcc make

# Make a directory for source code
RUN mkdir /src_dir

# Set working directory to the same directory
WORKDIR /src_dir

# Set this directory as a volume
VOLUME [ "/src_dir" ]

```

As you can see, I've installed the most basic `gcc` and `make` into this container with the goal of mounting a set of source files on my container host into the `/src_dir` directory within the container.

I next build the container image in the host directory containing the `Dockerfile`:

```
podman build -t my_image .

```

I then start the container with this command

```
podman run -it -v /host/foobar:/src_dir /bin/bash

```

Where `/host/foobar/` on my host is an arbitrary directory containing some arbitrary source code, all of which my local user on the host has full read/write access to. For example, there is one file `/host/foobar/test.c`. This then brings me to a `bash` prompt inside the container. I can see that I'm at the correct place because:

```
bash-5.0# pwd
/src_dir

```

However, I have absolutely no read/write access to `/src_dir`. Both `ls -lh` and `cat test.c` gave me permission denied errors. If I change to the root directory (or any other directory) of the container, I can see and access other things. Strangely, if I run `ls -lh /` I can see `/src_dir` as being owned by `root:root`, so I don't understand why as the container's root user I can't access anything in it.

I also tried `podman inspect [container ID]`, and in the output I can see:

```
...
"Mounts": [
            {
                "Type": "bind",
                "Name": "",
                "Source": "/host/foobar",
                "Destination": "/src_dir",
                "Driver": "",
                "Mode": "",
                "Options": [
                    "rbind"
                ],
                "RW": true,
                "Propagation": "rprivate"
            }
        ]
...

```

Which suggests that there *is* read/write permission?

Perhaps I'm missing something obvious as a beginner, but what do I have to do so that I can run the `gcc` and `make` inside this container on the source files mounted in `/src_dir` *so that the container essentially acts as a complete development environment*?

# Answer
> 20 votes
Thanks to the people here, the solution is quite simple (but not obvious):

My GNU/Linux container host has SELinux activated, and that's why I was having permissions problems. The solution is to simply append a `:z` to the `podman run` volume argument so that this:

`podman run -it -v /host/foobar:/src_dir /bin/bash`

becomes this:

`podman run -it -v /host/foobar:/src_dir:z /bin/bash`

That's it.

---
Tags: permissions, volume, persistent-volumes, podman
---


title: How can I monitor docker volume usage?

text:
# Question
Title: How can I monitor docker volume usage?
I'm planning to automate docker activities using python. However, I couldn't find any efficient command for monitoring docker volume. So now I wish to use shell commands for that and later it can be converted using subprocess in python.

My requirement is when I give a volume name then I need the used space, free space, and total space of that volume for finding the storage threshold of the same.

I tried using the `docker system df -v` But it showing all details.

For example, I give the volume name as my\_volume then I need storage details of the same only. Hope everyone understood my requirement and excepting a helping hand from someone.

# Answer
I'd recommend a temporary container for this. It will work on just about any version of docker, and supports named volumes with non-default settings, e.g. that store data on remote systems (e.g. NFS) or modify the bind mound settings to use another folder.

A simple example is:

```
$ docker container run --rm -v "${volume_name}:/volume" -w /volume busybox df -P .
Filesystem           1024-blocks    Used Available Capacity Mounted on
/dev/..........              898086280 350169736 502226484  41% /volume

```

You can parse that output or reformat with the `df` command as appropriate for your use case. The busybox version of `df` will have fewer options than you may be used to:

```
$ df --help
BusyBox v1.31.1 (2019-12-23 19:20:27 UTC) multi-call binary.

Usage: df [-PkmhTai] [-B SIZE] [FILESYSTEM]...

Print filesystem usage statistics

        -P      POSIX output format
        -k      1024-byte blocks (default)
        -m      1M-byte blocks
        -h      Human readable (e.g. 1K 243M 2G)
        -T      Print filesystem type
        -a      Show all filesystems
        -i      Inodes
        -B SIZE Blocksize

```
> 2 votes

---
Tags: docker, shell-script
---


# Question
Title: how to run multiple environments in single job in Jenkins
I have two environments - front-end and back-end. My front-end deployment has one job and my back-end deployment has another job. My team wants to deploy both front-end and back-end together. Can combine them into single job?

Ideally, they can choose whatever environment and respective branches they required to deploy for their testings in this one job.

# Answer
A pretty standard way of doing this would be to chain your jobs or your pipelines. This lets you keep separate jobs (which can be useful), but lets you trigger one job from the other and visualize the end-to-end flow.

So, you still just tell one job to run, but it runs both properly. Their config and logs are still well separated and you get all the other isolation related benefits.

Example: https://www.jrebel.com/blog/using-jenkins-build-pipelines

You can also do this in normal Jenkins jobs not using a pipeline; there is an option to trigger other jobs near the bottom usually.
> 1 votes

# Answer
Yes. Use multi-job project to combine FE phase and BE phase together. Or if you used pipeline, just add those pipelines under "build" step.
> 0 votes

---
Tags: jenkins
---


# Question
Title: Is there a policy that can limit the growth of an AWS S3 buckets total size?
We have an AWS S3 bucket that accepts uploads from untrusted sources which are then processed and moved out of the bucket.

Is there a Policy that can limit the growth of the bucket to some predetermined upper bound to protect us in case something goes wrong?

# Answer
> 1 votes
Not as far as I know.

You can create **Billing alerts** to get notifications when AWS thinks that your monthly spend may be over a given threshold.

# Answer
> 1 votes
There's not such a policy as far as I know.

That being said, you could trigger a lambda and then make this lambda check the properties of your bucket and then you would perform any actions you need.

---
Tags: amazon-s3, aws-cli
---


# Question
Title: How to make different permissions for different subdirectories of storage bucket
On Google Cloud Storage or Amazon S3, is there a way to give users a key to only a specific subdirectory of a storage bucket, for example:

```
gs://images.eng.cm.com/avatars/<user-uuid>/xyz

```

so the user has a user id of and they can read and write only to that subdirectory of the storage bucket, and only if they have a read/write access key in hand too?

Anyone know how to create this fine-grained access control? is it possible/advised?

# Answer
> 2 votes
Before giving the answer I'd like to clarify that S3 is meant for *object* storage, meaning that there's no file system involved and thus there are not "directories" per se.

So, an "object" with the name "/avatars/user-123/xyz" might give you the impression that it's stored in the "/avatars/user-123" directory, but there's no directory involved, that's just a name.

That being said, you can treat that portion of the name (/avatars/user-123) as a prefix, and then you can create IAM policies for your S3 buckets using said prefixes.

For futre reference, the following examples have been taken from: https://aws.amazon.com/premiumsupport/knowledge-center/s3-folder-user-access/

In case readers find out that link has been taken down I'll just copy and paste the examples here

AllowStatement1 allows the user to list the buckets that belong to their AWS account. The user needs this permission to be able to navigate to the bucket using the console.

AllowStatement2A allows the user to list the folders within awsexamplebucket, which the user needs to be able to navigate to the folder using the console. The statement also allows the user to search on the prefix media/ using the console.

AllowStatement3 allows the user to list the contents within awsexamplebucket/media.

AllowStatement4A allows the user to download objects (s3:GetObject) from the folder awsexamplebucket/media.

```
{
 "Version":"2012-10-17",
 "Statement": [
   {
     "Sid": "AllowStatement1",
     "Action": ["s3:ListAllMyBuckets", "s3:GetBucketLocation"],
     "Effect": "Allow",
     "Resource": ["arn:aws:s3:::*"]
   },
  {
     "Sid": "AllowStatement2A",
     "Action": ["s3:ListBucket"],
     "Effect": "Allow",
     "Resource": ["arn:aws:s3:::awsexamplebucket"],
     "Condition":{"StringEquals":{"s3:prefix":["","media"]}}
    },
  {
     "Sid": "AllowStatement3",
     "Action": ["s3:ListBucket"],
     "Effect": "Allow",
     "Resource": ["arn:aws:s3:::awsexamplebucket"],
     "Condition":{"StringLike":{"s3:prefix":["media/*"]}}
    },
   {
     "Sid": "AllowStatement4A",
     "Effect": "Allow",
     "Action": ["s3:GetObject"],
     "Resource": ["arn:aws:s3:::awsexamplebucket/media/*"]
   }
 ]
}

```

---
Tags: amazon-s3, images, google-cloud-storage
---


# Question
Title: Running two containers in a kubernetes job each with their own initContainers
In this SO article I understand that I can run a sequence of `initContainers` before running the containers defined in `spec.template.spec.containers`.

https://stackoverflow.com/questions/40713573/how-to-run-containers-sequentially-as-a-kubernetes-job

I have two independent containers, each of which requires a sequence of `initContainers` to run in series prior to the container running.

Is it possible to run both sequences of `initContainers` at the same time? Something like this:

```
apiVersion: batch/v1
kind: Job
metadata:
  name: my-test-job
spec:
  backoffLimit: 0
  template:
    spec:
      initContainers:
      - name: init1
        image: busybox
        command: ["echo", "Init 1 Ran!"]
      containers:
      - name: task1
        image: busybox
        imagePullPolicy: Always
        command: ["echo", "Task 1 Ran!"]
      restartPolicy: Never

    spec:
      initContainers:
      - name: init2
        image: busybox
        command: ["echo", "Init 2 Ran!"]
      containers:
      - name: task2
        image: busybox
        imagePullPolicy: Always
        command: ["echo", "Task 2 Ran!"]
      restartPolicy: Never

```

It's not valid to duplicate `spec` as I did in that example of what I'm attempting to achieve. But it demonstrates the sequence of events I want. Initially `init1` and `init2` kick off, and once each finishes, `task1` and `task2` start following their respective init containers.

# Answer
> 2 votes
What you are describing is best achieved by either using two jobs or let task 1 also be an init Container as they run in sequence. Alternatively you could use Argo Workflows. https://github.com/argoproj/argo

---
Tags: kubernetes, yaml
---


title: Run more pods than node's CPU with k8s

text:
# Question
Title: Run more pods than node's CPU with k8s
Let's suppose i have 5 k8s nodes with 2 cpu on each node.

This mean i have 10 cpus.

Now let's suppose i want to deploy a very small and basic application with 20 replicas.

Is there a limitation on k8s that will allow only one pod for one cpu and reject 10/20 replicas ?

Thanks

# Answer
> 1 votes
In addition to the good answer by @therianthropie, here are useful notes to help you understand the situation better.:

* You can set requests and limits very arbitrarily (and yes, as fractions of a CPU and very small amounts of memory).
* Kubernetes will *not* do its job well if you don't get real values for these.
* If you under-shoot your request, kubernetes will schedule a lot of apps without requesting enough resources and you'll have problems with the CPU/memory/disk, and so on.
* If you over-shoot your request a lot, your kubernetes nodes may be doing virtually nothing.
* Some apps are very CPU bound, some memory bound, some use virtually no resources. So, you really can't guess.
* Prometheus is a popular tool that you can install into kubernetes with basically a single command.
* You can chart things in grafana.
* So, then you choose your arbitrary values, then watch the charts as you use the app, and then you can adjust them to something sane / useful so kubernetes can do its job well. You can also set alerts with something called alert manager if your values are too ridiculous (it's a good idea once you're ready).
* Note - requests are what kubernetes allocates for your pod. Limits are the max resources your pod can spike to when it needs them. There is a general assumption that not all pods will spike in usage at once / or over a long term or else this idea would not work.

# Answer
> 0 votes
No, you can set requests and limits with fractions of vCPUs. For example, every pod has a request/limit of 100m (1/10 of a CPU). This means 20 pods would need 2 CPUs in total.

---
Tags: kubernetes
---


# Question
Title: Ansible: When 'directory/file exists' in a loop
I have written some Ansible code to put users from the users.yaml file in the variable 'nexus\_local\_users' to be used by a role. When doing this a random password is created and written to the 'credentials/{{ user }}/password.txt' file.

I'm looking for a condition that can make sure that once the user is created it won't be added to the variable again, to make sure the user isn't added twice. My first thought to achieve this was to check if a password has been generated already.

Is there any way to make sure a user is only added to this variable once and not on consecutive runs?

Here is a snapshot of the code I am using:

```
    - name: Include users.yaml and setup all vars for nexus
      include_vars:
        file: users.yaml
        name: users
    - set_fact:
       tmp_user:
        - username: "{{ item.username }}"
          first_name: "{{ item.first_name }}"
          last_name: "{{ item.last_name }}"
          email: "{{ item.email }}"
          password: "{{ lookup('password', 'credentials/' + item.username + '/password.txt length=15 chars=ascii_letters') }}"
          roles: "{{ item.roles }}"
      loop: "{{ users.nexus_local_users }}"
      register: tmp_users
    - set_fact:
        tmp_user: "{{ item.item | combine(item.ansible_facts.tmp_user) }}"
      with_items: "{{ tmp_users.results }}"
      register: tmp_users
    - set_fact:
        nexus_local_users: "{{ tmp_users.results | map(attribute='ansible_facts.tmp_user') | list }}"

```

I'm looking for something like this but can't figure out how to implement it (the when condition):

```
    - set_fact:
       tmp_user:
        - username: "{{ item.username }}"
          first_name: "{{ item.first_name }}"
          last_name: "{{ item.last_name }}"
          email: "{{ item.email }}"
          password: "{{ lookup('password', 'credentials/' + item.username + '/password.txt length=15 chars=ascii_letters') }}"
          roles: "{{ item.roles }}"
      loop: "{{ users.nexus_local_users }}"
      when: **directory/file credentials/{{ user }}/password.txt exists**
      register: tmp_users

```

Thanks in advance!

Gijs

# Answer
I solved this with the Jinja test "exists" and the select filter to filter away the resulting 'AnsibleUndefined' values. The "is exists" test checks if the path exists on the ansible control host. This is the place the credentials are stored.

The solved code is below:

```
    ---
    - hosts: localhost
      tasks:
      - name: Include users.yaml and setup all vars for nexus
        include_vars:
          file: users.yaml
          name: users
      - set_fact:
          tmp_user:
          - username: "{{ item.username }}"
            first_name: "{{ item.first_name }}"
            last_name: "{{ item.last_name }}"
            email: "{{ item.email }}"
            password: "{{ lookup('password', 'credentials/' + item.username + '/password.txt length=15 chars=ascii_letters') }}"
            roles: "{{ item.roles }}"
        loop: "{{ users.nexus_local_users }}"
        register: tmp_users
        when: "not '{{ playbook_dir }}/credentials/{{ item.username }}/password.txt' is exists"
      - set_fact:
          tmp_user: "{{ item.ansible_facts.tmp_user | combine(item.item) }}"
        with_items: "{{ tmp_users.results }}"
        register: tmp_users_sorted
        when: "not item is skipped"
      - set_fact:
          nexus_local_users: "{{ tmp_users_sorted.results | map(attribute='item.ansible_facts.tmp_user') | list | select('defined') | list }}"
      - debug:
          msg: "{{ nexus_local_users }}"

```

The following was added/changed:

```
        when: "not '{{ playbook_dir }}/credentials/{{ item.username }}/password.txt' is exists"

```

```
        nexus_local_users: "{{ tmp_users_sorted.results | map(attribute='item.ansible_facts.tmp_user') | list | select('defined') | list }}"

```

```
       when: "not item is skipped"

```

This makes for a compact solution without adding more tasks/files.

For more information see: https://docs.ansible.com/ansible/latest/user\_guide/playbooks\_tests.html
> 2 votes

# Answer
You should first stat for the password file, and then set the fact depending on whether the file.stat.exists or not. And both those tasks need to be performed for each user.

As of Ansible 2.4 you can dynamically include\_tasks and perform a loop on it. Meaning that you can group a few tasks in a file and loop those tasks for every value in your loop-variable.

I think this should be able to solve your problem by creating a separate tasks file containing these tasks that should be run for each user:

Stat for the password-file:

```
   - name: Check if password-file exists for user
     stat:
        path: "credentials/{{ item.username }}/password.txt"
     register: password_file

```

Next you can define your tmp\_user var when the password.txt-file exists:

```
   - name: Set fact for user
     set_fact:
       tmp_user:
        - username: "{{ item.username }}"
          first_name: "{{ item.first_name }}"
          last_name: "{{ item.last_name }}"
          email: "{{ item.email }}"
          password: "{{ lookup('password', 'credentials/' + item.username + '/password.txt length=15 chars=ascii_letters') }}"
          roles: "{{ item.roles }}"
      when: password_file.stat.exists
      register: set_tmp_user

```

Then, if previous task defined a new fact tmp\_user, that fact should be added to the tmp\_users list.

```
   - name: Add user to list
     set_fact:
       tmp_users: {{ tmp_users + [ tmp_user ] }}
     when: not set_tmp_user is skipped

```

(not sure by head if you can use `when: set_tmp_user is changed` but `not is skipped` should do the job)

Then in you main tasks-file, you create an empty tmp\_users list and call an include\_tasks in a loop to loop through the above tasks:

```
   - name: Include users.yaml and setup all vars for nexus
     include_vars:
       file: users.yaml
       name: users

   - name: Initialize tmp_users
     set_fact:
        tmp_users: []

   - name: Set user facts
     include_tasks: set_user_facts.yml
     loop: "{{ users.nexus_local_users }}"

```

Disclaimer: I have not been able to test this out, or check its syntax, but I assume the general idea of which I try to accomplish here should be clear?

Also be aware that you currently can't perform a loop inside the included tasks file as that loop will by default overwrite the `item`-variable defined by the tasks-loop. See defining inner and outer variable names with loop var to solve this.

And an additional tip: add a debug task as first task to your included tasks file to output the current `item.username` in the loop.
> 2 votes

---
Tags: ansible, automation, passwords
---


# Question
Title: RBAC to secure Azure Table Storage access
I want to limit my user assigned MSI to be able to read and write tables only in a storage account. When assigning roles on IAM blade in the portal I don't see any table related roles. Why is that? I don't wanna use any SAS.

# Answer
For now there is no support for accessing Azure Table Storage with Managed Identities.

Supported services are described here:

According to documentation, Azure Table Storage allows you to authenticate only by using account key. So SAS authentication is not solution here either.

Possible solution could be like this:

1. Create Azure Key Vault and store account key as a secret Assign
2. permission to this key vault to your Managed Identity Use Managed
3. Identity to fetch the key from key vault in a runtime
4. Use the key to Azure Table Storage

As additional step you can create separate Storage Account to store only tables. With this approach you should be able to restrict access to tables service properly.
> 2 votes

---
Tags: azure, role-based-access-control
---


# Question
Title: How to fetch resource group name( with pattern matching) using Azurecli
I set my az account to current subscription. And I am trying to fetch the resource name ending with (example "\***kafka" ). How can i do it.

I tried with `az group list -n '***kafka'`

> validation error: Parameter 'resource\_group\_name' must conform to the following pattern: '^\[-\w\\.\_\\(\\)\]+$'.

# Answer
There are at least two ways to achieve this.

First is to go with transforming json output to proper object for PowerShell as described in this thread:

Second it to use `--query` global parameter for Azure CLI:

```
az group list --query "[?contains(name,'kafka')].name"

```

But remember to use at least PowerShell 6.2 as previous versions have an issue:
> 2 votes

---
Tags: azure
---


# Question
Title: Azure DevOps service connection: lifetime of service principal
I have a hard time understanding the lifetime of the service principal created in Azure AD when creating the service connection between Azure DevOps and Azure. I do it like this:

I pick the Azure Resource Manager

And then i choose the recommended

Hereafter I fill in the blanks

I now have a valid service connection and I can deploy from DevOps to Azure. The only thing i need is a user with enough rights to do this. I can even see the service principal in Azure where it tells me that the client secret will expire in two years:

What i dont understand:

1. does the service connection then stop working when the secret expires? Or will DevOps somehow extend the expire date?
2. i cant read this anywhere, so it is up for my own interpretation on how i understand the "service principal" concept, but if the user used to create the principal gets disabled or deleted, then the service principal *will* persist, correct?
3. if 2) is correct, then when will the principal get deleted? When the scope of the principal gets deleted? Or when the service principal itself gets deleted?

# Answer
I will try to elaborate a lite about those topics.

1. When secret expires and you got error, you have to refresh connection. Go to Edit dialog for connection and save without any changes. Described here: Azure devops service connection expired and cannot edit/renew

But I have already tested scenario when I delete all secrets assigned to service principal used for the Azure DevOps connection. And it is still working. So I can assume, that it depends on how you create connection (automatic or manual service principal). When created with automatic path, it doesn't depend on secrets.

On the other hand, when I add connection with manually providing service principal, then delete all secrets, the connection stops working.
2. Yes, the service principal will exist. But best practice is to set up more then one Owner of the service principal. If the only one service principal owner is deleted, you need to have higher permission to manage all service principals.
3. Service principal will not expire itself, only secrets generated for it. To delete service principal object, you have to do it manually.
> 4 votes

# Answer
1. The service principal you created actually exists in your AAD. When it expires, then AAD will stop authorizing any token requests that AAD gets for that SP. So yes, then Azure Devops will start failing auth when it tries to do work.
2. The SP exists 100% independently of the user that created it so it will persist if the user that created it is deleted.
3. I believe when it expires, it will sit in AAD in expired state, you can then manually delete it.

SP's exist in AAD under the "application registrations" blade. Another thing to keep note of is that the default creation behavior of a SP is for it to have full RW access to the entire subscription it was created for. You can disable this at creation with an -- option flag.
> 1 votes

---
Tags: azure-devops, azure, active-directory
---


title: Ansible seems to execute infinite loop

text:
# Question
Title: Ansible seems to execute infinite loop
I wrote some roles for my home lab to setup a GlusterFS cluster.

But when I run it on my 2 nodes, some tasks get executed over and over again and it seems that it never ends.

Here is the log output (I cut it off at some point and also removed a lot of the * characters)

```
 ✘ user@pc  ~/ansible   feature/role-gluster  ansible-playbook -i inventories/playground/hosts.yaml site.yml -K
BECOME password:

PLAY [storageservers] ******************

TASK [Gathering Facts] *****************
ok: [node2]
ok: [node1]

TASK [include_role : gluster-node] *****

TASK [include_role : gluster-node-partition]

TASK [include_role : gluster-node-partition]

TASK [gluster-node-partition : Create data1 partition]
changed: [node2]
changed: [node1]

TASK [gluster-node-partition : Create filesystem on data1 partition]
changed: [node2]
changed: [node1]

TASK [gluster-node-partition : Create mount point for data1 partition]
changed: [node1]
changed: [node2]

TASK [gluster-node-partition : Mount data1 partition] *
changed: [node2]
changed: [node1]

TASK [Create data1 partition] **********

TASK [gluster-node-partition : Create data1 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create filesystem on data1 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create mount point for data1 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Mount data1 partition] *
ok: [node1]
ok: [node2]

TASK [Create data2 partition] **********

TASK [gluster-node-partition : Create data2 partition]
changed: [node1]
changed: [node2]

TASK [gluster-node-partition : Create filesystem on data2 partition]
changed: [node2]
changed: [node1]

TASK [gluster-node-partition : Create mount point for data2 partition]
changed: [node1]
changed: [node2]

TASK [gluster-node-partition : Mount data2 partition] *
changed: [node1]
changed: [node2]

TASK [Create data3 partition] **********

TASK [gluster-node-partition : Create data3 partition]
changed: [node2]
changed: [node1]

TASK [gluster-node-partition : Create filesystem on data3 partition]
changed: [node1]
changed: [node2]

TASK [gluster-node-partition : Create mount point for data3 partition]
changed: [node1]
changed: [node2]

TASK [gluster-node-partition : Mount data3 partition] *
changed: [node1]
changed: [node2]

TASK [gluster-node : Update apt-get repo and cache] ***
ok: [node2]
ok: [node1]

TASK [gluster-node : Upgrade all apt packages] ********
changed: [node2]

changed: [node1]

TASK [gluster-node : Install GlusterFS]
changed: [node2]
changed: [node1]

TASK [gluster-node : Enable GlusterFS on Boot] ********
changed: [node2]
changed: [node1]

TASK [gluster-node : Probe GlusterFS Nodes] *
changed: [node1] => (item=node1)
changed: [node2] => (item=node1)
changed: [node1] => (item=node2)
changed: [node2] => (item=node2)

TASK [gluster-node : Ensure Gluster brick and mount directories exist.]
changed: [node1] => (item=/data1)
changed: [node2] => (item=/data1)
changed: [node1] => (item=/gluster_volume1)
changed: [node2] => (item=/gluster_volume1)

TASK [gluster-node : Configure Gluster volume.] *******
changed: [node1]

TASK [gluster-node : Ensure Gluster volume is mounted.] *************
changed: [node1]
changed: [node2]

TASK [include_role : gluster-node-partition]

TASK [include_role : gluster-node-partition]

TASK [gluster-node-partition : Create data1 partition]
ok: [node2]
ok: [node1]

TASK [gluster-node-partition : Create filesystem on data1 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create mount point for data1 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Mount data1 partition] *
ok: [node1]
ok: [node2]

TASK [Create data1 partition] **********

TASK [gluster-node-partition : Create data1 partition]
ok: [node2]
ok: [node1]

TASK [gluster-node-partition : Create filesystem on data1 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create mount point for data1 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Mount data1 partition] *
ok: [node1]
ok: [node2]

TASK [Create data2 partition] **********

TASK [gluster-node-partition : Create data2 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create filesystem on data2 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create mount point for data2 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Mount data2 partition] *
ok: [node1]
ok: [node2]

TASK [Create data3 partition] **********

TASK [gluster-node-partition : Create data3 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create filesystem on data3 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create mount point for data3 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Mount data3 partition] *
ok: [node1]
ok: [node2]

TASK [Create default partitioning] *****

TASK [Create gluster volume for partition 1]

TASK [include_role : gluster-node] *****

TASK [include_role : gluster-node-partition]

TASK [include_role : gluster-node-partition]

TASK [gluster-node-partition : Create data1 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create filesystem on data1 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create mount point for data1 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Mount data1 partition] *
ok: [node1]
ok: [node2]

TASK [Create data1 partition] **********

TASK [gluster-node-partition : Create data1 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create filesystem on data1 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create mount point for data1 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Mount data1 partition] *
ok: [node1]
ok: [node2]

TASK [Create data2 partition] **********

TASK [gluster-node-partition : Create data2 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create filesystem on data2 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create mount point for data2 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Mount data2 partition] *
ok: [node1]
ok: [node2]

TASK [Create data3 partition] **********

TASK [gluster-node-partition : Create data3 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create filesystem on data3 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create mount point for data3 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Mount data3 partition] *
ok: [node1]
ok: [node2]

TASK [gluster-node : Update apt-get repo and cache] ***
ok: [node1]
ok: [node2]

TASK [gluster-node : Upgrade all apt packages] ********
ok: [node1]
ok: [node2]

TASK [gluster-node : Install GlusterFS]
ok: [node1]
ok: [node2]

TASK [gluster-node : Enable GlusterFS on Boot] ********
ok: [node1]
ok: [node2]

TASK [gluster-node : Probe GlusterFS Nodes] *
changed: [node1] => (item=node1)
changed: [node2] => (item=node1)
changed: [node2] => (item=node2)
changed: [node1] => (item=node2)

TASK [gluster-node : Ensure Gluster brick and mount directories exist.]
ok: [node1] => (item=/data1)
ok: [node2] => (item=/data1)
ok: [node1] => (item=/gluster_volume1)
ok: [node2] => (item=/gluster_volume1)

TASK [gluster-node : Configure Gluster volume.] *******
ok: [node1]

TASK [gluster-node : Ensure Gluster volume is mounted.] *************
ok: [node1]
ok: [node2]

TASK [include_role : gluster-node-partition]

TASK [include_role : gluster-node-partition]

TASK [gluster-node-partition : Create data1 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create filesystem on data1 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create mount point for data1 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Mount data1 partition] *
ok: [node1]
ok: [node2]

TASK [Create data1 partition] **********

TASK [gluster-node-partition : Create data1 partition]
ok: [node2]
ok: [node1]

TASK [gluster-node-partition : Create filesystem on data1 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create mount point for data1 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Mount data1 partition] *
ok: [node1]
ok: [node2]

TASK [Create data2 partition] **********

TASK [gluster-node-partition : Create data2 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create filesystem on data2 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create mount point for data2 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Mount data2 partition] *
ok: [node1]
ok: [node2]

TASK [Create data3 partition] **********

TASK [gluster-node-partition : Create data3 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create filesystem on data3 partition]
ok: [node2]
ok: [node1]

TASK [gluster-node-partition : Create mount point for data3 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Mount data3 partition] *
ok: [node1]
ok: [node2]

TASK [Create default partitioning] *****

TASK [Create gluster volume for partition 1]

TASK [include_role : gluster-node] *****

TASK [include_role : gluster-node-partition]

TASK [include_role : gluster-node-partition]

TASK [gluster-node-partition : Create data1 partition]
ok: [node2]
ok: [node1]

TASK [gluster-node-partition : Create filesystem on data1 partition]
ok: [node2]
ok: [node1]

TASK [gluster-node-partition : Create mount point for data1 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Mount data1 partition] *
ok: [node2]
ok: [node1]

TASK [Create data1 partition] **********

TASK [gluster-node-partition : Create data1 partition]
ok: [node2]
ok: [node1]

TASK [gluster-node-partition : Create filesystem on data1 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create mount point for data1 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Mount data1 partition] *
ok: [node1]
ok: [node2]

TASK [Create data2 partition] **********

TASK [gluster-node-partition : Create data2 partition]
ok: [node2]
ok: [node1]

TASK [gluster-node-partition : Create filesystem on data2 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create mount point for data2 partition]
ok: [node2]
ok: [node1]

TASK [gluster-node-partition : Mount data2 partition] *
ok: [node1]
ok: [node2]

TASK [Create data3 partition] **********

TASK [gluster-node-partition : Create data3 partition]
ok: [node2]
ok: [node1]

TASK [gluster-node-partition : Create filesystem on data3 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Create mount point for data3 partition]
ok: [node1]
ok: [node2]

TASK [gluster-node-partition : Mount data3 partition] *
ok: [node1]
ok: [node2]

TASK [gluster-node : Update apt-get repo and cache] ***
ok: [node1]
ok: [node2]

TASK [gluster-node : Upgrade all apt packages] ********
ok: [node1]
ok: [node2]

TASK [gluster-node : Install GlusterFS]
ok: [node1]
ok: [node2]

TASK [gluster-node : Enable GlusterFS on Boot]
ok: [node1]
ok: [node2]

TASK [gluster-node : Probe GlusterFS Nodes]
changed: [node1] => (item=node1)
changed: [node2] => (item=node1)
changed: [node1] => (item=node2)
changed: [node2] => (item=node2)

TASK [gluster-node : Ensure Gluster brick and mount directories exist.]
ok: [node1] => (item=/data1)
ok: [node2] => (item=/data1)
ok: [node1] => (item=/gluster_volume1)
ok: [node2] => (item=/gluster_volume1)

TASK [gluster-node : Configure Gluster volume.]
ok: [node1]

^C [ERROR]: User interrupted execution

```

I placed all the code here: https://github.com/CodingSpiderFox/ansible-loop-problem

# Answer
> 1 votes
I commited the fix to the Github repo. The problem was that a role was included recursively.

I split up the roles a bit to fix it. Also discovered a lot of other issues with repeated task executions (not all of the fixed yet because I'm still learning how to structure my ansible tasks)

---
Tags: ansible, yaml, glusterfs
---


# Question
Title: creating private registry, id and password with -Bbn option
Any one knows what does Bbn stands for in the below command

I am aware that,If the bbn option is not provided its not possible to write in the htpasswd file,but what is the meaning of it

```
docker run --entrypoint htpasswd registry:2 -Bbn testuser password > auth/htpasswd

```

# Answer
From docs:

> `-B` Use bcrypt encryption for passwords. This is currently considered to be very secure.
>
> `-b` Use batch mode; i.e., get the password from the command line rather than prompting for it. This option should be used with extreme care, since the password is clearly visible on the command line. For script use see the -i option. Available in 2.4.4 and later.
>
> `-n` Display the results on standard output rather than updating a file. This is useful for generating password records acceptable to Apache for inclusion in non-text data stores. This option changes the syntax of the command line, since the passwdfile argument (usually the first one) is omitted. It cannot be combined with the -c option.
> 1 votes

---
Tags: docker, docker-registry
---


# Question
Title: Mapping Outbound IPs to 1 IP in Azure App Services
I have 3 components of my architecture running in 3 separate Azure App Services. Each app service has 4 outbound IPs. So whenever I have to deploy my app onto the client's premise, I have to whitelist 12-16 IPs on their on-premise Firewall.

Is there any way I can deploy a NAT which uses only 1 public IP and I can provide just 1 IP to the client?

If I manage to make this architecture work then how would I modify my API calls to these services, since at the moment I'm using their 1 public IP to access their individual service, with just 1 IP how will I distinguish API calls made to different app service.

# Answer
> 1 votes
Azure Application Gateway allows you to provide one public IP for those App Services.

You have to make the following steps:

1. Provision Application Gateway
2. Add listeners with host name for eachof the App Services
3. Add backends for each of the App Service
4. Create HTTP Settings with *Override with new host name* and *Pick host name from backend target* options
5. Add rules to connect listeners corresponding to backends

At the end customer has to white-list one public IP assigned to Application Gateway and create DNS entries to point all App Services addresses to one public IP.

Materials:

---
Tags: azure, networking
---


# Question
Title: Replace/Consume Istio's Prometheus/Grafana by existing Prometheus-operator
I have multiple K8S clusters that have Prometheus-operator Helm chart installed in monitoring namespace. Each operator consists of Prometheus/Alertmanager/Grafana. Aside from it I have Istio Helm chart installed in istio-namespace. I want to integrate the Prometheus/Grafana from Istio into my Prometheus-Operator setup and remove them from istio-namespace and the helm release for Istio. Based on this PR - https://github.com/istio/installer/pull/71 it looks like it's possible, but I can't find exact guidance on how to do it, the end result should be prom-operator prometheus scrapping exactly what it was + what was being scrapped by Istio's Prometheus and then when using the Grafana from prom-operator I should have all mixins + istio dashboards.

# Answer
> 0 votes
Followed this guide https://medium.com/@emirmujic/collect-cluster-and-istio-metrics-with-prometheus-operator-42d4499d621a and it worked out for me on Istio 1.2.2: https://medium.com/@emirmujic/collect-cluster-and-istio-metrics-with-prometheus-operator-42d4499d621a

---
Tags: kubernetes, prometheus, istio, grafana
---


# Question
Title: GUI over GitLab CI Environment Deployments With Multiple Instances?
Let's say I have a SaaS style project (similar to Databricks). We may make release 2.0.0 of this project, have automated testing and deployment to dev, automated promotion to nonprod, etc.

In production, I may have 200 **instances** of it running though.

What is the best way using GitLab CI to manage 200 instances/production environments? Obviously you wouldn't want 200 branches with deployments controlled by merges/etc... I assume.

How can you get a good overview over all of these environments and ensure they're deployed completely/view their versions and roll-out progress, ensure smoke tests completed for a successful pipeline/etc?

Its been implied to me that Spinnaker may be a good tool to plug this gap. Can someone explain how that is if so? Are there any better options?

# Answer
Check out the GitLab Operations Dashboard feature (https://docs.gitlab.com/ee/user/operations\_dashboard/)

There is a screenshot on that page, too, so you can get an idea of it.
> 1 votes

# Answer
Regarding modern CD tools, such as Spinnaker, Argo, Flux, Jenkins X (depending on your stack and use case - most are k8s specific) - they will help you with roll-out of any specific code base (set of artifacts) to a large number of nodes / instances. However, they wouldn't help you with versioning or keeping track of versioning. Essentially, they would help you to deploy whichever version you tell them to deploy to a large scale.

I'm currently working on a project - https://relizahub.com \- which integrates with any CI/CD solution to solve the problem of versioning there. Here's my latest demo video on approvals across different environments - https://www.youtube.com/watch?v=PzdZjMby6Is. It may help with a versioning issue and seeing what version you have at which instance and when it was deployed.
> 0 votes

---
Tags: deployment, gitlab, release, environments, spinnaker
---