# Question
Title: Pushing code changes from Pipeline back to repo/branch, using Git!
Here is the scenario: We are trying to "automate" our Salesforce development with Azure Pipelines. Since Salesforce is very unique when it comes to deploying changes, I have tried to create a Pipeline that would be manually run to save any change made into a Salesforce Sandbox back to a Branch. The goal is to eventually merge these changes into other branches and deploy them into other Salesforce environments.

We are using the the SFDX cli for such tasks within the image. Below is a copy of my yml file:

```
trigger:
- developer

pool:
  vmImage: 'ubuntu-latest'

variables:
- group: DeveloperVariables

steps:
- checkout: self
  persistCredentials: true

- task: DownloadSecureFile@1
  inputs:
    secureFile: server.key
- task: Bash@3
  inputs:
    workingDirectory: ''
    targetType: inline
    script: |
      wget https://developer.salesforce.com/media/salesforce-cli/sfdx-linux-amd64.tar.xz;
      mkdir sfdx;
      tar xJf sfdx-linux-amd64.tar.xz -C sfdx --strip-components 1;
      sudo ./sfdx/install;
      sfdx force:auth:jwt:grant --clientid $CONSUMER_KEY --jwtkeyfile $DOWNLOADSECUREFILE_SECUREFILEPATH --username $USERNAME --setalias developer;
      sfdx force:mdapi:retrieve -u $USERNAME -k src/package.xml -r .;
      unzip unpackaged.zip;
      cd unpackaged;
      git config --global user.email "test@test.com"
      git config --global user.name "Test User"
      git checkout -t origin/developer;
      git add .;
      git status;
      echo "This is me: ";
      whoami;
      git commit -m "Retrieved latest Developer SB changes!"
      echo "Pushing now!!!";
      git push --set-upstream origin HEAD:developer;

```

As per Microsoft's documentation found here, we need to make sure we grant permissions to the build service. All the permissions have been granted as requested but we still can't get it to work.

Below is the copy of the logs generated when we run the Pipeline:

```
##[section]Starting: Bash
==============================================================================
Task         : Bash
Description  : Run a Bash script on macOS, Linux, or Windows
Version      : 3.151.3
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/bash
==============================================================================
Generating script.
========================== Starting Command Output ===========================
[command]/bin/bash --noprofile --norc /home/vsts/work/_temp/9d534f54-57b8-44a0-8d10-2e3d8c334bbf.sh
--2019-08-13 13:27:00--  https://developer.salesforce.com/media/salesforce-cli/sfdx-linux-amd64.tar.xz
Resolving developer.salesforce.com (developer.salesforce.com)... 151.101.249.145
Connecting to developer.salesforce.com (developer.salesforce.com)|151.101.249.145|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 19686680 (19M) [application/x-tar]
Saving to: ‘sfdx-linux-amd64.tar.xz’

     0K .......... .......... .......... .......... ..........  0% 25.6M 1s
    50K .......... .......... .......... .......... ..........  0% 27.2M 1s
        ...
 19150K .......... .......... .......... .......... .......... 99%  171M 0s
 19200K .......... .......... .....                           100%  247M=0.1s

2019-08-13 13:27:00 (133 MB/s) - ‘sfdx-linux-amd64.tar.xz’ saved [19686680/19686680]

sfdx installed to 4m/usr/local/bin/sfdx
sfdx-cli/7.8.1-8f830784cc linux-x64 node-v10.15.3
(node:3433) [EACCES] Error Plugin: sfdx-cli: EACCES: permission denied, open '/home/vsts/.cache/sfdx/lastrun'
module: @oclif/config@1.13.0
task: runHook init
plugin: sfdx-cli
root: /usr/local/lib/sfdx
See more details with DEBUG=*
Successfully authorized pipeline@coop.org.developer with org ID 00D560340009rbvEAA
(node:3469) [EACCES] Error Plugin: sfdx-cli: EACCES: permission denied, open '/home/vsts/.cache/sfdx/lastrun'
module: @oclif/config@1.13.0
task: runHook init
plugin: sfdx-cli
root: /usr/local/lib/sfdx
See more details with DEBUG=*
this.retrieveTargetPath: /home/vsts/work/1/s
Retrieving source...

=== Status
Status:  InProgress
jobid:  09S56000001G6fmEAC
...
=== Result
Status:  Succeeded
jobid:  09S56000001G6fmEAC

Wrote retrieve zip to /home/vsts/work/1/s/unpackaged.zip.
Archive:  unpackaged.zip
  inflating: unpackaged/settings/Account.settings
  ...
  inflating: unpackaged/pathAssistants/TMG_CBE_Path.pathAssistant
  inflating: unpackaged/package.xml
Switched to a new branch 'developer'
Branch 'developer' set up to track remote branch 'developer' from 'origin'.
On branch developer
Your branch is up to date with 'origin/developer'.

Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

    new file:   approvalProcesses/Billing_Item__c.Billable_Item_Less_than_0.approvalProcess
  ...
    new file:   workflows/Touchpoint__c.workflow
    new file:   workflows/User.workflow

Untracked files:
  (use "git add <file>..." to include in what will be committed)

    ../sfdx-linux-amd64.tar.xz
    ../sfdx/
    ../unpackaged.zip

This is me:
vsts

[developer f8ea8ed] Retrieved latest Developer SB changes!
 1332 files changed, 452246 insertions(+)
 create mode 100644 unpackaged/approvalProcesses/Billing_Item__c.Billable_Item_Less_than_0.approvalProcess
 ...
 create mode 100644 unpackaged/workflows/Touchpoint__c.workflow
 create mode 100644 unpackaged/workflows/User.workflow
Pushing now!!!
remote: 001f# service=git-receive-pack
remote: 0000000000aaTF401027: You need the Git 'GenericContribute' permission to perform this action. Details: identity 'Build\49a2cab3-a4ad-46c4-994c-35f429c806d2', scope 'repository'.
remote: TF401027: You need the Git 'GenericContribute' permission to perform this action. Details: identity 'Build\49a2cab3-a4ad-46c4-994c-35f429c806d2', scope 'repository'.
fatal: unable to access 'https://dev.azure.com/cooporg/test/_git/test/': The requested URL returned error: 403
##[error]Bash exited with code '128'.
##[section]Finishing: Bash

```

I added the "whoami" command to confirm the running user and to our surprise it's running as "vsts". We tried searching for this user to grant it the permissions needed, but we can't find it.

Has anybody implemented something similar? How do we execute Git commands as a specific user within a Pipeline, so we can push the changes back to our branch? The ideal scenario would be committing these changes based on the user that manually ran the Pipeline.

Thanks.

# Answer
Found the issue...

The documentation states that the "contribute" permission needs to be added to "Project Collection Build Service Accounts". There is also an USER called ""Project Collection Build Service". After explicitly adding the "Project Collection Build Service" user to be able to contribute it worked as expected!
> 3 votes

---
Tags: azure-devops, azure-pipelines
---


# Question
Title: Docker fails to mount volume inside Jenkins scripted pipeline
I'm working on a CI Jenkins build with scripted pipeline. From Jenkins on one machine, I'm connecting with already configured Jenkins slave on another machine (M2). I'm able to successfully execute docker commands within the jenkins slave, however it seems that docker is unable to mount the volumes from the slave to the freshly started docker image.

```
node('testImage') {
   ... //do some job including creation of volumes within the "testImage" container
    stage('Run WildFly instances') {
        //run wildfly relevant
        sh "docker run -d --restart=unless-stopped \
           -v /some/path/to/volume/in/jenkins/slave/container:/some/path/to/volume/within/wildfly14/container \
           -p 8380:8080 \
           -p 9380:9990 \
           -e JAVA_OPTS='-Xms256m -Xmx512m -Djava.net.preferIPv4Stack=true' \
           --name wildfly14 \
          ${DockerRegistryBase}${WildflyImageName}:${WildflyImageVersion}"
}

```

/some/path/to/volume exists within the jenkins slave docker image, because it's created within the lifecycle of the pipeline, but the wildfly14 container that I'm trying to run is unable to access /some/path/to/volume, because when I run

```
docker exec -it wildfly14 ls -lR /some/path/to/volume/within/wildfly14/container

```

it's simply empty. Any suggestions on why is it even happening? I've noticed that the docker containers are set up outside of the jenkins slave container, on the actual machine (M2). Could it be the reason that docker tries to map a volume from outside of the container (/some/path/to/volume) on M2 instead of the location within the jenkins slave?

# Answer
In configuration settings of Docker plugin for Jenkins, there were missing mountings for the volumes, because I was deploying docker images within docker slave container, so that the "master" docker container (with Jenkins slave agent) couldn't access the real files from the host machine.
> 2 votes

---
Tags: docker, jenkins, jenkins-pipeline, jenkinsfile
---


# Question
Title: When your micro service in production needs a secret, where is the secret stored and retrieved from?
The context of the question is Secrets management.

How do you manage storing and retrieving secrets for your micro service which is in production?

# Answer
# Short answer

Ideally, you should store secrets as environment variables, and retrieve them from a secrets management system like Hashicorp's Vault or AWS Parameter store.

# Long answer

I saw your questions out of turn, and kinda touched on this in your other question:

> Again, there are many perfectly valid options for handling secrets: Chef vault, 1password, environment variables injected by your build instances, home-grown solution using encrypted files stored on object storage, etc....

It really depends on how you run and deploy your apps. I worked at a company that used Amazon Beanstalk for production, and they shipped all their secrets and configs as a 'secrets.conf' file that was created by the Jenkins build server, and shipped with the app when it deployed. The problem there was that it was unruly to manage the same secrets across multiple jobs in Jenkins. So, I came in and moved them towards using Chef to create the file instead, and Chef pulled the file from Hashicorp Vault.

At my current company, they are heavy users of GitLab with it's built-in CI/CD build system. Because the the apps are all Docker instances running in Kubernetes, all secrets are actually set as environment variables set during the deployment. The build server gets those from 'secret variables' that is set from the GitLab UI.

These fit the business need at the time, and that's what really matters at the end of the day. However, you if want to be proper, you should store secrets as environment variables because you specifically mention micro services. It is highly recommend that you follow the 12 Factor App approach to configs: https://12factor.net/config

> The twelve-factor app stores config in environment variables (often shortened to env vars or env). Env vars are easy to change between deploys without changing any code; unlike config files, there is little chance of them being checked into the code repo accidentally; and unlike custom config files, or other config mechanisms such as Java System Properties, they are a language- and OS-agnostic standard.

So you'll definitely want to leverage environment variables. How you *set* those environment variables could be your Configuration Management system (Ex: Chef), or your build server when it does the deploy, or your apps could be coded to pull directly from your secret storage and set the variables internally.

For actual place where the secrets live (where they are retrieved *from*), you have a few options. But the 2 you mentioned in your comment would be really great ones to use: Hashicorp's Vault and AWS Parameter store. Others would be Chef Vault, home-grown solution using encrypted files stored on object storage, 1password, etc.... Ideally, you'd want something that can be programmatically access from your build pipeline or apps.
> 5 votes

# Answer
We use Chef Vault at my company. Here's a quick tutorial How to Create Chef-Vault
> 0 votes

# Answer
Take a look at https://pkhub.io its a cloud secrets manager that's easy to setup and use, see https://pkhub.io/usecases/environments, for a python example.
> 0 votes

---
Tags: amazon-web-services, security, microservices, github, encryption
---


# Question
Title: Environment variable names, when my development environment has multiple projects
The context is that I'm looking at doing staging/testing and production on a kubernetes cluster.

Where I might have some code that looks like this:

```
doSomething({
  username: process.env.DB_USERNAME,
  password: process.env.DB_PASSWORD
})

```

For my various kubernetes environments, this isn't going to be a problem, as those environment variables can be set specifically for the environment.

However, for my development environment, there's a good chance that I'm going to have multiple projects, (or even different images for the same project) that all have a environment variable named `DB_URL`.

One solution would be to just prefix all environment variables with `MY_APPLICATION_NAME_`.

Another would be not permanently set environment variables, but instead to just call a script that sets them when needed. But that sounds a little messy.

I'm wondering if there's a well established way that deals with this problem.

# Answer
> 1 votes
check out https://direnv.net/ it sets environment variables based on the directory you are in.

other options are: 1.docker-compose 2.plain bash to set the variables. 3. bash source .env 4. use prefixes but you would still need to set the environment you are in via an environment variable, so back to options 1-3.

if you are going to store passwords or keys in the env variables locally that are sensitive i.e not a localhost db pass, use a secrets manager like https://pkhub.io/usecases/environments, or you could script up something with aws' secrets manager

# Answer
> 0 votes
Not sure if this is the first, but it surely won't be the last time. Having a script that starts a new shell (new container, new VM) with environment setup for the development of the project you will be working on next is something you will definitely need in the future. If there are multiple people collaborating, it is even more important. If you get into the habit of using it now, future changes will be easier.

The reason why you set it in new shell (container/VM) is that you can easily exit to unset the environment and enter a new one for a different project and because you might need to set a different user or group memberships for the project.

---
Tags: environment-variables
---


# Question
Title: Why does Docker define invalid IP address of service?
I have web-app with PHP and database in docker-compose file:

```
version: '2'

networks:
  default:
    driver: bridge

services:

  yii2-app:
    build:
      context: ./web
    ports:
      - "80:80"
    networks:
      - default
    depends_on:
      - yii2-db
      - queue
    environment:
      DB_NAME: ${WEB_DB_NAME}
      DB_USERNAME: ${WEB_DB_USERNAME}
      DB_PASSWORD: ${WEB_DB_PASSWORD}

  yii2-db:
    image: mysql:5.7.16
    command: --default-authentication-plugin=mysql_native_password
    networks:
      - default
    expose:
      - "3306"
    environment:
      MYSQL_ROOT_PASSWORD: ${WEB_DB_PASSWORD}
      MYSQL_DATABASE: ${WEB_DB_NAME}
    volumes:
      - "./web/db:/var/lib/mysql"
  ...

```

When I start services via `docker-compose up` my PHP-app makes db migrations. But this error occured in database connection:

```
Exception 'yii\db\Exception' with message 'SQLSTATE[HY000] [1045] Access denied for user 'root'@'172.21.0.8' (using password: YES)'

```

It's strange, because in my PHP-app I use name of db service `yii2-db` and docker by itself to define the correct IP.

I connect to my PHP-app container via

```
docker exec -it passportrecognition_yii2-app_1 bash

```

and check mysql connection via mysql-client

```
mysql -h yii2-db -u root -p
ERROR 1045 (28000): Access denied for user 'root'@'172.21.0.8' (using password: YES)

```

But `172.21.0.8` is not the IP of my db, it's the IP of my PHP-app. That is PHP-app make db-connect to itself.

Database IP is `172.21.0.4` as shown with this `ping` command:

```
ping yii2-db
PING yii2-db (172.21.0.4): 56 data bytes
64 bytes from 172.21.0.4: seq=0 ttl=64 time=0.145 ms

```

And another strange moment: when I connect directly on database IP `172.21.0.4`, the following error occurs:

```
bash-4.4# mysql -h 172.21.0.4 -u root -p
Enter password:
ERROR 1045 (28000): Access denied for user 'root'@'172.21.0.8' (using password: YES)

```

Why can't I connect to my database? Why does Docker define an invalid IP for the db service?

# Answer
Ok, I have found the reason:

I tested starting the services with different environment variables for db. And after that I shut the services down, changed the db name, username and password, and started the services again.

But I forgot that db folder `/var/lib/mysql` is mounted in a volume, and it remembered my previous data; everything was mixed up. So it's not problem in network, it's mixed data in mysql (different passwords for the same user).
> 1 votes

---
Tags: docker, docker-compose
---


# Question
Title: Scale number of pods in OpenShift from oc command line
I am new to Openshift , and want know how to change the number of pods of specific deployment from `oc` command line.

I tried to do:

oc scale dc name\_of\_deployment--replicas=2

Error from server (NotFound): deploymentconfigs.apps.openshift.io "name\_of\_deployment" not found

My deployments were installed with helm, if i do

```
oc  get dc

```

I am getting some other deployment but not the one that I want to scale.

Note: The deployment that I want to delete is running without any issue.

# Answer
found the issue, instead of 'dc' need to use 'deploy'

```
oc scale deploy name_of_deployment--replicas=2

```
> 4 votes

---
Tags: openshift
---


# Question
Title: IOS_Command Module - Delayed Prompt Issue (Cisco 3850 - IOS XE)
I'm new to the Ansible world, so I'll do my best to explain everything. If I leave something out or need to clarify, just let me know.

So, I've been working on upgrades for various Cisco switches/routers, and the 3850s running IOS-XE 16.x are causing me a slight issue. Basically, I want to run the command `request platform software package clean switch all file flash:` which will result in a prompt that says `Do you want to proceed? [y/n]` to which you simply enter "y" without the need for a CR.

It's important to note that if you were just copying/pasting configuration from a txt file, you could not paste in the "y" and you would manually need to enter it in. Doing so prevents the prompt from appearing (you can interact with the console while you wait 15-45 seconds for the output).

My Ansible code is waiting for the prompt, but when I run in -vvvv I do not see the output of the command and it ultimately times out. If I change the prompt to '\[confirm\]' and the response to "\r" I do see the output, but the cleanup is not taken (I can confirm because there would be more output and I can manually check after the fact).

I thought the prompt parameter would wait until the prompt occurs, but I'm guessing not? How does this actually work? It has not been a problem before because the prompts are generally instant. In this case, there's a delay every time.

Here's my code:

```
- name: CLEAN UP OLD SOFTWARE
  ios_command:
    commands:
      - command: "request platform software package clean switch all file flash:"
        prompt: Do you want to proceed? \[y/n\]
        answer: "y"
  vars:
    ansible_command_timeout: 90
  when: compliant_ios_version == ansible_net_version

```

This is the result:

```
WARNING: The below traceback may *not* be related to the actual failure.
  File "/tmp/ansible_ios_command_payload_XyRXx3/ansible_ios_command_payload.zip/ansible/module_utils/network/ios/ios.py", line 145, in run_commands
    return connection.run_commands(commands=commands, check_rc=check_rc)
  File "/tmp/ansible_ios_command_payload_XyRXx3/ansible_ios_command_payload.zip/ansible/module_utils/connection.py", line 182, in __rpc__
    raise ConnectionError(to_text(msg, errors='surrogate_then_replace'), code=code)
fatal: [SWITCH1]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "auth_pass": null,
            "authorize": null,
            "commands": [
                {
                    "answer": "y",
                    "command": "request platform software package clean switch all file flash:",
                    "prompt": "Do you want to proceed? \\[y/n\\]"
                }
            ],
            "host": null,
            "interval": 1,
            "match": "all",
            "password": null,
            "port": null,
            "provider": null,
            "retries": 10,
            "ssh_keyfile": null,
            "timeout": null,
            "username": null,
            "wait_for": null
        }
    },
    "msg": "command timeout triggered, timeout value is 40 secs.\nSee the timeout setting options in the Network Debug and Troubleshooting Guide."
}
        to retry, use: --limit @/opt/ansible/Test_Clean.retry

PLAY RECAP *************************************************************************************************************************
SWITCH1                   : ok=0    changed=0    unreachable=0    failed=1

Playbook run took 0 days, 0 hours, 0 minutes, 92 seconds

```

Here's what would be displayed on the switch console:

```
SWITCH1# request platform software package clean switch all file flash:
This operation may take several minutes...
Running command on switch 1
Cleaning up unnecessary package files
  Scanning boot directory for packages ... done.
  Preparing packages list to delete ...
    packages.conf
      File is in use, will not delete.
    cat3k_caa-guestshell.16.06.06.SPA.pkg
      File is in use, will not delete.
    cat3k_caa-rpbase.16.06.06.SPA.pkg
      File is in use, will not delete.
    cat3k_caa-rpcore.16.06.06.SPA.pkg
      File is in use, will not delete.
    cat3k_caa-srdriver.16.06.06.SPA.pkg
      File is in use, will not delete.
    cat3k_caa-webui.16.06.06.SPA.pkg
      File is in use, will not delete.
  done.

Running command on switch 2
Cleaning up unnecessary package files
  Scanning boot directory for packages ... done.
  Preparing packages list to delete ...
    cat3k_caa-rpbase.16.06.06.SPA.pkg
      File is in use, will not delete.
    packages.conf
      File is in use, will not delete.
    cat3k_caa-guestshell.16.06.06.SPA.pkg
      File is in use, will not delete.
    cat3k_caa-rpcore.16.06.06.SPA.pkg
      File is in use, will not delete.
    cat3k_caa-srdriver.16.06.06.SPA.pkg
      File is in use, will not delete.
    cat3k_caa-webui.16.06.06.SPA.pkg
      File is in use, will not delete.
  done.

The following files will be deleted:
[switch 1]:
/flash/cat3k_caa-guestshell.16.06.05.SPA.pkg
/flash/cat3k_caa-rpbase.16.06.05.SPA.pkg
/flash/cat3k_caa-rpcore.16.06.05.SPA.pkg
/flash/cat3k_caa-srdriver.16.06.05.SPA.pkg
/flash/cat3k_caa-universalk9.16.06.06.SPA.conf
/flash/cat3k_caa-webui.16.06.05.SPA.pkg
/flash/packages.conf.00-
[switch 2]:
/flash/cat3k_caa-guestshell.16.06.05.SPA.pkg
/flash/cat3k_caa-rpbase.16.06.05.SPA.pkg
/flash/cat3k_caa-rpcore.16.06.05.SPA.pkg
/flash/cat3k_caa-srdriver.16.06.05.SPA.pkg
/flash/cat3k_caa-universalk9.16.06.06.SPA.conf
/flash/cat3k_caa-webui.16.06.05.SPA.pkg
/flash/packages.conf.00-

Do you want to proceed? [y/n]

```

I've searched around a lot and tried everything I could think of. I have not seen anyone come across this specific issue yet. Any ideas?

I'd appreciate any help. I'm pretty much dead in the water at this point.

# Answer
Been there done that... Try prompt '\]' and upgrade ansible to the version 2.8.1

```
    "stdout_lines": [
        [
            "Running command on switch 1",
            "Cleaning up unnecessary package files",
            "No path specified, will use booted path flash:packages.conf",
            "Cleaning flash:",
            "  Scanning boot directory for packages ... done.",
            "  Preparing packages list to delete ... ",
            "    cat3k_caa-guestshell.16.03.07.SPA.pkg",
            "      File is in use, will not delete.",
            "    cat3k_caa-rpbase.16.03.07.SPA.pkg",
            "      File is in use, will not delete.",
            "    cat3k_caa-rpcore.16.03.07.SPA.pkg",
            "      File is in use, will not delete.",
            "    cat3k_caa-srdriver.16.03.07.SPA.pkg",
            "      File is in use, will not delete.",
            "    cat3k_caa-wcm.16.03.07.SPA.pkg",
            "      File is in use, will not delete.",
            "    cat3k_caa-webui.16.03.07.SPA.pkg",
            "      File is in use, will not delete.",
            "    packages.conf",
            "      File is in use, will not delete.",
            "  done.",
            "  ",
            "The following files will be deleted:",
            "[1]:",
            "/flash/cat3k_caa-guestshell.16.03.06.SPA.pkg",
            "/flash/cat3k_caa-rpbase.16.03.06.SPA.pkg",
            "/flash/cat3k_caa-rpcore.16.03.06.SPA.pkg",
            "/flash/cat3k_caa-srdriver.16.03.06.SPA.pkg",
            "/flash/cat3k_caa-universalk9.16.03.07.SPA.conf",
            "/flash/cat3k_caa-wcm.16.03.06.SPA.pkg",
            "/flash/cat3k_caa-webui.16.03.06.SPA.pkg",
            "/flash/packages.conf.00-",
            "",
            "Do you want to proceed? [y/n]y",
            "",
            "[1]:",
            "Deleting file flash:cat3k_caa-guestshell.16.03.06.SPA.pkg ... done.",
            "Deleting file flash:cat3k_caa-rpbase.16.03.06.SPA.pkg ... done.",
            "Deleting file flash:cat3k_caa-rpcore.16.03.06.SPA.pkg ... done.",
            "Deleting file flash:cat3k_caa-srdriver.16.03.06.SPA.pkg ... done.",
            "Deleting file flash:cat3k_caa-universalk9.16.03.07.SPA.conf ... done.",
            "Deleting file flash:cat3k_caa-wcm.16.03.06.SPA.pkg ... done.",
            "Deleting file flash:cat3k_caa-webui.16.03.06.SPA.pkg ... done.",
            "Deleting file flash:packages.conf.00- ... done.",
            "SUCCESS: Files deleted."
        ]
    ]

```
> 1 votes

---
Tags: ansible, networking, ios
---


# Question
Title: How do I get more control over the NodePorts my Service assigns to my Deployment?
I have a Deployment with 5 replicas. All have ssh and telnet. They should not be load balanced. I would like each to select from a predictable list of 5.

Here is my Deployment

```
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
spec:
  selector:
    matchLabels:
      app: myapp
  replicas: 5
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
      - name: myapp
        image: myapp:1.0
        ports:
        - name: ssh
          protocol: TCP
          containerPort: 22
        - name: telnet
          protocol: TCP
          containerPort: 23

```

This is my Service with invalid nodePort values for illustrative purposes.

```
apiVersion: v1
kind: Service
metadata:
  name: myapp
  labels:
    app: myapp
spec:
  type: NodePort
  ports:
  - name: ssh
    port: 22
    nodePort: [30022, 30122, 30222, 30322, 30422, 30522]
  - name: telnet
    port: 23
    nodePort: [30023, 30123, 30223, 30323, 30423, 30523]

```

I am hoping to be able to accomplish 2 things:

1. Each pod replica instance will only get an ssh port from \[30022, 30122, 30222, 30322, 30422, 30522\] and a telnet port from \[30023, 30123, 30223, 30323, 30423, 30523\]
2. A pod replica instance that gets an ssh port of 30022 also gets the telnet port 30023. A pod replica instance that gets an ssh port of 30122 gets a telnet port of 30123 and so on.

Thank you!

# Answer
As far as I understand kubernetes services, that trick is not possible.

What could help you to have preserved hostnames is a StatefulSet. See more here

> The Pods’ ordinals, hostnames, SRV records, and A record names have not changed, but the IP addresses associated with the Pods may have changed. In the cluster used for this tutorial, they have. This is why it is important not to configure other applications to connect to Pods in a StatefulSet by IP address.
> 1 votes

# Answer
If the pods (quote) "...should not be load balanced", each pod is to be accessed on different NodePort ports, the order of starting them up doesn't matter...you can:

* Create 5 identical Deployment-s, say, myapp-1 thru myapp-5, with 1 replica each.
* Create a Service for each Deployment making sure the selector is correct for each myapp-1 thru myapp-5 Deployment-s. Configure each Service with the concrete nodePort-s for ssh and telnet as you wish.

It is to note that this way each pod will be independent of the rest and pods cannot be scaled by simply adding a replica. Still, if configured, the pods will be rescheduled and restarted in case of pod or node failure.
> 0 votes

---
Tags: kubernetes
---


# Question
Title: How to make an AWS S3 bucket accessible over HTTP?
Does `AWS S3` provide a way to make bucket's files available over `HTTP` preferably protected with `BASIC_AUTH`? I mean listing the files (index) and make it possible to download them over `HTTP`.

# Answer
1. **Download - yes.** Simply set each Object ACL to be publicly readable.
2. **List - sort of.** If you set the Bucket ACL to publicly readable anyone can list the content. However it's a rather human-unfriendly XML listing.

You can generate your directory listings to some `index.html` files through an external process and configure the bucket to serve those.
3. **Basic auth - nope.** You can look at something like S3 pre-signed URLs to protect the content.

Hope that helps :)
> 2 votes

# Answer
One possibility is mounting the bucket as `s3fs` and serving the root folder as usually done by Nginx.
> 2 votes

# Answer
In addition to Mlu answer, you can put Cloud front in front of the bucket and implement basic auth and probably listing/sorting with a Lambda@edge function.
> 1 votes

---
Tags: amazon-web-services, amazon-s3, http
---


# Question
Title: Ansible ec2_vpc_net and ec2_vpc_subnet - where to set subnet default public ip
Since upgrading to these 2 Ansible modules I am having difficulties in setting the subnets to have default public IP when new instances are created by autoscaling?

I can't find any setting on the documentation for this, how do I achieve this?

https://docs.ansible.com/ansible/latest/modules/ec2\_vpc\_net\_module.html

https://docs.ansible.com/ansible/latest/modules/ec2\_vpc\_subnet\_module.html

# Answer
> 1 votes
map\_public parameter set to true in the second link should do this.

---
Tags: ansible, amazon-ec2, aws-vpc
---


# Question
Title: How to get the AWS Access key of an already created Access ID
Inside `IAM management`, I navigate to `Users - User` and got to the `Security Credentials` tab, It looks like this:

I see no link to show the `access key` of the already generated `access ID`, how to get it?

# Answer
**Access Key** is that `AKIA...` one. What you need is a **Secret Key** \- that can't be recovered after the initial setup.

Since you have never used this particular access key (it says "Last used: N/A") it's safe to *Make inactive* and then *Delete* and then create a new one through **Create access key** button above the form.

Hope that helps :)
> 4 votes

# Answer
You can download or read the **Secret Key** only when you set it up for the first time.

If you lost the **Secret Key** / didn't manage to save it and is not being used anywhere, it is better to *Delete* and create a new one.

Don't forget to copy the **Secret Key** before you move to any other screen this time :)
> 1 votes

---
Tags: amazon-web-services, aws-iam
---


# Question
Title: CI pipelines, concurrency and build results possibly overwriting each-other
I'm currently using gitlab-ci to generate docker images for some repositories, however i dont feel as though this solution is as reliable as it should be.

I can forsee a issue with multiple builds happening close together and build #100 overwriting the newly built docker image produced by build #101, if for whatever reason jobs dont finish sequentially (eg: multiple build hosts, etc...).

Currently the tags i'm hoping to use follow this pattern: `namespace/application:branch-name`

The crux of the question is as follows:

1. How do other services solve this problem? (presumibly running builds sequentially per branch?)
2. How should this issue be resolved in gitlab environments in a totally watertight manner

I understand this problem would go away if i use the SHA of the commit for the build tag, but i'd like to stick to the branch name for the following reasons:

* orphaned / old images will be grabage collected by the docker registry
* branch name is easier to use

# Answer
> 1 votes
I am using AzureDevOps and the trick we use when naming containers is to add build number to the label

```
namespace/application:branch-name-bulidNumber

```

So in your case, you could try gitlab variable `CI_CONCURRENT_PROJECT_ID`

# After comment

for images clean-up you can use:

```
  docker image prune --all --filter until=48h

```

As docker system prune and docker image prune have the until filter. So docker image prune --all --filter **until=48h** would remove all (not just dangling) images that were created more than 48 hours ago. Hopefully that helps. source here

---
Tags: docker, gitlab, gitlab-ci-runner, docker-registry
---


# Question
Title: Terraform | Upgrade to 0.12 error Reference to "count" in non-counted context
I'm trying to upgrade terraform from 0.11 to 0.12 but receiving error:

```
 Error: Reference to "count" in non-counted context

  on main.tf line 33, in data "aws_subnet" "public":
  33:   id = data.aws_subnet_ids.public.ids[count.index]

The "count" object can be used only in "resource" and "data" blocks, and only
when the "count" argument is set.

```

Here is my part of code:

```
data "aws_subnet_ids" "public" {
  vpc_id = data.aws_vpc.vpc.id

  tags = {
    Tier = "public"
  }
}

data "aws_subnet" "public" {
  id = data.aws_subnet_ids.public.ids[count.index]
}

```

Maybe someone know how to fix this error? Thanks in advance.

# Answer
> 1 votes
It looks like your underlying goal here is to retrieve the full set of attributes for each of your subnets that are marked as `Tier = public`. Here's a different way to do that using features from Terraform 0.12.6:

```
data "aws_subnet_ids" "public" {
  vpc_id = data.aws_vpc.vpc.id

  tags = {
    Tier = "public"
  }
}

data "aws_subnet" "public" {
  for_each = data.aws_subnet_ids.public.ids

  id = each.value
}

```

With the above, you should find that `data.aws_subnet.public` is a map from subnet id to the attributes of that particular subnet.

---

The reason for the error in your case is that you used `count.index` but didn't set the `count` argument, and so there is no count index to return.

The `count`-based equivalent of the above, which is more similar to your original example, would be this:

```
data "aws_subnet_ids" "public" {
  vpc_id = data.aws_vpc.vpc.id

  tags = {
    Tier = "public"
  }
}

data "aws_subnet" "public" {
  count = length(data.aws_subnet_ids.public.ids)

  id = sort(data.aws_subnet_ids.public.ids)[count.index]
}

```

The result of this is very similar to the `for_each`-based example I gave above, but with one significant difference: in this case `data.aws_subnet.public` will be a *list* of subnet objects, ordered by the lexical ordering of their subnet ids. That will usually make the result harder to use elsewhere in the configuration, so I'd suggest using the `for_each` approach unless there's a specific reason why you need a list.

(If you use a map and then later find that you need a list in a *specific context*, you can always use `values(data.aws_subnet.public)` to take the values from the map, in the same lexical order.)

---
Tags: terraform
---


# Question
Title: Problem with logging in docker when using fluentd for logging of three flask apps running in a docker network behind an nginx
I am facing a strange problem with logging when I set up a network of three docker containers each running a small flask app.

My python application looks as follows:

```
from flask import Flask, request
import sys
import logging
import logging.handlers
import os

app = Flask(__name__)

logger = logging.getLogger('Logger')
logger.setLevel(logging.DEBUG)
log_handler = logging.StreamHandler(sys.stdout)
logger.addHandler(log_handler)

service_name=""
if 'SERVICE_NAME' in os.environ:
    service_name = os.environ['SERVICE_NAME']

logger = logging.LoggerAdapter(logger, {"serviceName" : service_name})

@app.route("/ping")
def ping():
    logger.info(f'answering ping request')

    return f'{service_name} OK'

if __name__ == "__main__":
    logger.info("Started")

    app.run(debug=True, host='0.0.0.0')

```

I start three instances of this in a docker network behind nginx:

```
docker network create tNet

docker run -it -d -p 24224:24224 -v `pwd`:/fluentd/etc -e FLUENTD_CONF=fluentd.conf --name=fluentd_logging fluent/fluentd

docker run -d --log-driver=fluentd --name ms1 -e SERVICE_NAME="ms1" --network tNet testflask
docker run -d --log-driver=fluentd --name ms2 -e SERVICE_NAME="ms2" --network tNet testflask
docker run -d --log-driver=fluentd --name ms3 -e SERVICE_NAME="ms3" --network tNet testflask

docker run -d --name=nginx --network tNet --volume `pwd`/nginx.conf:/etc/nginx/conf.d/default.conf -p 2000:2000 -p 3000:3000 nginx

```

The nginx configuration is as follows:

```
upstream app {
    server ms1:5000;
    server ms2:5000;
    server ms3:5000;
}

server {
    listen 2000;

    location / {
        proxy_pass http://app;
    }
}

```

My fluent configuration looks as follows:

```
<source>
    @type forward
</source>

<match>
    @type stdout
</match>

```

When I use curl I can see that each instance of the service is responding:

```
curl localhost:2000/ping
ms1 OK
curl localhost:2000/ping
ms2 OK
curl localhost:2000/ping
ms3 OK

```

When I however inspect the logs I can only see logs from the 3rd instance of the service:

```
docker logs <fluent container ID> | grep -Eis "answering ping"

19-08-06 15:34:46.000000000 +0000 be338a3d5aa0: {"container_id":"be338a3d5aa0b8920366fdf1ba2a267082cee47e681adcea16b6c51c9c82cf0d","container_name":"/ms3","source":"stdout","log":"answering ping request"}

```

Has anyone an idea? Thank you in advance.

# Answer
I finally found out the problem: the fluent container needs some time to start up; because it's started with `--detach` the subsequent containers try to connect before fluentd is actually running, only the 3rd instance of the flask application connects when fluentd is already up. So, delaying the startup of the remaining containers solves my problem.
> 0 votes

---
Tags: microservices, logging, flask
---


# Question
Title: CORS and subdomains - do subdomains matter?
Will CORS ever be a problem when you have x.example.com going to y.example.com or only if the root domain is different?

# Answer
> 4 votes
CORS is not allowing subdomains, so you need to specify them in your server configuration.

If you are using NGINX (or you could use it as a proxy and solve your problem) by providing dynamic cors header response

```

server {

    root /path/to/your/stuff;

    index index.html index.htm;

     set $cors "";

    if ($http_origin ~* (.*\.yoursweetdomain.com)) {
        set $cors "true";
    }

    server_name yoursweetdomain.com;

    location / {

        if ($cors = "true") {
            add_header 'Access-Control-Allow-Origin' "$http_origin";
            add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS, DELETE, PUT';
            add_header 'Access-Control-Allow-Credentials' 'true';
            add_header 'Access-Control-Allow-Headers' 'User-Agent,Keep-Alive,Content-Type';
        }

        if ($request_method = OPTIONS) {
            return 204;
        }

    }
}

```

Source here

---
Tags: amazon-web-services, cors
---


title: Microservices Deployment Manager

text:
# Question
Title: Microservices Deployment Manager
I was wondering how information on deployed applications are kept in real, production environments (yeah, I am a novice) . For example, in a microservices based set up, if 5 microservices-based applications were deployed in production environments. Let's say each of those had 4 - 6 instances each, are there special software for keeping record of these applications? I know the records can be easily written into a database but I can imagine some better solutions out there.

# Answer
According to the Cloud Native Landscape, you are looking for a `Coordination & Service Discovery`:

https://etcd.io/

> A distributed, reliable key-value store for the most critical data of a distributed system

https://github.com/Netflix/eureka

> Eureka is a REST (Representational State Transfer) based service that is primarily used in the AWS cloud for locating services for the purpose of load balancing and failover of middle-tier servers.

https://github.com/alibaba/nacos

> Nacos (official site: http://nacos.io) is an easy-to-use platform designed for dynamic service discovery and configuration and service management. It helps you to build cloud native applications and microservices platform easily.
> 1 votes

---
Tags: deployment, continuous-deployment, microservices
---


# Question
Title: What's are the advantages of using CloudBees Core over Jenkins?
I have tried Jenkins on my own and about to propose an enterprise grade CI/CD product to my development team which is around 40 people. CloudBees Core come at the top of my selected list since it is based on Jenkins and enterprise friendly as well.Apart from enterprise level support, I still can't really see the benefits to use CloudBees Core over Jenkins.

Is there anyone who used Cloudbees through your DevOps transformation journey ?

I would appreciate if you could discuss proprietary plugins and out of the box features that CloudBees Core provides.

# Answer
> 1 votes
I personally prefer Jenkins over CloudBees as we have enough Jenkins knowledge in our engineering department. However, if their is a lack of such knowledge or interest then it could be useful to use a Software as a Service (SaaS) product like CloudBees. In general, this means that the department will be unburdened as the platform will scale automatically and the SaaS provider is responsible for keeping Jenkins up and running.

A downside of an Enterprise solution is that a fee is required. However, maintenance and in particular the number of resources that is required to apply it, is often underestimated in my opinion. Maintaining Jenkins by Engineers from the department means that they have less time to develop code. At the moment, getting a CloudBees subscription will not solve our problems as we are currently busy splitting the monolith.

Whether you should get a CloudBees membership really depends on your situation. Do you have a monolith? Has all code been untangled. Do you have microservices with clear boundaries or are they nanoservices? Long story short, do you have a robust architecture? If false, then CloudBees will also not be the holy grail.

While Googling, another post on stackoverflow was found that could be useful:

> The first difference is support (as others have mentioned). CloudBees offers enterprise grade support as well as a fully vetted and tested version of Jenkins that will be more stable under various plugins and deployments. You can actually purchase "Support Only" from CloudBees if you are satisfied with your OSS Jenkins deployment and simply want support during upgrades, patching, break/fix, etc.
>
> From a feature perspective, CloudBees brings a lot from an enterprise manageability, scalability, and security standpoint.
>
> * Manageability: CloudBees comes with CJOC (CloudBees Jenkins Operations Center) built into the software. This is a single pane of glass management console that allows organizations or large teams to centrally manage the jenkins environment. Things like folders, RBAC, pipeline and master templates, and the ability to rapidly spin up/tear down a containerized jenkins master are all managed from this single console.
> * Scaleability: CloudBees leverages Kubernetes to provide organizations with the ability to elastically scale Jenkins environments as needed. With CloudBees, your oganization can move away from a single "Monolithic"/"Frankenstein" master and into a multi-master and distributed pipeline architecture. This greatly reduces upgrade and administration complexity. This also eliminates the risk from a single point of failure that a monolithic architecture exposes.
> * Security: CloudBees allows organizations to install Role-Based Access Control within Jenkins. This keeps users from accidentally or intentionally accessing repositories that they shouldn't be allowed to interact with. CloudBees also provides "folders" to segregate specific job executions onto specific agents. Lastly CloudBees allows organizations to create pipeline templates and associated plugins for each team. These templates can be as rigid or loose as desired per the organizations security policies.
>
> CloudBees is regularly adding enhancements to further differentiate themselves from Jenkins Open Source and make themselves more appealing to large enterprise requirements.
>
> On top of the above, CloudBees has developed a presentation layer that rides on top of Jenkins for SDLC pipeline, CD monitoring, and metric tracking called DevOptics.

---
Tags: jenkins, cicd
---


# Question
Title: Is there a way to set build parameters outside of the pipeline directive?
My build parameters are defined in my jenkinsfiles

All builds run on container slaves on google kubernetes engine cluster

This means users need to wait for a container to start before anything happens, including just updating the "Build with parameters" choices.

Can I set parameters outside of the pipeline directive so it doesnt need to start a container to update the parameters?

or even better maybe there is a way to have Jenkins run functions in my jenkinsfile to get the updated parameters when it scans branches?

# Answer
EDIT: It might have a chance for me to misread your question at the first time, so let me add another one.

If you meant by Jenkinsfile as you have a manual parameter input in Jenkinsfile for Multibranch or Github Org pipeline, then probably bad idea.

Those kind of jobs should run with predefined parameters or take them on-the-fly like manipulating some environment variables without user input.

You can simply make another pipeline job for manual trigger if the predefined parameters can't cover all the case you need.

And for run function, it looks like `build job` is what you are looking for.

https://jenkins.io/doc/pipeline/steps/pipeline-build-step/#-build-%20build%20a%20job

```
build job: 'Another job in same Jenkins server', parameters: [string(name: 'param1', value: ${new_value})]

```

## --------------------------------------

Are you sure the parameter input is waiting for slave containers? or do you use some special choice plugins for build parameter?

I think a job is even not starting until a user fills out the parameters and hit the build button in Jenkins UI - so no container required until then. We had same k8s slaves, but never had that issue.

(But some parameter plugins might need some pre process..)
> 1 votes

---
Tags: jenkins, jenkins-pipeline, jenkins-plugins, jenkinsfile
---


# Question
Title: How to avoid a high number of requests for avoiding Amazon AWS EC2 instance AutoScalin?
If I set autoscaling for an EC2 instance in Amazon Web Services, how can I set optimized autoscaling for large numbers of connections in order to avoid unusual autoscaling?

# Answer
> 1 votes
First of all, the number of connections is not a common metric to use for autoscaling. Much more common is CPU utilization or possibly the amount of network traffic.

If you are happy with the performance of your application at a certain utilization level, you can modify the corresponding alarms configured in the auto-scaling group, so that autoscaling happens only at higher levels of your metric.

Secondly, you can limit auto-scaling to a certain number of instances. When this number is reached, no further scaling will occur.

Finally, if your traffic has a significant proportion of GET requests, it may be easiest to ensure caching headers are set appropriately and traffic is routed through a CloudFront distribution before it reaches the load balancer. This will likely eliminate most of the GET requests from the load balancer altogether.

---
Tags: amazon-web-services, amazon-ec2
---


title: MSSQL and AD in a Docker Container

text:
# Question
Title: MSSQL and AD in a Docker Container
I am attempting to use Active Directory authentication with SQL Server on Linux running inside the official MSFT Docker container for MSSQL (https://hub.docker.com/\_/microsoft-mssql-server)

I am in the midst of working through this tutorial, but I heard from a coworker that it wouldn't work in a container. He did not say why. He did say which particular step he was doubtful about ('Join SQL Server host to AD domain')

https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-active-directory-authentication?view=sql-server-2017

Is there any reason why I can't complete the 'Join SQL Server host to AD domain' step if I'm using containers?

# Answer
> 7 votes
You absolutely can do this. However, you will need to make sure that the following is happening:

* Setup the Docker host on the MSSQL computer in such a way that it can reach the AD server. Using a bridge network may help (especially if you are using Docker for Windows). If using Docker Toolkit you may need to look into a solution like port forwarding.
* Commit your configuration changes to a new Docker image. This will allow you to move the MSSQL container to a new computer or even run it with Kubernetes/Orchestration tooling.
* Map your MSSQL container to a Volume for persistence. Otherwise the data in the container will be eliminated between container restarts.

# Answer
> 0 votes
As per the attached document, I see no issue with executing this task. The one thing that you need to do update your container with a configuration that is needed to reach AD server.

That means those changes can be done in live SQL server container and committed or add to a Dockerfile and build.

---
Tags: docker, sql
---


title: ECS auto deploy with ECR

text:
# Question
Title: ECS auto deploy with ECR
I'm using github, jenkins, aws ECR, aws ECS.

I want to deploy automatically when github have new commit. When github have new commit, github sends a event webhook to jenkins, jenkins builds the images and pushes to ECR with tag 'latest'.

I wonder how can I make my ECS service restart task and redeploy images automatically when the ECR image is changed?

# Answer
I don't think you can trigger re-deployment based on ECS image being updated. You can however do it as part of your *Jenkins pipeline* \- after you push to ECR as a **next step in the pipeline** can run a script that will reload the task.

In fact in my pipelines I deploy the *ECS Tasks* through *CloudFormation templates* and the CloudFormation stack update is part of the pipeline. That way I update the ECS Task definition automatically every time I push a new image to non-prod ECR.

Hope that helps :)
> 3 votes

# Answer
There's a good article about enabling auto-deployment everytime an image is pushed to ECR here: https://medium.com/@YadavPrakshi/automate-zero-downtime-deployment-with-amazon-ecs-and-lambda-c4e49953273d

Check the following part out:

# Automate Deployment whenever Docker Image gets uploaded on ECR

EC2 Container does not write logs directly to CloudWatch but CloudTrail records API activity for ECS. The flow of automation will be like this: Whenever a Upload occurs to ECR, CloudTrail records PutImage event and can write it to CloudWatch Logs. An Alarm can be triggered whenever a PutImage event is written in CloudWatch Logs which can further trigger a Lambda Function through SNS. Lambda Function Contains the code to Create Task Definition Revision and Update Service with latest Task Definition.
> 1 votes

---
Tags: docker, amazon-web-services, amazon-ecs
---


# Question
Title: What software ihandles shipping to ElasticSearch the standard output of a .NET Core web application running in a container?
I have a ballpark idea of how the ELK stack works. I read that by default, standard output is captured and that output will make its way to the Elasticsearch repository. If I post something to stdout, that content piece will be when visualized with Kibana in one field called "message".

So, besides the title which was anyway getting too long, here's the additional ask. How do I send a structured message, JSON I guess? How do I configure it? I read that it is possible, but I have some difficulties piecing it together. There's something about a JSON template that needs to be registered somewhere, but I lack a clear example.

# Answer
**How do I send a structured message?**

If you want structured messages check out some of the recommended third party logging options. Some of these options have built-in support for Elasticsearch logging. For instance, Serilog provides a "sink" for putting logs directly into Elasticsearch.

**How do I configure it in code?**

These options all tie into the ILogger interface, so testing different logging frameworks should require very little code changes. However, the initial entry point of the application (e.g. Program.cs) will need to be configured to use the 3rd party logger. The official .net core docs go over this in detail.

**How do I configure it for Elasticsearch?**

This will depend on the logger. The appsettings.json will most likely be where you configure the .net core application's connection to Elasticsearch.
> 1 votes

---
Tags: logging, elastic, json
---


# Question
Title: Always execute post statement in a jenkins pipeline when a conditional statement is met
Although it seems counterintuitive, I would like to execute an always statement when a conditional is met. In my pipeline, my post statement should always destroy my AWS CloudFormation Stack if a variable is set to true. Below is the post statement which is not correct.

```
post{
    always {
        when {
            expression { DeleteStack == 'true'}
        }
        stage("Delete All Stacks") {
            steps {
                script {
                    withAWS(region:'us-west-2', credentials:'jenkins-deploy') {
                        def outputs = cfnDelete(
                            stack:"${CFTBatchStackName}",
                            pollInterval:1000)
                    }
                    withAWS(region:'us-west-2', credentials:'jenkins-deploy') {
                        def outputs = cfnDelete(
                            stack:"${CFTVPCStackName}",
                            pollInterval:1000)
                    }
                    withAWS(region:'us-west-2', credentials:'jenkins-deploy') {
                    def outputs = cfnDelete(
                        stack:"${CFTSelfDeleteName}",
                        pollInterval:1000)
                    }
                }
            }
        }
    }
}

```

Can I achieve this using a declarative pipeline?

# Answer
According to the documentation, the `when` directive is only allowed inside a `stage`. You have to use the `script` directive if you want to use conditional statements:

```
post {
  always {
    script {
      if (DeleteStack == 'true') {
        // Destroy what you want !
      }
    }
  }
}

```
> 2 votes

# Answer
I'm not aware of any way to do what you want with Declarative, but it certainly is possible with Scripted:

```
try {
  // your Pipeline code here
} finally {
  if (!DeleteStack) { return }

  [
    CFTBatchStackName,
    CFTVPCStackName,
    CFTSelfDeleteName,
  ].each { stack->
    withAWS(region:'us-west-2', credentials:'jenkins-deploy') {
      def outputs = cfnDelete(
        stack: stack,
        pollInterval: 1000,
      )
    }
  }
}

```
> 0 votes

---
Tags: jenkins, amazon-web-services, jenkins-pipeline, aws-cloudformation
---


# Question
Title: Jenkins pipeline running using containers on an orchestration platform
I have a specific implementation of jenkins on docker.

My requirement is to start slave containers on demand and only when they are needed and then stop them after the pipeline has ran.

I have the jenkins master instantiated as a container, then using the docker socket I am able to run my pipeline in one or more other containers containers provisioned by jenkins, that I have specifically built for that purpose following the implementation described in the jenkins book.

The above mentioned implementation is implemented on one EC2 machine acting as a docker host.

However, that implementation has the flaw that is not highly available and I am only able to use the resources available on on server.

How can I implement this on a swarm cluster where I still have just one master jenkins container starting slave containers from the pipeline on a swarm cluster?

I cannot seem to find something similar after extensive searching, but I am sure it is in some way possible.

# Answer
You can run your application on a particular node of the cluster by defining it in the application level. For this, you need to add the "placement " parameter in the yaml file and specify the node's name to it, then it will deploy the container on that node.

Secondly, what type of scheduling have you implemented on Swarm CLuster? Let me clear your thought, there three of them: Spread, Bin-pack, and Random. By default, it is Spread, which identifies a number of containers on each node and deploys the new container on the node that has a small number of containers. Check the rest yourself.

Both of these points will solve your problem.
> 1 votes

# Answer
You can use the docker-swarm-plugin.
> 0 votes

---
Tags: docker, jenkins, kubernetes
---


# Question
Title: Kubernets docker registry behind nginx-ingress
I try to setup `docker-registry` in `Kubernetes` cluster behind the `nginx-ingress` controller. The issue is, when I try to push the image into private registry, it tells me:

> Get https://registry.local/v2/: x509: certificate is valid for ingress.local, not registry.local

When I `curl` it, I get response from ingress' backend - 404.

Here's ingress' manifest:

```
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
  name: docker-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  tls:
  - hosts: [ 'registry.local' ]
  - secretName: registry
  rules:
  - host: registry.local
    http:
      paths:
      - backend:
          serviceName: docker-registry
          servicePort: 5000
        path: /

```

And here's docker-registry manifest:

```
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  labels:
    app: docker-registry
  name: docker-registry
spec:
  replicas: 1
  selector:
    matchLabels:
      app: docker-registry
  template:
    metadata:
      labels:
        app: docker-registry
    spec:
      containers:
      - command:
        - /bin/registry
        - serve
        - /etc/docker/registry/config.yml
        env:
        - name: REGISTRY_HTTP_ADDR
          value: 0.0.0.0:5000
        - name: REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY
          value: /var/lib/registry
        - name: REGISTRY_HTTP_TLS_CERTIFICATE
          value: /certs/ca.crt
        - name: REGISTRY_HTTP_TLS_KEY
          value: /certs/domain.key
        image: registry:2.6.2
        imagePullPolicy: IfNotPresent
        name: docker-registry
        ports:
        - containerPort: 5000
          name: http
          protocol: TCP
        volumeMounts:
        - mountPath: /var/lib/registry
          name: image-store
        - mountPath: /certs
          name: certs
      volumes:
      - name: image-store
        emptyDir: {}
      - name: certs
        configMap:
          name: certs
---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: docker-registry
  name: docker-registry
spec:
  ports:
  - name: http
    port: 5000
    protocol: TCP
    targetPort: 5000
  selector:
    app: docker-registry
  type: ClusterIP

```

---

Separetly, nginx-ingress and docker-registry, work fine, but don't together.

Nothing valuabe from the ingress controller logs can be found.

# Answer
First of all 404 error is resource error in kubernetes. It means your container in the pod is not created by your first script. Even if it is created and you can see it then your labeling is wrong in the second script.

I would suggest you check matchlabels of both YAML scripts. So in your case I can see that matchlabel is docker-registry in the spec section of the first script, but I don't see any docker-registry in the second script. I believe that in your metadata part of ingress script should have "name = docker-registry" instead of "name=docker-ingress".
> 2 votes

---
Tags: kubernetes, ssl, ingress, docker-registry
---


# Question
Title: Terraform | After upgrading to 0.12 receive a lots of error "unsupported value for "%s" at 0: string required."
After upgrading to 0.12 I receive a lots of error like:

```
Error: Error in function call

  on main.tf line 136, in data "template_file" "inventory":
 136:       formatlist(
 137:
 138:
 139:
 140:
    |----------------
    | module.mongodb.instance_names is tuple with 1 element
    | module.mongodb.private_ip is list of string with 1 element

Call to function "formatlist" failed: error on format iteration 0: unsupported
value for "%s" at 0: string required.

```

Here is from my main.ft:

```
connection_strings_mongodb_standalone = join(
      "\n",
      formatlist(
        "%s ansible_host=%s ansible_user=ubuntu",
        module.mongodb.instance_names,
        module.mongodb.private_ip,
      ),

```

And module:

```
module "mongodb" {
  source                      = "../modules/mongodb"
  instance_count              = "1"
  environment                 = var.env
  instance_type               = "i3.large"
  subnet_id                   = element(tolist(data.aws_subnet_ids.private.ids), 0)
  associate_public_ip_address = false
  vpc_id                      = data.aws_vpc.vpc.id
  tags = merge(
    {
      "Environment" = var.env
    },
    var.default_tags,
  )
  iam_instance_profile = var.iam_instance_profile
  route53_zone_id      = data.aws_route53_zone.zone.zone_id
  ami                  = var.ami
}

output "mongodb_private_ip" {
  value = module.mongodb.private_ip
}

output "tags" {
  value = module.mongodb.instance_names
}

```

from ../modules/mongodb output.tf:

```
output "instance_names" {
  value = [
    "${data.template_file.instance_tags_name.*.rendered}",
  ]
}

```

As I correct understend need to set type of output value, but can't find how, hope someone can help me. Thanks in advance.

# Answer
There is a clue in your error message, but it's only helpful if you know what you're looking for:

```
    | module.mongodb.instance_names is tuple with 1 element
    | module.mongodb.private_ip is list of string with 1 element

```

Your declaration of the `instance_names` output includes this expression:

```
["${data.template_file.instance_tags_name.*.rendered}"]

```

The `data.template_file.instance_tags_name.*.rendered` part of this returns a list of strings, and then the `[ ... ]` around it then wraps an extra list (or rather, tuple) around it, creating a single-element tuple containing a list of strings. That's why the tuple contains only one element, rather than having an element for each of your `count` instances.

This situation is what the upgrade guide discusses under Referring to List Variables, and that documentation includes some more background information on why this previously worked in 0.11 and why the change is required now.

To fix this, you should refer to just the splat expression alone, producing the flat list of strings that your `formatlist` call is expecting:

```
output "instance_names" {
  value = data.template_file.instance_tags_name.*.rendered
}

```

The automatic configuration upgrade tool described in the upgrade guide can make this change automatically in some cases, so I'd recommend using that as a starting point for your 0.12 upgrade process. It isn't able to fix everything, but it will do a lot of the straightforward rewriting work for you and will emit warnings about some more complex situations it's unable to handle.
> 3 votes

---
Tags: terraform
---


# Question
Title: Problem with Vagrant : Nothing happens on vagrant up command
**Versions**
Vagrant: 1.9.5
Provider: Virtualbox 5.2.30
Guest: precise32
Host: windows 7 -32bit, build:7600
Powershell: 2.0

**Vagrant destroy & up output**
nothing happens here...just an empty directory " .vagrant " is being created

**Expected behavior**
virtualbox should be picked up by vagrant

**Actual behavior**
Cursor blinks ..that's it nothing happens on -\> vagrant up and on -\> vagrant global-status -\>it shows "There are no active Vagrant environments on this computer! "

**Possible reasons for this behavior**
I think there are some version dependecies between vagrant, virtualbox & powershell. I don't know I'm just guessing.

**NOTE**
I have read numerous official docs and similar previous issue, nothing worked. when vagrant up command is issued, it does nothing, the cursor keeps blinking forever.

**NOTE**
I know this is dumb question, I can use latest releases but is there anything I can do with this setup

# Answer
> 2 votes
If it's an Intel CPU, you may have to enable Intel VT-x in the bios. Sometimes the virtual machine will tell you this, and sometimes not. Here's a link that might help (ignore the Windows 10 part since you're on Win7). Also getting everything to the newest version would be a good idea if you can.

---
Tags: vagrant, powershell, virtualbox
---


title: Restore Github organization account

text:
# Question
Title: Restore Github organization account
Organization's Github account was deleted but there is backup of all the repos on hard disk. So will creating new repo of same names and then pushing the code restore the code base ?
(The account was deleted more than 2 months ago.)

# Answer
That should work. If preserving the history is not important, collaborators can just re-clone the repo "fresh" from the new URL.

If you want to preserve the history you can use the `git remote set-url` command from inside a current repository clone. That will switch the remote repository name from the old (deleted) account to the new one.
> 1 votes

---
Tags: github, organization
---


# Question
Title: How to create ALB that can handle traffic on 2 ports
I want my HTTP traffic on port 3701 and my WS (websocket) traffic on 3702. For a moment I thought that using 2 target groups was the answer, but there doesn't seem to be a way to differentiate between WS and HTTPS traffic using an ALB.

So perhaps using a NLB (network lb) will allow me to differentiate between the two? How can I have one load balancer forward traffic to 2 different ports?

# Answer
Add listeners to the ALB

1 listener for 3701 1 listener for 3702

update the ALB action for each Listener to point to the the correct Target Group
> 4 votes

---
Tags: amazon-web-services
---


# Question
Title: How to execute a Jenkins job until the build is green
We have got test automation in Jenkins, and we would like to execute it at midnight.

But sometimes it fails, and sometimes it passes. Our idea is to execute the daily builds from 1:00 AM until it is green. How we can execute a job until it is green?

For example:

* Execute job at 1:00
* Execute job at 2:00 if the previous is green, etc.

We can run it hourly, but how is it possible to insert the condition?

# Answer
> 6 votes
This pipeline will run each day at 1 AM. It'll restart if the build fails.

```
pipeline {
    agent any

    triggers {
        cron('0 1 * * *')
    }

    stages {
        stage ('Build') {
            when {
                expression {
                    // When last build has failed
                    !hudson.model.Result.SUCCESS.equals(currentBuild.rawBuild.getPreviousBuild()?.getResult()) == true
                }
            }

            steps {
                sh "./myjob.sh"
            }

            post {
                failure {
                    // If the current job has failed, trigger it without
                    // waiting.
                    build job: "${JOB_NAME}", wait: false
                }
            }
        }
    }
}

```

If you want to wait before to re-trigger the job, you can add the option `quietPeriod: (seconds)` to the `build` step (https://jenkins.io/doc/pipeline/steps/pipeline-build-step/).

You might have to add these methods to the scriptApproval:

# Answer
> 2 votes
First of all, a job should always be green on the first run in my opinion. I know that it can be very difficult, but every issue should be solved. That having said, we have also some jobs that get green after 5 retries. It are selenium tests that are very brittle and we do not get the time to fix them. Once a `retry` was added the builds are green for a couple of months now:

https://jenkins.io/doc/book/pipeline/syntax/

> retry
>
> ```
> On failure, retry the entire Pipeline the specified number of times. For
> example: options { retry(3) }
>
> ```

I prefer a `retry` as it will only restart the job if there is a failure over `cron` as it does not check previous failures.

# Answer
> 1 votes
You can use Naginator plugin in Jenkins to retry the build. This plugin has multiple options.

Step 1: Install the Naginator plugin via the Plugin Manager.

Step 2: In the post-build action of the job, the new option appears as `Retry after the build failure`

Step 3: Configure `Retry after the build failure` action. Example below

Step 4: If a job fails, you can identify whether successfully triggered or not it the build history

---
Tags: jenkins, builds
---


# Question
Title: How to run WSGI Python server without Apache or NGINX
Is there a way to run a Python server that uses WSGI bindings directly? I am trying to go with the one process per container philosophy which will naturally get rid of Apache/NGINX from the container and just run a python process. Is there a way to run python server directly with WSGI bindings without using Apache or NGINX?

https://www.quora.com/unanswered/Is-there-a-way-to-run-a-Python-WSGI-server-directly-without-Apache-or-NGINX

# Answer
This question doesn't really make sense - WSGI is the mechanism that a web server such as Apache would use to invoke an application or script, such as something written in Python. If you have no web server, then what would be using WSGI in the first place?

Is the question really, how do I take a pre-existing WSGI application and turn it into a standalone? Flask is one such framework that can run as WSGI or using its own internal web server.

But, taking a step back, I still think this is the wrong question. There is no need to blindly apply the "one process per container philosophy" - which is really just some random person's opinion, at the end of the day - if it doesn't make sense in your use case. Think about it for a second. A container containing one process is just... a process. We already had those. Containers work best for bundling things together that logically belong together and administering them as a single "thing". What's inside them should be completely opaque to who- or whatever is managing them. You will never see developers jumping through hoops to follow a "one shared object per executable philosophy" because that doesn't make sense either!

If you have something that already works well with Apache/Nginx fronting your application, then containerize it that way, it will be fine.
> 1 votes

---
Tags: python, nginx
---


# Question
Title: Error passing arguments to ansible playbook
I am trying to create a playboook that executes a python3 script with 2 parameters, name and password in localhost. However, when i try to execute the paybook with this command `ansible -b --become-user=ras createdeploy.yaml --extra-vars "user=ras pass=ras"` I get this error message:

```
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

ERROR! Invalid variable name in vars specified for Play: 'pass' is not a valid variable name

The error appears to have been in '/home/ras/createdeploy.yaml': line 3, column 5, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  vars:
    user: "{{ user }}"
    ^ here
We could be wrong, but this one looks like it might be an issue with
missing quotes.  Always quote template expression brackets when they
start a value. For instance:

    with_items:
      - {{ foo }}

Should be written as:

    with_items:
      - "{{ foo }}"

```

Here you are the .yaml file:

```
- hosts: all
  vars:
    user: "{{ user }}"
    pass: "{{ pass }}"

  tasks:
    - name: Stop services
      shell: "bash stopservices.sh"

    - name: Edit real hosts file
      shell: "bash editrealmachinehosts.sh"

    - name: Create dirs and cloning
      command: python3 user pass

```

# Answer
I'd suggest that you first put some default values within the playbook (if you explicitly want your variables there, for the purpose of .. reducing interactivity/command line args lets say):

```
- hosts: all
  vars:
    user: greatuser
    passa: greatpassword

```

Second, I'm pretty sure that `pass` is a reserved word somewhere, although I'm failing to find it in the documentation. That's why I've changed your variable to `passa`. The final version of your playbook should look like this:

```
- hosts: all
  vars:
     user: greatuser
     passa: greatpassword

  tasks:
     - name: Stop services
       shell: "bash stopservices.sh"

     - name: Edit real hosts file
       shell: "bash editrealmachinehosts.sh"

     - name: Create dirs and cloning
       command: python3 {{ user }} {{ passa }}

```

Note the way used to reference to the username and password variables in the third task - `{{ variable }}` instead of `variable`.

Execute your playbook with the following command:

```
ansible-playbook createdeploy.yaml --extra-vars "user=ras passa=ras" -b --become-user=ras

```

As your target host is actually `localhost` (if I did understood you correctly), and since you're not showing your inventory file, I'd like to suggest that you might have to add `-c local` to the list of arguments of your `ansible-playbook` command.
> 3 votes

---
Tags: ansible, configuration, yaml
---


# Question
Title: How to get Docker Container stats using DataDog API?
I have a running DataDog-Agent on a Docker server.

Is there a function in the DataDog API to get the stats of every single container, so I could print the stats to an external Webpage?

Or is there an alternative?

# Answer
> 1 votes
I'm not sure I understand what you're trying to do, but .. DataDog has a Docker Intergration, which when configured provides a status of every single container on a host. Then, by simply querying the DataDog metrics, you can see if a particular container is up or not, or how many containers you have available per image, or what's the CPU consumption etc.

---
Tags: docker, monitoring, api
---


# Question
Title: Triggering a SNS notification when an object is accessed (downloaded) in AWS S3
I have managed to trigger events on my bucket when objects are uploaded (`PUT`) but I am surprised that there are no triggers called when objects are downloaded (`GETs`), as can be seen in this screenshot of the interface:

I want to be notified of each download so I will be able to limit the downloads of any given object to a maximum of 10 times.

How to do that?

I am aiming at setting a tag named for example `limit_reached`, value: `true` on the file object if the download limit is reached and returning `403` by conditional access policy based on the `limit_reached` tag.

Alternative to all this, is to generate signed url's by my web application and track how many signed urls have been generated for a given object and denying any further after generating signed urls 10 times.

Your ideas are highly appreciated.

# Answer
> 2 votes
As you point out, AWS S3 currently does not have an event type for GET calls.

To do something similar to what you are trying to do, I had to do the following:

1. Create an application/lamdba that's only purpose was to retrieve items from the S3 Bucket. This application was given exclusive privileges to the S3 bucket.
2. The application keeps a counter of the number of times that S3 resource has been accessed.
3. The application returns the 403 or whatever you want if you exceed the download limit.

Basically create a "middleware" that's sole purpose is to go from your website/application to S3 and track the number of requests.

While the above solution meets your requirements, it is vulnerable to common attacks such as DDoS or people spoofing their IP. If you take this approach consider additional security measures (e.g. only authenticated users can invoke the application's GET call to S3).

---
Tags: amazon-web-services, amazon-s3
---


title: How to get started with CloudFormation?

text:
# Question
Title: How to get started with CloudFormation?
There is an official website and there are multiple ways to learn CloudFormation. Just curious what worked best for you.

# Answer
> 3 votes
Linux Academy has many great courses on CloudFormation as well as other AWS and DevOps tools and services. They also have many guided labs where you will be given hands on guided learning of how to use various technologies.

Otherwise, creating templates and launching CloudFormation stacks is something that you could learn by just creating infrastructure piece by piece in your CloudFormation template (ex. add an S3 bucket to your CFT -\> launch stack -\> add and EC2 instance -\> launch stack -\> etc. etc.). The AWS documentation is very helpful along with a good plugin for your IDE.

---
Tags: aws-cloudformation
---


# Question
Title: Is there any way of getting code coverage with molecule?
I'm currently working on testing Ansible roles with Molecule. I have one role with 2 scenarios that are executed in Jenkins but just before that, those are submitted to a SonarQube analysis. So I installed the YAML plugin for it to recognize the files, but now, I am asked to add code coverage to these reports.

I'm wondering if there's a way of having code coverage using Molecule with its default stack (Docker/Testinfra)?

# Answer
You can use another verifier. Inspec should have a reporter that will report in the correct format. https://molecule.readthedocs.io/en/stable/configuration.html#inspec
> -1 votes

---
Tags: ansible, code-coverage, sonarqube, molecule
---


# Question
Title: Deploy Vm Vmware with simple Ansible playbook
I try to test one simple playbook with ansible. My goal is to create a new VM with it.

```
---
# create a new VM from a template

- name: VM from template
  hosts: localhost
  gather_facts: false
  connection: local
  vars:
    vcenter_hostname: vcenter-app
    vcenter_user: john@doe
    vcenter_pass: blabla6
    esxhost: esx-4.cbalo.fr
    datastore: VM-PROD-02-NORMAL
    vmtemplate: Centos7-template
    name: "newvm2"
    notes: Ansible Test
    dumpfacts: False
  tasks:
    - name: Create VM from template
      vmware_guest:
        validate_certs: False
        hostname: "{{ vcenter_hostname }}"
        username: "{{ vcenter_user }}"
        password: "{{ vcenter_pass }}"
        esxi_hostname: "{{ esxhost }}"
        datacenter: CD06
        folder: Test
        name: "{{ name }}"
        template: "{{ vmtemplate }}"
        disk:
          - size_gb: "{{ disk_size | default(17) }}"
            type: thin
            datastore: "{{ datastore }}"
        hardware:
          memory_mb: "{{ vm_memory | default(1024) }}"
        wait_for_ip_address: True
        state: present
      register: newvm2

```

The connection to my VMcenter is fine but i see that error message :

```
**ansible-playbook test_vmware.yml**

PLAY [Create VM from template] ******************************************************************************************************************************************************

TASK [Clone VM from template] *******************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Failed to create a virtual machine : Cannot complete the operation because the file or folder /vmfs/volumes/5d25e76b-5475ac54-9cdb-e4434b69d662/newvm2/newvm2.vmdk already exists"}

```

Even if i change the name **newvm2** for an other one, i still have the same message. Someone can maybe help me with that case ?

Many Thanks

# Answer
https://github.com/ansible/ansible/issues/55551

The fix is included in Ansible 2.8.3 or higher. I had the 2.8.2 version
> 1 votes

---
Tags: ansible, vmware
---


# Question
Title: ansible error 'first argument must be string or compiled pattern'
I have this code in my playbook:

```
- hosts: standby
  remote_user: root
  tasks:
  - name: replace hostname in config
    replace:
      path: /opt/agentd.conf
      regexp: #\s+Hostname\=
      replace: Hostname={{hname}}
      backup: yes
  - name: add database array in files
    lineinfile:
      path: /opt/zabbix_agent/share/scripts/{{ item }}
      line: 'DBNAME_ARRAY=( {{dbname}} )'
      insertafter: DB2PATH=/home/db2inst1/sqllib/bin/db2
      backup: yes
  with_items:
   - Connections
   - HadrAndLog
   - Memory
   - Regular
 - name: restart service
   shell: /etc/init.d/agent restart
   register: command_output
   become: yes
   become_user: root
   tags: restart
 - debug: msg="{{command_output.stdout_lines}}"
 tags: set_config_st

```

it will replace `# Hostname=` in a config file with `Hostname= givenhostname` and add an array in 4 scripts. array is the name of given database. then it will restart the agent to apply the changes.
when i run this command:

```
ansible-playbook -i /Ansible/inventory/hostfile /Ansible/provision/nconf.yml --tags set_config_st --extra-vars "hname=fazi dbname=fazidb"

```

i get this error:

`first argument must be string or compiled pattern`

i searched a bit but couldn't find the reason. what should i do?

# Answer
The task which fails is this one:

```
 - name: replace hostname in config
   replace:
     path: /opt/agentd.conf
     regexp: #\s+Hostname\=
     replace: Hostname={{hname}}
     backup: yes

```

Try like this:

```
 - name: replace hostname in config
   replace:
     path: /opt/agentd.conf
     regexp: '#\s+Hostname\=.*$'
     replace: Hostname={{hname}}
     backup: yes

```

Note that if you don't add `.*$`, the task will only replace `# Hostname=` and you will end up with a `Hostname=oldhostnamenewhostname`.
> 1 votes

---
Tags: ansible
---


# Question
Title: How to build Java project on existing virtual machine in Azure?
In my project we are using Azure DevOps in which we have a pipeline which

* pulls code from code repository
* downloads Ubuntu container
* installs Maven
* downloads all the dependencies
* then tries to build code

With this approach, every time we commit code in the repository, new container gets downloaded and Maven dependencies are downloaded as well.

Downloading dependencies itself is taking 10+ minutes which is kind of frustrating.

Is there any way in which we can cut down this build time? We have couple of virtual machines in Azure, is it possible to tell Azure DevOps pipeline to use these virtual machines to build the code instead of using containers?

I am sorry if this question is too broad/generic but I am new to DevOps and Azure in general so I don't know specific terms from DevOps world.

Thanks, Chandra

# Answer
> 1 votes
You can use a custom build container image for your pipeline with Maven and all of your other dependencies. You will need to:

1. Create a Dockerfile that installs Maven and all other dependencies that you have
2. Build the image and upload it to a custom Azure Container Registry
3. Modify your YAML pipeline file to reference the new build image

Sample:

```
resources:
   containers:
     - container: build_container
       image: demoxyz.azurecr.io/pipeline-build-image:latest
       endpoint: AzureCR
       options: '-v /usr/bin/docker:/usr/bin/docker -v /var/run/docker.sock:/var/run/docker.sock'

```

Here is an article that discusses the topic.

---
Tags: git, azure-devops, cicd, maven, java
---


# Question
Title: Is there something like Jenkins pipeline scripts for Bamboo?
In Jenkins I setup my tasks using groovy pipline scripts stored in or .

This way I only need to setup the input of the required parameters. The processing (additional checkouts, installation, builds, tests and other ansible tasks) are defined in the pipline script.

My question is: Is there a way to do somtehing similar with bamboo?

# Answer
I don't think Bamboo has that natively but there are plugins such as this one: (https://marketplace.atlassian.com/plugins/com.cobalt.cdpipeline.cdpipeline/server/overview).

Atlassian also has a different product called Bitbucket pipeline (https://bitbucket.org/product/features/pipelines) that may be more similar to what you want but it's more of a CircleCI type of Pipeline.
> 2 votes

# Answer
You can have Bitbucket pipeline if you are using Bitbucket Cloud https://bitbucket.org/product/features/pipelines

Or Bamboo Specs https://confluence.atlassian.com/bameap/bamboo-specs-883185038.html if you are hosting your own Bamboo Server.

I'm looking into Bamboo specs right now and looks good and easy to implement.
> 0 votes

# Answer
As of Bamboo 6.3+ there is now support for YAML specs and configuration as code. If you are using Bitbucket cloud then this functionality is built into Bitbucket Pipelines.
> 0 votes

---
Tags: jenkins, jenkins-pipeline, bamboo
---


# Question
Title: How do I access the web UI for Kubernetes clustered that were created via EKS?
I need a URL and the web UI log in credentials for Kubernetes. I created Kubernetes clusters in AWS with EKS. I have API endpoint URLs for the clusters based on what I see in the EKS section of the web UI of AWS. But when I go to them in a web browser, there is no prompt for a username or password.

I ran this command: "kubectl config view | grep http"

But when I go to those URLs I see "403 Forbidden" errors.

I expect a URL that would allow me to log in for the new Kubernetes clusters that I created. What should I do to determine the URLs? Hopefully the default credentials will be ascertainable.

# Answer
With EKS, the Kubernetes UI need to be deploy manually. You have a fully detailed documentation on how to deploy Kubernetes UI in EKS.
> 1 votes

---
Tags: amazon-web-services, kubernetes
---


# Question
Title: Looking for good books on Immutable Infrastructure
I am looking for the best books on Immutable Infrastructure. I am interested in books on theory and application. Thank you.

# Answer
Although I am willing to bet that this answer may not age well, I would suggest

"Infrastructure as Code" by Kief Morris for the "theory" part.

For the "application" part, I would suggest "Terraform up and Running" by By Yevgeniy Brikman.
> 2 votes

---
Tags: immutable-servers
---


# Question
Title: How to manage Hashicorp Vault patching in AWS?
I'm setting up a Hashicorp Vault open source deployment in AWS, but I'm trying to figure out the best approach for keeping the underlying EC2 instances patched. I'm using AWS Systems Manager for patching, but won't that cause the patched instances to restart, and then the Vault instances will need to be unsealed every time?

One workaround for this would be using the AWS KMS driven unseal approach. Anyone have any experience with this?

# Answer
The canon solution is to use KMS auto-unseal: https://learn.hashicorp.com/vault/operations/ops-autounseal-aws-kms
> 0 votes

---
Tags: amazon-web-services, amazon-ec2, hashicorp-vault, secret-management
---


# Question
Title: initContainer with huge size software like magento2
As a practice I have pods with these 2 containers:

1. php-fpm
2. nginx

The image of my project is called testimage. The solution uses the 'testimage' as initContainer and then copies the all folders and files of the testimage into a shared volume called volume1. Then this volume is mounted into nginx and php-fpm containers with a Kubernetes volume mount.

When the size of testimage goes beyond 1gb (for example a magento2 project with full pre-built static data and compiled code), it takes a long time for copying these files and it seems like a bad practice.

**Should I put the project files into the php-fpm container (first container)?** For nginx to use it again, I have to copy paste data into a Kubernetes volume and then mount it to nginx.

The end solution is putting all app code + fpm + nginx into a single container than this makes a huge image with all of these. There is no per-container resource management and etc available with this solution.

# Answer
Static assets should not be in the image, but rather on a PersistentVolume or better yet, on an object store such as AWS S3 then let a CDN to use it.

At the end of the day, Magento is a CMS, so the admin users will change the assets and other part of the content. It is not practical to bake assets these in to an container image doe the nature Magento.
> 1 votes

# Answer
Based on the description of your problem, it looks like the solution would be to optimize your testcontainer image:

1. Double check your base for the testimage. Using something like Alpine for your base image can save a lot of space.
2. Check what you are baking into your testimage. Your "pre-built static data" should not be included as part of the image (as others have pointed out). If you need this data to compile the initial image, you should purge it after to get the size down.
3. Any tooling installed that is needed for building testimage can be purged from the container before pushing the image.

With a 5MB Alpine base image and proper management of your "pre-built static data" you should be well below 1GB. A base php-fpm image clocks in around 82MB and nginx at 9MB.
> 0 votes

---
Tags: docker, kubernetes
---


title: Create multiple S3 buckets at once

text:
# Question
Title: Create multiple S3 buckets at once
I am trying to create a static website using S3 buckets. I want to create a bucket for www and non-www versions. I can't work out how to create two buckets at once. I read that you can chain two entries together using square brackets. And I read that you can use variables in the json file.

So far I haven't managed to get variables or multiple entries to work. Can anyone suggest what I am doing wrong?

I have used this code:

**Main.tf**

```
provider "aws" {
  shared_credentials_file = "C:/Users/Eoin.Oliver/.aws/terraform.tfvars"
  profile                 = "customprofile"
  access_key = "${var.access_key}"
  secret_key = "${var.secret_key}"
  region     = "${var.region}"
}

resource "aws_s3_bucket" "b" {
  count      = "${length(var.domain_name)}"
  bucket     = "${var.domain_name[count.index]}"
  acl        = "public-read"
  policy     = "${file("bucket-policy.json")}"

  website {
    index_document = "index.html"
    error_document = "error.html"

    routing_rules = <<EOF
EOF
  }
}

```

**Variables.tf**

```
variable "access_key" {}
variable "secret_key" {}
variable "region" {
    default = "eu-west-2"
}
variable "domain_name" {}

```

**terraform.tvars**

```
domain_name             = ["website.com", "www.website.com"]
region                  = "eu-west-2"
access_key              = "xxxxxxxxx"
secret_key              = "xxxxxxxxxxx"

```

**bucket-policy.json**

```
{
    "Version": "2012-10-17",
     "Statement": [
        {
             "Effect": "Allow",
            "Principal": {
                "AWS": ["arn:aws:iam::240479180938:user/${iams_user_name}"]
            },
            "Action": "s3:PutBucketPolicy",
            "Resource": "arn:aws:s3:::${domain_name}"
        }
    ]
}

```

# Answer
> 1 votes
You can make changes in `Main.tf` as specified below:

```
resource "aws_s3_bucket" "b" {
  count      = "${length(var.domain_name)}"
  bucket     = "${element(var.domain_name, count.index)}"
  acl        = "public-read"
  policy     = "${file("bucket-policy.json")}"

  website {
    index_document = "index.html"
    error_document = "error.html"

    routing_rules = <<EOF
EOF
  }
}

```

---
Tags: terraform
---


# Question
Title: Does the Chef bash resource create history entries?
I know that using a password in the command line is a bad idea because bash commands can be recorded in history.

If I run a command in the chef bash resource though, when I look up the history I don't see any entries.

Does this mean that the Chef bash resource does not record it's history? Is it safe to use a password in the Chef bash resource?

# Answer
> 1 votes
In this case, the best way would be to use chef `data bags` which is specifically known for this. A great documentation is given at https://docs.chef.io/data\_bags.html

Also, As Roland pointed above in the comment, make sure to use `sensitive true` in your bash resource to avoid any such information printed on screen or going to the logs.

---
Tags: security, chef
---