# Question
Title: How to performance test AWS Lambda functions?
AWS Lambda cost is dependent on the time a function runs, and to some extent on memory footprint. Having functions that finish faster, and take up less memory can save quite a lot of money. Especially when such a function is executed often.

**How can a Node.js Lambda function be tuned for speed and small memory footprint to save on cost?**

Are there other aspects for Lambda that are advantageous to improve?

# Answer
Dawny33 response is good, but I would start earlier in the development process.

Keeping an eye on your cloud environment to make sure that your functions behaves as you expect (including your "production" functions, which might operate on a different data set) is crucial, because it might reveal things that are impossible to reproduce locally or with a test data-set.

Nonetheless, I would say that this performance testing that you do in an optimization purpose should start straight from the developer's machine. Or, at least, from some local environment before pushing to the cloud.

The reason I say so is that while AWS Lambdas are amazing on many points, the fact that you don't have a full control on the server will limit your instrumentation abilities. I'm not saying instrumentation is impossible when in serverless, but try figuring out how many CPU interrupts you have (and how many are caused by *your* code) just for fun ;)

So what I advise, and this is actually not limited to serverless, is to start the profiling early. NodeJS profiling can be made with many different tools, NewRelic, dynatrace and AppDynamic are some of the big players. There is also smaller player, some of them are just a NPM package to install (like Nodefly). It is also possible to do some NodeJS without any additional tool, as there is a profiler built into the V8 engine. This documentation from NodeJS will get you started.

Whatever tool you choose, you want to install it locally and collect profiling data. That might involve running an agent or including a package in your package.json. Your tool's instructions will tell you how to install it. A good profiler will let you know how much memory and CPU you use. Better tools will give you insight on how many remote calls were made, how long they took.

Use the profiling data that the tool gives you to identify bottlenecks and address them. There's no limit on how much profiling you can make. Some people (crazy?) will look at the system calls of their most critical function. You might have to do that kind of thing if you want to shave nanoseconds of your function (but then, maybe AWS Lambda isn't the best choice to start with).

It's also worth noting at this point that I haven't mentioned anything specific to AWS Lambda. That's because your optimizations will most likely not be AWS Lambda specific (after all, in serverless you shouldn't be worried about the server/environment).

Make sure that not only your code works, but that it works the way you expect it to. Don't over optimize, but keep a critical eye on CPU and memory usage. Should a 2MB array really grow to 10MB when you sort it? Probably not.

Then you will be able to use tools mentioned by Dawny33, or some other tools, to confirm that your functions perform similarly when deployed to Lambda. You will however already have a very high level of confidence in your function and will only need to validate that they behave properly, not fully profile them.
> 7 votes

# Answer
I might not be able to answer it completely, but here are my views from my limited experience with lambda:

1. **Speed-Performance:** The duration of a single run of a lambda function will be present in the dashboard of the Lambda's `monitoring` tab, which looks something like this:

>

So do throttles/errors/number of invocations.

2. **One can also set up CloudWatch filters for the memory usage** of a lambda function, whose data can be used to further optimize your handler. This is a decent post on how to set one up.
3. Also, **set up proper logging**, so that you can later go through the logs in CloudWatch. This isn't an optimization hack per se, but more of a best practise
4. **Testing**: Test your lambda functions thoroughly, so that you don't miss out on edge cases. This is important, as lambda retries failed functions before giving up. So, you can save up on time forehand if you tested the function properly. Guide to testing Lambda functions
> 5 votes

# Answer
Check out the IOpipe profiler to get full v8 profiler dumps from Lambda invocations. You can load those into Chrome Devtools to see exactly where CPU time is spent, and how memory is being used for your functions.

Disclaimer: I work for IOpipe
> 5 votes

# Answer
Use StackImpact agent to get CPU and memory profiles. More details in the blog post: AWS Lambda CPU and Memory Profiling (Node.js). Because Lambda Node.js process is freezing between requests, most of the other tools will probably not work out of the box.

Disclaimer: I work for StackImpact
> 0 votes

---
Tags: aws-lambda, performance
---


# Question
Title: How can integration tests for micro services be written?
Writing unit and integration tests for individual micro services is part of standard development processes. However, how can these services be tested with one another? And then integrated into a CI system? As well as respecting the correct ports, shared network and secrets. Should the individual micro services be versioned in the same Git repo? For context, I use docker compose with a Flask app that acts as the oauth2 server for a Grafana instance. Other instances include PostgreSQL, Redis, nginx and InfluxDB.

A simple approach would be to test the individual APIs, but what if they change on one side but not updated on the other? How can it be made sure that the APIs are being used correctly in the first place?

# Answer
> 1 votes
Well, the name "integration" test *means* to test different pieces of the system together, I would not call a test that only uses one component an "integration" test.

You'd wire them up like in production (in a testing environment, of course), provide test data and let them do their thing. The components themselves do not need to know - they should not know - that they are under test.

The "wiring up" should be as similar to the production environment as possible, re-using configuration code where possible, to test those bits&pieces as well. Don't put them in the same repo just because you do integration tests.

To see whether the components work together correctly, you would have a test driver that exercises several of them together. It will set up the test data and look at the end results. If everything is green, then the APIs have, by definition, been used correctly. This means that there should be plenty of integration tests - if in doubt, I'd err on the side of having too many integration tests rather than too many unit tests. Obviously this depends a lot on what you are actually doing...

---
Tags: docker, continuous-integration, deployment, containers, configuration-management
---


title: SaltStack Reactor fails with exception

text:
# Question
Title: SaltStack Reactor fails with exception
I have the next conf files for salt reactor:

1. reactor.conf

> * 'salt/netapi/hook/cd':
>     + /srv/reactor/dev.sls

2. dev.sls

> ```
> # vim: sts=2 ts=2 sw=2 et ai
>
> {% set payload = data.get('post') %}
>
> {%      set master = [grains.fqdn] %}
>
> core|api|{{ master }}|jenkins-cd:
>   runner.state.orchestrate:
>     - tgt: {{ master }}
>     - mods: myreactor.deploy
>     - pillar:
>         temp_var_transmit: {{ payload }}
>
> ```

3. deploy.sls

> ```
> {%      set payload = salt.pillar.get('temp_var_transmit') %}
> {%      set master = [grains.fqdn] %}
> my_state:
>   salt.function:
>     - name: cmd.run
>     - tgt: {{ master }}
>     - arg:
>       - "salt-run state.orch my-reactor.cd_state pillar='{\"temp_var_transmit\":{\"host\":\"{{payload['host']}}\",\"artifact_id\":\"{{payload['artifact_id']}}\",\"artifact_version\":\"{{payload['artifact_version']}}\"}}'"
> {% endfor %}
>
> ```
>
> 4. cd\_state.sls
>
> ```
> {% set data = salt.pillar.get('temp_var_transmit') %}
> my_deploy:
>   salt.state:
>     - tgt: {{ member }}
>     - expr_form: compound
>     - sls: '{{ data.artifact_id }}_{{ data.artifact_version }}'
>     - queue: true
>     - test: {{ dry_run_mode }}
>
> ```

When I send API request and run salt-runner.events, I get the next error:

```
jid": "20180119065728472630",
    "return": "Exception occurred in runner state.orchestrate: Traceback (most recent call last):\n  File \"/usr/lib/python2.7/dist-packages/salt/client/mixins.py\", line 395, in _low\n    data['return'] = self.functions[fun](*args, **kwargs)\nTypeError: orchestrate() argument after ** must be a mapping, not list\n",
    "success": false,
    "user": "Reactor"

```

Any thoughts?

# Answer
I was able to find the root of issue. The problem was in the configuration of salt master. After every change of reactor config, I must run `salt-call state.highstate` command on master to sync reactor source code changes with it. I missed this step and thus my reactor config differed from my original source code in IDE. So once I ran mentioned command, everything became in sync. Thanks everybody for help!

P.S. regarding mentioned error, it occurred because I used kwarg in dev.sls for pillar and represented pillar as list:

```
>     kwarg:
>      - pillar:
>          temp_var_transmit: {{ payload }}

```

Kwarg can not contain a list in this case. It may contain key:value format only. So the correct way is either use without kwarg as I did or remove hyphen near pillar.
> 1 votes

---
Tags: configuration-management, salt
---


# Question
Title: How can secrets be distributed in a docker swarm?
Currently I use env and sql files so that docker compose can load secrets from there and build sql database docker instances with specific user/password. The question is, how can this be ported to docker swarm? Docker secrets seem to only provide secret files over encrypted channels, but not set environment variables or be able to configure docker builds.

# Answer
> 2 votes
> Once the secret reaches a manager node, it gets saved to the internal Raft store, which uses NACL’s Salsa20Poly1305 with a 256-bit key to ensure no data is ever written to disk unencrypted. **Writing to the internal store gives secrets the same high availability guarantees that the the rest of the swarm management data gets.**

https://blog.docker.com/2017/02/docker-secrets-management/

# Answer
> 1 votes
Extending Peter Muryshkin’s answer with an significant restriction. Distributing secrets has to be adapted to a file-based approach.

> Note: Docker secrets do not set environment variables directly. This was a conscious decision, because environment variables can unintentionally be leaked between containers (for instance, if you use --link).

Some images already support this such as Wordpress and MySQL while others have to be updated to use this approach.

---
Tags: docker-compose, docker-swarm
---


# Question
Title: The right way to Ansiblically deploy environments directly from Github:
Can one deploy an environment for some Linux OS (say, Ubuntu) from a **raw** Github **playbook** in a way similar to this (I wonder if it's the correct syntax)?

```
bash <(curl -s https://raw.githubusercontent.com/user/repo/master/nginx-cms-addons.yml | tr -d '\r' | ansible-playbook)

```

<sub>**Edit: This is my own Github account so there isn't a risk.**</sub>

In use a similar pattern to **download and execute** bash scripts but I want to know if the principle is identical for Ansible playbooks.

If it matters, the environment I want to deploy directly from the Github raw `yaml` configures Ubuntu a bit (firewall), installs some trivial software like `zip`, a server environment (Nginx/Postfix) and that's basically it.

My purpose to execute my playbook with `ansible` as a substitute for the long Bash script I currently use, made of this code, basically:

```
#!/bin/bash
ufw --force enable && ufw allow 22,25,80,443,9000/tcp
apt-get update -y && add-apt-repository ppa:certbot/certbot -y && apt-get update -y
apt-get upgrade zip unzip tree unattended-upgrades sshguard nginx python-certbot-nginx mysql-server php-fpm php-mysql php-mbstring php-mcrypt -y
DEBIAN_FRONTEND=noninteractive apt-get upgrade postfix -y
sed -i "s/post_max_size = .M/post_max_size = 250M/ ; s/upload_max_filesize = .M/upload_max_filesize = 250M/" /etc/php/*/fpm/php.ini
sed -i "s/;cgi.fix_pathinfo=1/cgi.fix_pathinfo=0/" /etc/php/*/fpm/php.ini
# sed -i "s/# gzip_/gzip_/g" /etc/nginx/nginx.conf
/etc/init.d/php*-fpm restart && systemctl restart nginx.service

```

# Answer
> 3 votes
There is no "right" or "wrong". If the commands work when you type them in, then they work, and we won't keep you from doing it.

Everything else is opinion. Sure, there are some best practices, for example some people find it unwise to directly fetch scripts from a public (3rd party) website and execute those locally without having a look inside first.

It goes without saying that your approach is in fact a rather significant risk. You are effectively giving root access on the target machine (and any other machine your local user has ssh keys for...) to anyone who can push into your repository. How earnest that risk is is something only you can decide.

---
Tags: ansible, github, shell-script, bash, curl
---


# Question
Title: Is JS an reasonable choice for a DevOps internal tool (i.e. cloud-formation runner, scripts)?
I'm working on a project to move our current stack from a mix of half-manual, half-undocumented, half-ansible into a solid code-defined architecture on AWS. Our plan is to use AWS CloudFormation heavily and off-load the following tasks to an internal tool:

* Pipelining (running CFs / scripts in order)
* Scripting tasks (that cannot be accomplished through CloudFormation)
* Templating (if/as necessary)
* Internal Validation

This tool would be expected to build our entire environment more or less from scratch (bar a few things which can't be automated).

Our product (at least on the backend) is built entirely in Javascript, and while it isn't my first language of choice I'm leaning towards writing this tool in Javascript. My other option would be Python as I favor that language more. However I see major value in being able to stick with a single language (libraries, tooling, paradigms, etc.)

* Is JS an acceptable solution for this type of tool?
* Would it raise major eyebrows if we were to bring on a DevOps Engineer in the future?
* Is it unusual to use JS in the DevOps community?

We already leverage JS (via https://serverless.com/) for some of our automation.

# Answer
> 3 votes
Definitely. Especially when using AWS services, the `aws-sdk` for JavaScript is superb.

Writing code using JavaScript in a shop that has both frontend people writing JS in some framework (Angular/React/Vue/...) and backend people using Node.js removes barriers. Just pushing to use JS provides the org. with even more developers who don't have barriers to reading and fixing infrastructure code.

For example, I am quite proficient in using Python, Ruby, Shell, JavaScript (even Perl). But when it comes to fixing or adding something to HashiCorp's tools like Terraform, I just don't have the time to learn Go sufficiently enough - so I don't do it. If a tool like Terraform was written using JavaScript, I would have added my contribution for that missing service a long time ago.

JavaScript support is excellent in AWS Lambda as well, and that is quite a big deal. For example, one of the ways to make CloudFormation do things you never thought it could is to write an `AWS::CloudFormation::CustomResource` implemented in a Lambda function.

Another good example is streaming CloudWatch Logs into ElasticSearch. If you ever try to press that button you discover that AWS just creates a JavaScript function for you in Lambda.

JavaScript is everywhere in DevOps and AWS, doing the same thing with Java, or C# or even Go would take much more time.

Should you bring a "DevOps Engineer" in the future (please don't), and that person does not know JavaScript while all your code is the backend and frontend is in JavaScript then simply don't hire such a person. It is a bad fit for both you and them.

---
Tags: amazon-web-services
---


# Question
Title: Where should I put my application configuration?
I've been reading lately a debate about "*Where should properties that depend on the environment be stored?*".

The classical way is to have multiple property files, one by environment, and based on an environment variable (DEV, PROD...), you choose where to read them when starting the application (like with Spring profiles).

On the other hand, if you are using a container to deploy your application, it's said that this kind of configuration should come from the environment itself (using environment variables that the application reads), so the image doesn't change between environments.

What are the pros and the cons of each approach? Is there a "best" approach for the container scenario?

# Answer
Who said that properties files and environment variables where mutually exclusive?

There is a distinction to be made between "where do I **store** my app configuration?" And "where does my app **source** it's configuration? "

The most likely outcome is that everyone probably *should* just keep doing what they are doing with configuration files as a **storage** mechanism (think long term, persistent state for as long as the environment exists).

However, rather than dropping that configuration file into the application context and letting it run the application should be able to just expect those variables to already be available in the environment when it starts up.

This means you need to have two deployment work flows -

1. I deploy may application into an environment by going through X change control process and doing Y reviews with Z tool, whatever.
2. I deploy my environment configuration into an environment by going through A change control process and doing B reviews with C tool, same process, different outcome.

To use an example of managing environment variables as Key Value pairs in a tool like consul, if you are storing configuration files in git then tools like git2consul with handle getting that configuration into the environment when it's updated.

If you have an app that is expecting that there will be config available as a configuration file then you can avoid shipping multiple copies of the configuration file with the app by building a deploy process with something like consul-template which has the capability to turn your consul values back into a file.
> 6 votes

# Answer
 The way we do it is we have 3 pieces (or artifacts) for every running application.

1. The application we are developing. This is the same regardless of environment. To match your example, that will be the Spring application as a jar/war.
2. The container that will run the application. This is the same regardless of environment. If using Spring Boot, you don't need Tomcat anymore and just the Java runtime. So use the openjdk Docker container.
3. The configuration that the application needs. This is the only thing that is different across environments. In a Spring app, you'll likely be using a properties file.

The configuration file lives in a separate source control. This used to be Git, but we are now using a SaaS we built called Config, at http://www.configapp.com. Config's core feature is the easy handling of environment specific configuration. To run our application on a new server, we pull the Docker container, the application artifact, and the configuration file for that environment. In the container, we mount the directory where the application and the configuration file is stored, as part of the container run. Our application is the same. Our container/image is the same. Only the configuration file is different.

Regarding configuration file vs environment variables. For the longest time we were using configuration files. When we used PaaS/cloud, we used environment variables. It was extra work if you have a lot of configuration so we ended up using environment variables to determine the correct configuration file. We have one application that turned properties to environment variables, but that's atypical. If we have a company sanctioned centralized configuration server, we use that, otherwise we like the simplicity of configuration files.

So to summarize, we pull app.jar, app.properties, openjdk Docker. Then we run openjdk Docker mounting the location of app.jar and app.properties. The only thing environment specific is app.properties. To easily manage app.properties, regardless of how many property keys, environments, cluster/region instances, we use Config.
> 0 votes

---
Tags: docker, containers, configuration
---


# Question
Title: What is the difference between Tasksel and CMTs like Ansible/Salt/Puppet/Chef?
What is the difference between Tasksel and Configuration Management Tools (CMTs) like Ansible/Salt/Puppet/Chef?

I'm not sure Tasksel should be considered a "CMT". Maybe "Package Installation Automizer" or similar **term** is the correct one.

The purpose of this question is to find out the "software type" of Tasksel so I could search data on similar software (given Tasksel currently doesn't support LEMP, I'd like to look for a similar tool that might does indeed support them).

# Answer
> 0 votes
From ubuntu documentation:

> Tasksel is a Debian/Ubuntu tool that installs multiple related packages as a co-ordinated "task" onto your system.
>
> This function is similar to that of meta-packages, and, in fact, most of the tasks available from tasksel are also available as meta-packages from the Ubuntu package managers (such as Synaptic Package Manager or KPackageKit).

So in brief, tasksel is just a helper to select and install a bunch of package with one command (acting as a **meta-package**).

Ansible/Chef/Puppet/Salt are configuration managers, they are able to install packages but that's only a small part of their goal, they aim at converging the system into the state you describe in their respective DSL (Domain specific Language), being package to be installed as well as the content of services configuration files and even up to cloning a git/svn repository somewhere and running install/configuration/database migration tasks for an application.

Building a LEMP stack is just installing nginx, mariadb and php on a linux system, configuring it is another step but that's a common enough task such that there's already existing tutorials about it for every CMT.

Quick list taken from the first page of a google search with 'product nginx mariadb php':

---
Tags: terminology
---


# Question
Title: What's difference between on-premise and SaaS?
I just curious what's difference between on-premise and SaaS?

# Answer
* SaaS (Software as a Service), often referred to "on-demand Software", is a model where you're provided with access to the software in the cloud. There's no installation, setup or running of the setup, as the service provider will do this for you. You just have to pay the service provider. *Example: Office 365*
* On-Premise is the "traditional" model where you have to install, setup and run the software at a computer at your physical location. It takes time, personnel and possible equipment. It may also have additional cost to hardware and software. *Example: Office 2016*

Office 365 is SaaS which provides an online version of the MS Office suite. While Office 2016 is On-Premise Software that will require installation and running.

In addition

* PaaS (Platform as a Service) provides a platform which typical includes operation system, databases, web server, containers for executing programming languages. *Examples: Windows Azure, Azure SQL, Heroku*
* IaaS (Infrastructure as a Service), provides as name suggesting infrastructure, like virtual machines, storage, network, network service (firewalls). *Examples: Windows Azure, Google Compute Engine*
> 7 votes

# Answer
Both on-premise and SaaS are typically used to define how software is being used:

* **`on-premise`**: the software runs on some hardware which is located in your own house, building, company, etc.
* **`SaaS`** (= short for "**S**oftware **a**s **a** **S**ervice\*"): the software runs somewhere (typically in "the cloud"), where you as a user of the softwre don't really know (care?) where the hardware it's running on is located.

What the purpose of the software is, doesn't really matter. It could by CRM, CMS, BI, Accounting ... even DevOps!

Nor does it matter what the hardware is about, or the OS it's running on: a small server (running some sort of Linux, etc), a huge mainframe running zOS, etc.

### Examples

* In the context of DevOps, let's assume you want to use GIT (and you know what GIT is used for): how are you going to make your GIT repositories available for other users to share your code. Are you going to use infrastructure (eg: some server) that you manage yourself, in your own building (= roll your own cookies), or do you rather want to "outsource" all that (e.g. because you do not have the skills or resources) and use something like GitHub?
* Assume you want to start using "the Cloud" for sharing files, pictures, contact data, etc with other people. Are you going to use any of the available Cloud services (with all sorts of "strings attached" like fees, terms & conditions, etc.)? That would be the Saas version of it. Or do you rather want to be in full controle of it all (the server on which it is all stored/managed) and use something like (open source based) OwnCloud or NextCloud, running on a server in "your" house, building, etc. In this case "you" decide about what the terms and conditions are, and the only kind of fees you'll be facing is the cost to create and operate your own server (and manage the users allowed to access it). This is clearly the on-premise version.

PS: Obviously, both approaches have their pros and cons ... use this link for more details on that.
> 4 votes

---
Tags: terminology, software-as-a-service
---


# Question
Title: Is it possible to securely delete a file from a virtual disk in a cloud provider?
Considering that virtual hard drives are generally formatted using something like `ext3` that abstracts away from the underlying block device, which in and of itself is an abstraction over the top of a physical and network topology.

Is it possible to use something like `wipe` or `shred` against a single file or an entire device to mitigate the risk of data being recovered at a later date if that machine was compromised?

# Answer
> 2 votes
You can't really use those tools with ext3 anyway. Wipe manpage has a large note about it:

> No secure deletion program that does filesystem-level calls can sanitize files on such filesystems, because sensitive data and metadata can be written to the journal, which cannot be readily accessed. Per-file secure deletion is better implemented in the operating system.

Moreover, the fact the data will be written really in-place is not guaranteed and you may be writing aside the original data.

That said the reason to do this goes back to a point where a data was written on disk on consecutive sectors on the same physical medium, usual storage nowadays use striping technology to spread data on multiple physical medium, each write in the system toward this storage will have an impact on all the stripes this file was part of. Each provider will have its own custom storage backend, but I'm pretty confident the same principles are used.

When you get the picture even higher, you're writing a file on a virtual hard disk which is itself a file on a storage pool, probably streched on multiple lun composed of raid array of physical disks.
There's so many IOs running on all those strates of striping/aggregation that the probability someone could recover a file by magnetic reading of physical disks (what wipe and shred should protect against) is very very very low as the storage used for this file has probably been reused a bunch of time already.

Even on existing ext3/ext4 filesytem, tools like extundelete have a hard time recovering any file if the disk is not unmounted/mounted read only very quickly after the deletion.

---
Tags: linux, security, cloud
---


# Question
Title: Enabling CSRF Protection prevents editing/adding parameters to jobs - intentional or bug?
Recently Jenkins has been putting out a number of security warnings related to CSRF Protection. We tried enabling it on our version of Jenkins (ver. 2.89.2) only to find that with it enabled you can no longer add or edit parameters on either freestyle or pipeline jobs via the GUI.

I did fairly extensive testing to narrow it down as the culprit, but I still don't understand why the option exists and is being recommended if it cripples Jenkins jobs.

Is this a feature or a bug? If it is a feature, why? What's to gain?

# Answer
I believe this may be a result of running Jenkins behind a proxy, which can cause legitimate requests to perhaps appear to Jenkins as cross-site requests. From the official wiki:

> If you are using nginx as a reverse proxy in front of Jenkins, you need an extra system property on Jenkins "-Dhudson.security.csrf.requestfield=Jenkins-Crumb". See JENKINS-23793 for more details
> 1 votes

---
Tags: jenkins
---


# Question
Title: Test coverage of changed code in pull requests in git-tfs
We have a lot of legacy code that have poor test coverage. We wish that all new code that is checked in through pull requests has (close to) full test coverage. What is the best approach or tools to achieve this?

We're using git-tfs, .NET C# and TypeScript. And we'd like this to be part of the code review process and prevent developers (or at least give a warning) to check in with insufficient test coverage.

# Answer
IMHO from the workflow prospective code coverage measurements are fundamentally similar to builds, smoketests or any other types of QA checks and can thus be performed in similar manner. I can see several options:

* include them in the PR verification path, just like builds and/or smoketests, similarly positioned relative to the code review stage.
* include them as a step in the pre-commit gating system, if you use one - most effective IMHO, especially for large projects where scalability comes into play (mentioned in my answer to How do you scale your integration testing?).
* include them in the CI pipeline, maybe on a parallel execution path if they're lengthy, to not add too much to the total CI execution time. Very useful to use these runs as up2date baselines for results comparisons for the other 2 cases listed above.

I'm not familiar with MS tools, but if MSTest doesn't offer good integration with the other tools in your workflow you could build custom wrappers to implement a typical submit/execute API with a pass/fail result (or set of results) which most tools can integrate with.
> 1 votes

---
Tags: git, tfs
---


# Question
Title: terraform chef provisioning - cannot validate certificate for 10.90.239.223 because it doesn't contain any IP SANs
While configuring the chef provisioner I am running into the following error after running 'terraform apply'

```
chef_environment.terraform_01: Post https://10.90.239.223/organizations/mykitchen/environments: x509: cannot validate certificate for 10.90.239.223 because it doesn't contain any IP SANs

```

I tried this

```
openssl req -x509 -newkey rsa:4096 -keyout chef.key -out chef.pem -nodes -days 365 -config req.conf

sudo chown opscode:opscode chef.*
sudo mv chef.* /var/opt/opscode/nginx/ca/.

sudo chef-server-ctl stop
sudo chef-server-ctl reconfigure
sudo chef-server-ctl start

```

my req.conf

```
[req]
default_bits = 2048
prompt = no
default_md = sha256
req_extensions = req_ext
distinguished_name = dn

[ dn ]
C=US
ST=mystate
L=mycity
O=DevOps
OU=myname
emailAddress=myemail@mycompany.com
CN = 10.90.239.223

[ req_ext ]
subjectAltName = @alt_names

[alt_names]
IP = 10.90.239.223
DNS.1 = 10.0.90.45
DNS.2 = 10.90.50.8
email = DevOps-3@10.90.239.223
URI = https://10.90.239.223/

```

but I am still getting

```
chef_environment.terraform_01: Post https://10.90.239.223/organizations/mykitchen/environments: x509: cannot validate certificate for 10.90.239.223 because it doesn't contain any                                       IP SANs

```

In my chef provisioner config I also have

```
ssl_verify_mode = ":verify_none"

```

So far I am stumped. What am I missing?

# Answer
Answered on Slack, the likely first issue here is that the output from `openssl req` is not a cert, it's a CSR. But there are some other lingering issues here too. It is recommended to use something like Certstrap to do internal cert generation rather than raw `openssl` commands as the modern standards are quite fiddly.
> 3 votes

---
Tags: chef, terraform, security, ssl
---


title: COPY and ZIP files in Dockerfile

text:
# Question
Title: COPY and ZIP files in Dockerfile
Using a `COPY` statement creates a new image layer, right?

```
COPY files/foo.zip /tmp/

```

Now, to save space I could take the `ADD` statement... for a GZ arhive. But `ADD` does not support automatic extraction of ZIP files.

So the `foo.zip` will be dangling around somewhere in a below layer. Just a `RUN rm` statement does not help.

How do you solve this?

# Answer
Convert the `zip` file into a `gz` file before running `docker build`, possibly in a wrapper script.

Or even go one step further, and extract the archive first, then use COPY to copy the resulting folder contents.

When you are using archives from the internet on the other hand, `ADD http://somewhere/file.gz /data` will extract the downloaded file directly into that folder. If you have a problem with zip/gz format there, then use `RUN curl -o somewhere/file.gz /tmp/file.gz && tar xzf /tmp/file.gz && rm /tmp/file.gz` to only create a single layer without leaving the file behind.
> 7 votes

---
Tags: dockerfile
---


# Question
Title: How can a Sys. Admin skill up to work as a DevOps engineer?
I am looking for recommendations on how to skill up to be a dedicated DevOps Engineer.

I have always wanted to work on Development related projects as I love coding but unfortunately I have never got the chance. I have previously worked on freelance projects using .Net and PHP and I am certified in Java SE but I have never used it.

My work experience incorporates being in system and application administration. Mostly I am a senior now in this field so a switch to Development is not preferred as I would have to start over as a junior. However I think DevOps will be perfect for me but I need to know what I need to study to fill the gaps in my knowledge to be a good dedicated DevOps engineer (preferably the part about writing automation and tools).

Please recommend what should I study, for example I heard Python and Perl and Ruby are very good for DevOps but what else should I start with and what is the mandatory skills that i need to cover first?

**My current skill set:**

Sys. Admin

* Unix (HP-UX, Solaris) and Linux (Ubuntu, Red hat, CentOS) Administration
* Middleware Technologies administration (WebLogic, Apache-Tomcat, WebSphere, MQ)
* Bash/Shell Scripting
* Web server configuration and administration
* Excellent knowledge of common database systems (SQL Server, Oracle, MySQL, etc.)
* Web services and Integrations SOAP/REST/API
* Oracle Business Intelligence Suite (OBIEE, DAC, Informatica)
* ITSM Remedy Development and Administration.
* SAP Business Object (Analytics).
* Splunk
* Service Center –ITIL-enabled IT service Management
* Strong understanding of Web Application technologies.

Developer

* Expert Development skills in Microsoft Visual Basic.
* Excellent Development Skills in Java SE.
* Microsoft .Net Development
* MS SharePoint development.
* Expert in MS Office with Expert knowledge in Macros and Automation across all MS office Applications.
* Intermediate PHP development (CakePHP)

# Answer
> 9 votes
Have a look for DevOps Engineer roles and check out the list of technologies that they are seeking. Typically you'll need automation scripting skills (like Bash, Python or PowerShell), configuration management skills (like Chef, Puppet or Ansible) and some fundamental experience maintaining and configuring a Build/Release pipeline. For this, look to get experience with CI/CD tools like AppVeyor (.NET), Jenkins, Travis CI etc. You should also be familiar with Git and source control systems in general such as BitBucket or TFS. You would also benefit from becoming familiar with automated testing techniques.

If you haven't already, consider hosting some open source products in Github and setting up a pipeline for them that performs automated tests on commits as well as (if possible) publishing to a package management system. This is great experience as well as demonstrates your skills.

For a more general understanding of the philosophy of DevOps (as beware that it's a culture as well as a technology set) do as much reading as you can on the topic and start following (e.g via Twitter) the likes of Gene Kim, Jez Humble, and Patrick Debois. Also I recommend you read The DevOps Handbook which they authored. It is a very thorough explanation of what DevOps is as well as how organisations might adopt it.

You could also check out DevOps related Meet Ups and Conferences (such as Velocity, DevOpsDays and Config Management Camp). The opportunities will of course vary depending on where in the world you are.

# Answer
> 3 votes
Make a list of DevOps tools that seem to be popular, download, follow the manual, and just do it.

* Vagrant
* Ansible
* Docker
* Kubernetes/OpenShift

This will give you a good running start with the tools closer to the "Ops" side, and - if you manage to also pick up *why* and *what for* these tools are used - will make you a worthy member of most teams. You can pick up the more Dev-sided bits and pieces as you go (i.e., CI/CD pipelines etc.).

---
Tags: sysadmin
---


# Question
Title: Scaling replication controllers via API in Kubernetes
Is there a way to scale replication controller via API in kubernetes.

By using kubeclt command, it may be done by running:

```
kubectl scale --replicas=1 --namespace=kube-system rc my_replication_controller

```

But how to achieve it using API?

Moving through docs, I found only "replace scale of the specified Scale" and "partially update scale of the specified Scale" among write operations. But I think this is not what I need.

# Answer
After long playing with it and consulting on slack DevOps chat, I was able to figure it out. Here is a workable curl:

```
curl -sSk -H \"Authorization: Bearer ${KUBE_TOKEN}\" -H 'Content-Type: application/merge-patch+json' -X 'PATCH' $URL_NAME/api/v1/namespaces/kube-system/replicationcontrollers/kube-registry-v0 -d '{"spec": {"replicas":  1}}'

```

One of the important components here is "`Content-Type: application/merge-patch+json`" Without it specification, Kubernetes will not recognize data in API request correctly.
> 3 votes

---
Tags: kubernetes
---


title: Javascript based alternative to Fabric

text:
# Question
Title: Javascript based alternative to Fabric
Is there a similar DevOps tool to fabric with JavaScript as its scripting language? I'm specifically interested in the remote execution side.

Most of the tools I've found rely on python (e.g fabric) or Ruby (e.g. Capistrano, Chef). However, in my team we don't use these languages for anything else. While these languages may be great, they are not as universal as JavaScript in the web development industry.

I mention fabric because it's ideal for what I want except that I have minimal python skills (sadly).

# Answer
> 1 votes
Shipit is what you are looking for.

> Shipit is an automation engine and a deployment tool written for node / iojs.
>
> Shipit was built to be a Capistrano alternative for people who don't know ruby, or who experienced some issues with it. If you want to write tasks in JavaScript and enjoy the node ecosystem, Shipit is also for you.
>
> You can automate anything with Shipit but most of the time you will want to deploy your project using the Shipit deploy task.

# Answer
> 6 votes
Fabric (and Capistrano, presumably the unnamed Ruby tool you came across) are a bit unusual in that they're task runners with extra features for easily running tasks on remote hosts. I'm not aware of any other popular tools that do exactly the same thing, but depending on your needs there are a few other options that may work for you.

Grunt and Gulp are the foremost task runners in the Javascript ecosystem. While not designed specifically to deploy software, they both have plugin-based systems that allow you do this, and you'll find plenty of articles of people using either one to do so. For instance, this tutorial uses the `grunt-ssh` plugin to integrate ssh with grunt, while this one relies on `gulp-rsync`.

Using task runners to deploy code tends to be an approach favored by developers who are wearing an ops hat. When you ask an operations engineer, they're more likely to suggest hacking this behavior into a configuration tool. This can take many forms (for instance, packaging your app into an rpm or deb, then installing it like any other software), but I'm going to focus on what I think would be the simplest and most direct method for you.

Ansible is ostensibly a configuration management tool, but it really shines at **task orchestration**, which I would define as "running a set of actions across a set of servers". The official documentation is a good place to get started, but to give you an idea of what this might look like, you might have a *playbook* (a YAML file that defines a set of tasks) like this:

```
- hosts: webservers
  tasks:
    - name: update code checkout
      git:
        repo: ssh://git@github.com/mylogin/hello.git
        dest: /var/www
        version: master

    - name: restart webserver
      command: restart-command-you-run

```

When you run this file with `ansible-playbook`, it will look up your host list (as simple as an ini file, but can also be a dynamic list written in any programming language) to determine what servers exist and what groups they're in, and then run the defined tasks on the servers that you tell it to.

Now, you specified that you wanted to stay in the Javascript ecosystem. However, while Ansible is a new thing for you, it doesn't have all the complexities of another programming language, since configurations are written in YAML. It will also allow you to do much much more later, and unlike several of its competitors, uses ssh to communicate, so there's very little setup involved (nothing to install on the remote machines).

---
Tags: deployment
---


# Question
Title: From Java Developer to DevOps and will devops die?
Hi all :) I want to switch from java developer to devops because i want to work something more interesting. I know that as a devops i will have to code but beside that i hope i will learn some new things like server administration, CI, CD, networking and a lot of new stuff. I have been asking my self many times but with no answer... will devops die around all this AI stuff. I think that server administration profession can disappear but for devops... i\`m not sure. What do you think ? And is it a good idea to switch professions (hope some of you have done this already) ?

# Answer
> I want to switch from java developer to devops because i want to work something more interesting.

There is no need to *switch*, in IT, generally. You can just add more and more stuff to your toolkit over the years and decades. Just because your new job description is "DevOps" instead of "Java" does not mean that you somehow immediately forget your Java background.

Hunting for something more interesting is always good. It means more learning. You will very likely (hopefully) alternate between hunting for something and deepening that something for your whole life.

> will devops die

Yes, of course it will. *Everything* dies. I have no idea how the IT world will look in 20 years, but one certainty is that it will not look like today.

> around all this AI stuff

Huh? What AI stuff? There is no AI. Machine learning has nothing whatsoever to do with AI, it's just the "next" technique. It's not like Matrix-style AIs take server administration jobs from us, or like AIs build our OpenShift servers.

> I think that server administration profession can disappear

Not for a *long* time. People forget that below the virtualized stuff (VMs, containers, etc.) there still are the same technologies as always, requiring people who know how to manage the basic systems. Sure, it will get different, but not disappear for any time soon.

> I have been asking my self many times

You are overthinking this. Pick a DevOps technique, anyone which seems interesting to you, download some stuff, and just tinker around with it at home for a bit. Or find a nice little project at work. You'll find out in no time whether it is your thing or not. Nothing about DevOps is particularly "special" or "magic" or a totally "different" thing.
> 3 votes

---
Tags: terminology
---


# Question
Title: Automated Docker swarm reinitialization and token distribution
Imagine you want to reset your Docker swarm from time to time.

How do you distribute join tokens to those zillions of nodes? How much automation is possible here?

# Answer
Have the Swarm Manager place the worker join token in a Key/Value store. We use AWS Parameter Store but I believe Consul would be another option (I don't use it).

Each node can then retrieve the token while it is bootstrapping.
> 2 votes

# Answer
Simple answer with infrastructure as code, you don't rejoin nodes, you trash old nodes and bring fresh nodes to join the cluster.
> 1 votes

---
Tags: docker, automation, docker-swarm, distributed-systems
---


# Question
Title: Why can I build this image from Docker for Mac but not on Docker CE for Ubuntu?
Here's my Dockerfile:

```
FROM php:5-fpm

RUN apt-get update
RUN apt-get install -y php5-mysql
RUN apt-get install -y php5-xdebug
RUN apt-get install -y php5-mcrypt
RUN apt-get install -y php5-gd

```

When I run from Docker for Mac everything works great. But from my Linux machine I get:

```
E: Package 'php5-mysql' has no installation candidate
The command '/bin/sh -c apt-get install -y php5-mysql' returned a non-zero code: 100

```

Shouldn't these behave the same?

**UPDATE 1**

Thanks to @caylorme for recommending I check the base image on both hosts. They are indeed different image ids.

On the Linux host:

```
$ sudo docker images php
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
php                 5-fpm               9b179c4d385e        7 days ago          367MB

```

On the Mac host:

```
$ docker images php
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
php                 5-fpm               707631468b8b        11 months ago       362MB

```

But how is that possible? I had assumed (wrongly I guess) that Docker image tags are immutable so you could guarantee reproducibility and rebuild/run on any Docker host. What can I do get the image that's on my Mac host on the Linux host?

# Answer
It is possible that the base image is different on each host. Can you confirm the images for php:5-fpm on each host are identical?

If the image from the Mac host is older, try updating it. You may need to update the package name in the dockerfile.

If you want to preserve the same behavior on your newer host, make sure you are pulling the image with the same hash or version-specific tag. It’s quite possible the newer version of the php image is on a newer version of Ubuntu. The package name may have changed between repos.
> 2 votes

---
Tags: docker, dockerfile
---


# Question
Title: What is the best Windows Infrastructure to run Adobe After Effects like AWS Lambda?
I have an app where we allow users to create videos by choosing an AE template and modifying the text, images, etc. We run AE on a windows server which renders the final video.

For other scenarios where we do not use AE, we use AWS Lambda of which we can spin off 100 instances simultaneously and they all use their own resources. Whereas for AE we use a Windows server and all parallel rendering use the same resources and hence very slow.

Are there any PaaS where I can run AE and spin up multiple instances?

# Answer
> 1 votes
In general a solution runs the application code on top of a provider-supplied sandbox infrastructure which complements your app code so that together they implement the complete app functionality. But this puts specific requirements (often quite restrictive) in front of the application code, for example:

* a limited choice of (typically high-level) programming languages
* certain pieces of mandatory functionality in your app code (like health checks, etc)
* certain limitations of what the app can do (for example not allowing writes to the local filesystem)

So when looking for a solution you need to be looking from both the app perspective and the sandbox perspective.

Now your case brings additional requirements to the game:

* running a 3rd party app which you cannot modify, so you need to find a PaaS offering whose requirements are met with the application as-is, unmodified
* a Windows-based sandbox - rare, most providers use open-source infrastructure, I suspect primarily because of the licensing terms. Your highest chance of finding such offering would be on Azure, but not necessarily the only one
* AE licensing terms - how many AE instances can you run?

Due to the extra requirement on the sandbox a solution would be, IMHO, a better fit for your case (you're actually using it presently, maybe just scaling it would be a much simpler solution for you?).

AFAIK the only offering (outside of Azure, with which I'm not familiar) that comes very close to an but still is, in a way, a is Google App Engine Flexible Environment:

* meeting the app requirements *might* be possible by actually writing an app implementing a rendering service which uses AE under the hood, as a dependency
* it supports (docker-based) custom runtimes, you *might* be able to build one with AE/Windows (maybe how it's done for `ASP.NET` apps would help)

---
Tags: azure, aws-lambda, platform-as-a-service
---


# Question
Title: Multiple runners for all projects OR single runner per project
**What do we have:**
We have a Gitlab EE with multiple projects running on a droplet, next to that we have a runner server for the CI runners. All projects are Docker projects and 4 runners can be busy at the same time.

**What we need to know:**
Now is my question, is it better to have like 4 runners on the runner server for all the projects OR is it better to have 1 runner per project?

I know when the CI gets stuck, the runner will be stuck too and it will mean that it is missing 1 runner till it gets reset. So at that point single runner per projects seems better, but it will make a large list of runners which perhaps use some idling resources from the server ?

*See here nothing is running or doing work:*

Anybody can tell me what is better in your opinion and why it is better in your opinion as both ways seem legit in my opinion and i want to know others experience.

# Answer
After some research i found out that shared runners are good for the projects that are very much the same, and specific runners are better for the projects that are not like any project.

Shared runners can eat all resources available in the end with its queue's which makes specific runners more interesting.

My choice now is more like a little bit of both, so shared for all small and likewise projects, and specific for the large new projects as the documentation recommends.

From Shared vs specific Runners:

> ... The main differences between a shared and a specific Runner are:
>
> * **Shared Runners** are useful for jobs that have similar requirements, between multiple projects. Rather than having multiple Runners idling for many projects, you can have a single or a small number of Runners that handle multiple projects. This makes it easier to maintain and update them. Shared Runners process jobs using a fair usage queue. In contrast to specific Runners that use a FIFO queue, this prevents cases where projects create hundreds of jobs which can lead to eating all available shared Runners resources.
> * **Specific Runners** are useful for jobs that have special requirements or for projects with a specific demand. If a job has certain requirements, you can set up the specific Runner with this in mind, while not having to do this for all Runners. For example, if you want to deploy a certain project, you can setup a specific Runner to have the right credentials for this. The usage of tags may be useful in this case. Specific Runners process jobs using a FIFO queue.
>
> A Runner that is specific only runs for the specified project(s). A shared Runner can run jobs for every project that has enabled the option **Allow shared Runners** under **Settings ➔ CI/CD**.
>
> Projects with high demand of CI activity can also benefit from using specific Runners. By having dedicated Runners you are guaranteed that the Runner is not being held up by another project's jobs.
>
> You can set up a specific Runner to be used by multiple projects. The difference with a shared Runner is that you have to enable each project explicitly for the Runner to be able to run its jobs.
>
> Specific Runners do not get shared with forked projects automatically. A fork does copy the CI settings (jobs, allow shared, etc) of the cloned repository.

Still i would like to know everyone's mindset about this, as im still interested about your way of doing it!
> 4 votes

# Answer
Thanks for your conclusion, it fits in mine too.

In my situation, I have projects of different types like pure Java, .NET and some packaged in Docker. For the moment, the number of jobs to build is not consequent but I'm afraid about some jobs creation peak for specific project before a major release for example...

I only configured shared runner with tags (dotnet, java, docker,...) and the project specifies the correct tag in their .gitlab-ci.yml :

```
job:
 tags:
  - java

```

If I detect/anticipate a peak of job creation, I can assign temporary a runner to a project. Of course the peak detection and the lock/unlock can be automatized with Runner API but it's not my case for the moment.
> 0 votes

---
Tags: continuous-integration, gitlab, docker-in-docker
---


# Question
Title: Is zipping a file supposed to happen in Jenkins or on Nexus?
I am trying to figure out the correct way to do my pipeline. Am I supposed to zip my code in Jenkins with maybe the File Operation plugin and then upload it to my private Nexus repository? I don't send all kinds of files to Nexus and do something there do I?

# Answer
> 4 votes
If you are building these artifacts for a release/deploy, you will want them to be in as ready to release of a package as possible. If that is as a .zip, then yes you should create the .zip on the Jenkins server, then upload to Nexus. Your deploy system will then download, decompress, and manipulate the artifact as seen fit. In general, I would only use the node running Nexus to store artifacts and not execute any "build-like" processes. That is exactly what Jenkins was made for.

---
Tags: jenkins, artifacts, nexus
---


# Question
Title: Creating ansible host_vars and group_vars files dynamically
Is it possible to populate `host_vars` and `group_vars` with results from a database like we do in dynamic inventories?

Actually I'm doing it inside my playbook either with lookups, either by using `shell` module and registering queries result.

If anyone has a more elegant way of populating variables like making an executable `host_var` file printing JSON to stdout or so ..

# Answer
> 2 votes
The most elegant way would be to use dynamic inventory, rather than static inventory and dynamic group variables.

However, if for some reason you don't want to do that, one other option is to use custom facts; these are scripts that live in `/etc/ansible/facts.d` on the client machines. Like dynamic inventory, they be executable scripts (and therefore, query redis or whatever else), but in this case they define variables just for that particular host. This isn't *quite* what you were looking for, since you'd have to query the database on each individual server instead of once for the entire group, but it can have a similar effect.

---
Tags: ansible, automation, databases
---


# Question
Title: Time series data stores in CI context - meaning?
From reading this job ad:

> You believe CI servers, push button deploys, **time series data stores**, metrics dashboards, and centralized logging are not just “nice to haves” they are critical pieces of infrastructure that rapidly pay for themselves.

What does using a time series data store mean for CI?

# Answer
> What does using a time series data store mean for CI?

Nothing in the context of that job description. CI and time series databases are separate entries in the list. To put it another way:

```
infrastructure_pieces = [
    'CI servers',
    'push button deploys',
    'time series data stores',
    'metrics dashboards',
    'centralized logging',
]

for infrastructure_piece in infrastructure_pieces:
    assert(candidate.important_of(infrastructure_piece) == 'CRITICAL')

```

That is, the job involves maintaining CI, but it also involves monitoring production servers and applications using TSDBs, and various other activities.
> 1 votes

# Answer
Since time series datastores are indexed by timestamps they can be very useful in a CI/CD environment, depending on what kind of information they store, for things like:

* incident analysis and management
* service operation and SLAs monitoring
* overall SDLC pipeline performance/efficiency analysis and improvements
* resource capacity planning
* budget and ROI computations
> 1 votes

---
Tags: continuous-integration, terminology
---


# Question
Title: Creating a cron job with Ansible (ERROR! 'cron' is not a valid attribute for a Play)
I am relatively inexperienced when it comes to Ansible however I have previously created playbooks which will create a cron job successfully. For the life of me I cannot work out what is wrong with my .yml file.

I am receiving the below error when attempting to run my playbook locally with:

```
ansible-playbook -i "localhost," -c local crond.yml

ERROR! 'cron' is not a valid attribute for a Play

The error appears to have been in
'/etc/ansible/cds/roles/backups/tasks/crond.yml': line 3, column 3, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

- cron:
  ^ here

```

I have seen similar questions asked on SE/Ansible forums but to no avail. Any help/guidance would be much appreciated.

Here is my crond.yml file:

```
- cron:
    user: "root"
    minute: "30"
    hour: "02"
    job: '{{/usr/bin/rsync -avhz -e "ssh" --rsync-path="sudo rsync" -H --delete --numeric-ids <ipaddress>:"/data/rsnapshot-backups/<ipaddress>" /data/rsnapshot-backup}}'
    state: present

```

# Answer
I got around this by adding my crond.yml task to the global playbook and now call it via tags (ansible-playbook -vvv -i "localhost," -c local site.yml --tags backups).

I added the cron\_file section to define the location of the cron job as /etc/cron.d/rsync.cron rather than the root users crontab. I then added state: present to ensure that the cronjob exists on the server and doesn't delete other entries in the file.

I also updated the job section of the task to remove the '{{ }}' and replace with double quotes surrounding the command as well as '\\'s around any double quoted strings like below:

```
- cron:
    name: "Copy all rsnapshot backups from <hostname>"
    minute: "30"
    hour: "02"
    job: "/usr/bin/rsync -avhz -e \"ssh\" --rsync-path=\"sudo rsync\" -H --delete --numeric-ids <ipaddress>:\"/data/rsnapshot-backups/<ipaddress>\" /data/rsnapshot-backup"
    user: root
    cron_file: rsync.cron
    state: present

```
> 0 votes

# Answer
`crond.yml` in your example is not a playbook. It is a tasks file (list of tasks), while playbook is a list of plays.

A play must have `hosts` directive at minimum. Please see http://docs.ansible.com/ansible/latest/playbooks\_intro.html#hosts-and-users

This is a playbook:

```
- hosts: localhost
  tasks:
    - cron:
        user: "root"
        minute: "30"
        hour: "02"
        job: '/usr/bin/rsync -avhz -e "ssh" --rsync-path="sudo rsync" -H --delete --numeric-ids <ipaddress>:"/data/rsnapshot-backups/<ipaddress>" /data/rsnapshot-backup'
        state: present

```
> 1 votes

---
Tags: ansible, linux
---


# Question
Title: simulate SQL DB usage pattern of a typical cloud native web app with sysbench?
As far as I understand `sysbench` is for bencharking / test max. performance of a database server. I wish to simulate DB usage pattern of a typical cloud native web app (Twelve-Factor App). Is this possible with `sysbench`? If yes how? Or any other Open Source tool you recommend?

I work in DBaaS (MariaDB) so can't say what exact app (shop, historic info, social media, forum). I wish to deploy app on Cloud Foundry.

# Answer
> 2 votes
> ... \[I\] can't say what exact app (shop, **historic info**, social media, forum)

This makes this question very difficult to answer. This is because you have two basic types of queries:

## OLTP and OLAP queries

> **Online transaction processing (OLTP)** is information systems that facilitate and manage transaction-oriented applications, typically for data entry and retrieval transaction processing.1

These kinds of queries tend to be very high volume, but short, and targeted and are a great fit for a system like MariaDB. On the other hand,

> **Online analytical processing, or OLAP**, is an approach to answering multi-dimensional analytical (MDA) queries swiftly in computing. OLAP is part of the broader category of business intelligence, which also encompasses relational database, report writing and data mining. Typical applications of OLAP include business reporting for sales, marketing, management reporting, business process management (BPM), budgeting and forecasting, financial reporting and similar areas, with new applications coming up, such as agriculture.2

These types of queries perform extremely poorly at scale in Maria DB unless it is properly tuned and certain database management and retrieval techniques are used (such as partitioning, for example). These performance issues are largely a result of a confluence of disk I/O issues and the fact that queries are single-threaded in terms of CPU. While sysbench can do a great job of testing the performance of OLTP queries, for OLAP queries, you will need something else. For example, one programmer forked sysbench to add the ability to do OLAP queries.

## Conclusion

Unless we can get a bit more information about how much data you are working with and what kind of I/O operations you will be doing (reporting or transactional) it is hard to say if this specific tool will do the job. Ultimately database management is a complex discipline and there is a reason DBAs are able to make careers out of just database management. I have seen a lot of programmers think that they don't really need to worry about database management and they don't really need to integrate a DBA into their team. As their app scaled, they very quickly found out they were dead wrong. If you are planning a multi-tenancy DBaaS environment, you are going to need to architect your system very carefully in order to prevent one tenant from affecting the performance of the other and consider offering DBA consulting if your customers begin to suffer performance issues - and factor that into the price. If they can't afford their own databases and find that managing that in house is untenable and unaffordable, you will need to provide both the platform ***and*** the expertise as part of your service as one customers performance issue can easily become *all* of your customers' performance issue. That means you are going to really need to know your stuff or make sure you hire people who do. In short, databases are *hard*.

1 https://en.wikipedia.org/wiki/Online\_transaction\_processing

2 https://en.wikipedia.org/wiki/Online\_analytical\_processing

---
Tags: cloud, databases
---


# Question
Title: Traefik configuration issue with local proxy
Edit: I have updated my configuration to reflect Henrik Hoegh's tutorial. I am still running into issues.

I am setting up a local development environment with lots of docker containers running applications. I am using Traefik to manage and locally proxy these containers (each container having a named local URL). Right now my setup is as follows

**Traefik.toml**

```
[web]
  address = ":8088"
  [web.statistics]

################################################################
# Docker configuration backend
################################################################
[docker]

endpoint = "unix:///var/run/docker.sock"
domain = "local.io"
watch = true

# Expose containers by default in traefik
exposedbydefault = true

# Use the IP address from the bound port instead of the inner network one. For specific use-case :)
usebindportip = true

# Use Swarm Mode services as data provider
swarmmode = false

```

Docker-compose.yml file for container

```
...
container_name: mcserviceface
labels:
  - "traefik.port=8080"
...

```

I also have added `local.io` and `mcserviceface.local.io` (both pointing to my local machine) to my `ect/hosts` file.

When I start up the Traefik container and the mcserviceface container, it shows up on the Traefik dashboard, but it does not show the URL as `mcserviceface.local.io` rather something like `http://185.28.0.3:80`. I also hit a `ERR_CONNECTION_TIMED_OUT` when I follow that link.

How do I resolve both of these issues?

Edit: I have made sure that both containers on are on the same docker network via `docker network connect ...`. It does not resolve the issue.

# Answer
Make sure you connect your traefik instance to the client docker-network (as specified on the bottom of this page):

```
When running inside a container, Træfik will need network access through:

docker network connect <network> <traefik-container>

```
> 1 votes

---
Tags: docker, docker-compose
---


# Question
Title: How much it will cost for AWS for deploying a inventory management application
I am developing an inventory management application i want to use AWS EC2 . which will have 10-15 users and storage about 2GB .as a newbie i could not understand the pricing can somebody help me how much it will **cost me per month**

# Answer
With cloud solutions like AWS, this is not so straightforward to figure out like with "classic", always-on servers that wholly belong to you, and where you are paying fixed monthly prices for power, network, backup and rack-space, with maybe a little bonus on-top if you exceed some bandwidth limits.

So, in your case, have a good look at what "sizes" of VMs AWS has to offer. According to Wikipedia, in 2016, prices on AWS ranged from $0.0065/hour to $6.82/hour, depending on performance and other factors, with similar ranges for storage, and then further complicated by other factors of your plan (i.e., 3-years prepaid or whatever).

Secondly, you have to factor in when and how long you actually need your application to be available. If none of your users are working before 7am or after 9pm, then you can have the application running only in this time, not paying anything in the "off" time.

Thirdly, there are even more options. For example, maybe you can use lambdas for some functionality of your application (even more finegrained control over costs) and so on.

Lastly, there are things like spot prices, where you bid for unused resources at drastically lower prices. You might become creative here and run your background batch processes on such instances, for example.

More variations are mentioned in the comments.

So, you see, we cannot really predict what your application will cost. It would certainly be possible to give you a ballpark figure in the case that you would run your application 24/7 just like a classic VM, but that would be rather useless.

You need to sit down, read some more about the AWS pricing structure (they do have an online calculator where you can plop a few values in). As you are not doing a play or for-fun project, it would also be more than worthwhile to actually experiment, get an AWS account, develop a small prototype and do some performance tests so you see what you actually do need.

Check out their calculator at https://aws.amazon.com/tco-calculator/ if you didn't already, but be prepared that their breakdown of on-premise costs is, in my opinion, not very useful (you'd better figure that out yourself, if on-premise is at all an option for you).
> 1 votes

---
Tags: amazon-web-services, amazon-ec2, cloud
---


# Question
Title: How can I build & maintain a personal, cross-platform 'toolkit' of commonly used applications?
I've discovered a number of open source tools over the years which have become an essential part of my personal workflow & productivity in my current day job (project manager / business analyst). I can often move from company-to-company and even account/machine (and OS too) and want to get up and running with the minimum of fuss.

What I'm trying to combine:

* **apps/tools** (which use a variety of frameworks and languages including python, flask, javascript, node, java etc)
* **config files**
* **other misc. files** (e.g. fonts)

**However, by necessity, I am often behind corporate firewalls and restrictive IT policies** which make it time-consuming to get internal license approvals for each tool, download each of the tools individually, keep them up-to-date, blocked external package repos, maintaining & building-upon the associated configuration files over time...

### Docker is the obvious solution here, but...

### ... it is often blocked by these same IT policies, as are its alternatives.

# What I've tried

Using a **GitHub repository**, I've since settled on building/installing everything locally, making it 'portable' (using batch scripts, symlinks and a number of config file tweaks) and then uploading.

Drawbacks of this approach include: the complexity, difficulty maintaining this cross-platform (currently windows-only), working around applications installed via git-clone, working around quirks of 'portable' versions of runtime dependencies like Python, github 50mb single file-size limits...

# Another possible approach I considered

**Using continuous integration** (like Travis-CI and AppVeyor), my main concern here is that I've not actually 'built' some of the apps I use (e.g. emacs-w64, which I download the pre-built binaries for) so wondering if this will just add more complexity than my current solution ... and since Travis-CI doesn't support windows it adds an additional dependency, as well as the overhead of learning the continuous integration world & maintaining build scripts for linux, osx & windows..

I'm not looking to open up discussion here, but an answer that will meet my needs with the minimum of fuss (this is for personal use rather than devops for my company / others after all) from the experts ... even if the answer is simply that I need to put the effort into a CI solution...

# Answer
> 1 votes
For your own personal use, I think a GitHub repository with all of your bootstrap and installation scripts is a great idea. It serves as a portable way to download and distribute any scripts you need where firewalls and restrictions aren't an issue.

As for within your organization, one suggestion that your IT might be open to as a compromise is to create a machine (barebones, VM, etc.) that is solely dedicated to downloading and storing the artifacts that you need. The criteria for this machine would be:

1. It has access to download artifacts (files, apps, etc.) from a source (preferably either the open internet or from white-listed addresses).
2. It is able to store the artifacts in a central location.
3. It is not able to access internal infrastructure
4. It is has auditing to track any artifacts that it downloads/stores

With this setup, you could download something like the latest version of Python onto the server (via logging in with credentials, API, etc.) and have it store Python in a central location for your other servers to download and install. You could store your artifacts in a multitude of ways (artifact repository, cloud storage, file server, etc.), but only that server should be able to upload to your storage. Your IT team could even set alerts when a new artifact is downloaded on this server or uploaded to your storage so that they can inspect it.

I think this approach demonstrates a few things to your IT department:

1. These artifacts are important enough to you and your teams that you are willing to go out of your way to establish a safer and more traceable method to retrieve them.
2. You have given your IT department an automated way to track who and when an artifact was downloaded in the event an artifact is malicious and needs to be deleted and traced to a source (something they probably didn't have in place before).
3. The method you propose not only gives you more accessibility to the tools you need to be productive, but you are proposing ways to help your IT team be more productive by not having to manage all aspects of downloading artifacts.

---
Tags: windows, configuration, methodology
---


# Question
Title: Can Ansible alone be a complete CI-CD tool?
I am new to Ansible and was wondering about the capabilities of Ansible. Can Ansible alone be a complete CI-CD tool?

Scenario that needs to be implemented:

As soon as a developer perform a `git commit`, the build should be generated and placed some central server or maybe Amazon S3. This is a `JAR` file assuming a Java application. One can then download the build and deploy that build to the testing environment and after deployment invoke the automated test cases. Upon successful implementation of the test cases,one should be able to deploy this build onto the production environment in a rolling fashion. Note that one must not deploy this build every time, but it should be a discretionary power. I.e. even if during a day let us say, 5 builds were created and 3 of them passed the test cases, It should not go and deploy the builds on production. I should decide when to deploy the build, this could be a flag that I set to "Go" and then the deployment to production should take place.

Points that I am apprehensive about Ansible's capability are:

1. Will ansible be able to detect the `git commit` event. Is there a way my playbook to constantly poll the GIT server on a regular basis to detect any commits?
2. Is it possible to create Build using just Ansible. (Can I use command module to just execute the build creation process(assuming there is some command to create the build and I was thinking of firing those command using the command module, maybe `maven` or `ant` command).

Kindly forgive me for my ignorance if there is no such thing. I have never done development and hence am complete beginner.
3. How can I invoke automated test cases on the testing environment where the build was deployed (I am assuming a successful deploy on the stage environment).
4. Can I capture the success or failure result? (It maybe be a suite of test cases and won't be an objective o/p as pass or fail.)
5. Is there a way to achieve the discretion based deployment to production systems using Ansible?

Note that I don't want to use Jenkins or any such tools for creation of builds or deployment to test environments .

This may sound like a beginner question on DevOps, but I just wanted to assess the scope of Ansible as a CI tool. Is Jenkins (or any other variant) inevitable for achieving this? Or can people do away without using Jenkins.

Also if someone can comment on what is the recommended way to achieve this pipeline.

# Answer
> 5 votes
This is totally possible. Of course you can achieve it without jenkins or similar tools being required. We're always free to reinvent the wheel. The question becomes is it worth the effort? When it helps you avoid jenkins I'd be inclined to put the effort in. (I'd also suggest looking at Concourse before doing jenkins again, but that's not what this question is about...)

For your case you need two things:

1. a playbook that automates your build using `ansible`.
2. a git hook to invoke your ansible playbook, probably post-commit.

If you are using gitlab or github you should look at their specific documentation for creating hooks, but otherwise the generic help I linked to should do it.

To answer your specific questions:

1. Yes, use the git hook.
2. Sure. You probably want ansible to do a fresh `git checkout` first. Eventually you will want to add any prerequisites to be verified too.
3. You don't have to assume anything, just add these steps to the end of your playbook. If the build fails, it will stop before wasting effort on testing anything. You could split build, deploy, and validate into their own roles within one larger playbook. This will help make each of the sets of tasks easier to manage and you could call them manually if needed for debugging or emergencies.
4. You could do this in a myriad of ways. I'd create a text file with a date/time stamp and the results inside. You could write it to a database or K/V store.
5. I'm not sure what you mean.

# Answer
> 5 votes
Frankly, if you go this route, you do not even need Ansible as a CI/CD driver. Ansible does not bring any infrastructure anyways, it just uses an existing ssh connection, so you can just use said ssh connection directly with your own scripts.

If the ultimate goal is to avoid any of the established solutions (Jenkins, Gitlab CI, whatever), then nothing really keeps you from rolling your own:

* Create some scripts "prepare.sh", "build.sh", "test.sh", "deploy.sh" etc. which do their thing. "prepare.sh" would maybe use Ansible itself to setup a working environment somewhere, do the clone etc.
* Either in a git hook on the server side, or if you cannot install it there, in some permanently running (or cron-started) master script which polls the git repository regularly, implement a detector which looks for new commits and then executes the other scripts in order, until one of them fails or you are done.
* If you need nothing fancy regarding reporting etc., then just redirect the output of all those scripts to some logfile, or log files tagged with the current time.
* It's easy to send a quick mail to the devs when the build or test goes wrong, if you are so inclined.

All in all, this would probably not be *that* much more effort than using Jenkins - with Jenkins, you'd write the content of your scripts somewhere in the job definition, anyways.

Obviously, you do not get all the fluff that Jenkins or other solutions give you - statistics, reports, GUI, load-balancing the worker nodes etc., but as long as you do not need those, why not.

---
Tags: continuous-integration, ansible, continuous-deployment, maven
---


# Question
Title: Git Repository Manager conditionally deny pull request
I am looking to migrate a handful of Subversion repositories to Git. Each of these repositories has a development and production branch. When ready to release, a developer will submit a pull request to production. There is a version file that contains the version that has to be incremented in some way. Is there a Git repository manager (Gitlab, Bitbucket, etc.) that allows you to automatically deny pull requests based on a condition? My condition in this example would be the version in a version file is not incremented. I am also willing to consider other methods, such as tagging the repositories instead of a version file, if auto-denying in that way is possible.

# Answer
> 1 votes
Most repository managers have some way of implementing this.

GitHub (and GitHub Enterprise) has the idea of statuses on a commit; you'd set up a webhook that pings your tool's webserver when a commit is pushed, and then it would use the status API to set whether or not it passes the version check.

Bitbucket, when self-hosted, supports plugins to integrate nicely with it, including ScriptRunner, which can be used to do all sorts of things similar to what you want.

I'm not familiar with GitLab, but it probably has some similar method.

You will need to do some programming to implement the checking logic, but these pieces should allow you to kick that off when appropriate and use the results to block the merge.

---
Tags: git, repository
---


# Question
Title: Jenkins libraries configuration in Jenkinsfile
Right now we have many projects with similar Jenkinsfile's. Instead of *copypasting* we're developing a shared library with some utilities. This library is loaded dynamically in each Jenkinsfile, and has some parameters that repeat among calls. For example, an utility has a custom Slack message posting function, and the **channel**, **bot token**, and some **fixed text** are always the same.

Instead of providing those parameters to every call to this Slack function we have thought of making a config file and loading it globally at the beginning with all these *"constant"* parameters.

We tried putting a `config.json` and using the readJSON step like this:

```
def load(path) {
    env.CONFIGURATION =  readJSON(file: path)
}

```

So the CONFIGURATION is an environment variable and so it's global. The problem is that it seems that environment variables can only be strings, so the JSON is broken (we cannot do `env.CONFIGURATION.slack.channel` for example)

We tried the following also in **Jenkinsfile**:

```
def CONFIG = readJSON(file: path)

```

But CONFIG then is not visible inside the library scripts.

So now, the questions:

1. Is our approach correct? Putting all common config in one file and loading it for the library scripts to see.
2. If it's not correct, how would you deal with our problem? Simply passing the config as parameter on every call and that's all?

Thanks in advance for your help.

# Answer
> 5 votes
You should look at Global Shared Libraries, which will allow you to reuse logic across different Jenkinsfiles.

Rather than depending on the environment, it's probably safer to explicitly pass parameters and returns values from the functions, which will make their behavior more explicit. Another side benefit is that you will be able to unit test them using

# Answer
> 1 votes
I also have the same idea(different config files for different pipeline). I use the pipeline.properties to store my variable.

```
properties = readProperties file: 'pipeline.properties'
echo "Immediate one ${properties.repo}"

```

**Drawbacks:** Due to the groovy early evaluation problems. The value will be null when you use the ${properties.repo} in some shared library closure (eg, agent { label $properties.agentLabel }, the agentLabel will be null).

See: https://stackoverflow.com/questions/46630168/in-a-declarative-jenkins-pipeline-can-i-set-the-agent-label-dynamically

http://jenkins-ci.361315.n4.nabble.com/can-i-use-variable-to-specify-the-agent-label-in-my-declarative-pipeline-td4897177.html

---
Tags: jenkins, jenkinsfile, groovy
---


title: Validating custom Jenkinsfile

text:
# Question
Title: Validating custom Jenkinsfile
Currently we're using Jenkins shared libraries which we're planning to use with custom Jenkinsfiles. Idea here is that developers should be able to manage those custom Jenkinsfiles without any additional help from DevOps team.

In order to implement this feature I need some way how to validate this custom Jenkinsfile.

Even if I have running Jenkins instance with configured shared libraries post to /pipeline-model-converter/validate wont lint my Jenkinsfile correctly as it wont load my custom libraries.

Is there a convenient way how to check custom Jenkinsfile before actually executing it on running Jenkins instance? e.g., checking if method names in Jenkinsfile are correctly written.

# Answer
> 2 votes
As far as I see, there are two possible approaches, either by ~~misusing~~ reusing the Blue Ocean plugin or using the Groovy-based Jenkinsfile unit test tool.

The groovy approach might be a bit of overhead, since it is not a client-side Javascript verification. An idea would be to run an async task, server side that tests the code the developer provides and returns the results to the client. You mentioned that the shared libraries are not being parsed properly. This might be resolvable through the use of mock Jenkins commands and objects. This might also avoid the security risk which is created through shared libraries, which requires users having push rights.

The client-side solution might be to provide a playground via the Blue Ocean plugin or directly present the Jenkinsfile that has been created via a redirect of the Jenkinsfile to Blue Ocean.

All in all, there is no straight forward JS library solution to your problem, as far as I can tell.

---
Tags: testing, jenkinsfile
---


title: Ansible communication medium

text:
# Question
Title: Ansible communication medium
I am a little confused as to what is the protocol that Ansible uses for communication between Controller node and Managed Remote nodes.

I believed that it was "SSH" protocol.

However, the official documentation mentions this "On the managed nodes, you need a way to communicate, which is normally ssh. By default this uses sftp. If that’s not available, you can switch to scp in ansible.cfg"

I am a little confused as I used to believe SSH and SFTP and SCP are completely different protocol. AFAIK, SFTP is used to securely transfer files and SCP is for secure copy. This may be over SSH connection, but why would ansible use SFTP or SCP?

Is it about transferring some files/python modules using these protocols? If yes, what files are getting transferred actually?

Kindly Excuse me if this question sounded silly, nevertheless any elaborate response would be much appreciated.

# Answer
> 3 votes
This is a great question as some of the terminology can be very confusing - particularly surrounding the (unfortunate) use of "SFTP" to describe multiple similar technologies.

## Will the real SFTP please stand up

This comes mainly from the fact, that RFC 913 defines one SFTP protocol, the "Simple File Transfer Protocol" which offers no encryption and went largely unadopted, with users favoring TFTP and FTP depending on their needed application.

## Since the FTP protocol, defined in RFC 1738 in 1985 is completely encrypted and packet sniffers are able to dumping plaintext passwords from this protocol, Some thought that it would be a good idea to secure this protocol by implementing FTP over an SSL connection. In 1997 a mechanism for doing that was released in RFC 2228 which often was referred to as SFTP.

Meanwhile, back at the ranch...

Meanwhile, a similar security issue was occurring around the telnet protocol - passwords and sensitive information were unencrypted and subject to sniffing. In response, SSH-1 was released in 1995 which provided an encrypted equivelent to telnet for use over network connections.

This protocol became very popular very quickly and before a formal specification allowing for FTP over SSH was released, users of the SSH protocol had implemented a version of BSDs rcp (remote copy) over SSH. This protocol was extremely limited however, which led to a revision in SSH - SSH2 which added an SSH File Transfer Protocol (not to be confused with FTP over SSH, *another* SFTP) because, according to wikipedia:

> Compared to the SCP protocol, which only allows file transfers, the SFTP protocol allows for a range of operations on remote files which make it more like a remote file system protocol. An SFTP client's extra capabilities include resuming interrupted transfers, directory listings, and remote file removal.

And In fact, the IETF set up a working group to codify a formal standard for SSH-2 (released as RFC 4251) but

> Eventually, development stalled as some committee members began to view SFTP as a file system protocol, not just a file access or file transfer protocol, which places it beyond the purview of the working group.

And eventually SSHFS was released as just that.

## Conclusion

So the answer is a bit complicated.

SSH and SFTP and SCP *could* be regarded as completely different protocols depending on *which* SFTP you are referring to and which version of the SSH protocol you are referring to. Yet, the command for actually doing a file transfer remained "scp" even after the SFTP and SSH-2 protocol was developed. If one is referring to SSH-2 however, these could arguably all be considered the *same* protocol.

Side note: SSH was released by Tatu Ylönen of the Helsinki University of Technology in 1995. As Linus Torvalds released Linux in 1991 as a student at the University of Helsinki until 1996, it was kismet that SSH would be so popular and widely used in Linux.

# Answer
> 1 votes
Yes, Ansible transfers packaged task file to remote host and executes it there. Still, the process can be different with pipelining mode.

This is how task execution looks like in general (without pipelining):

* Generate Python-file with module and its parameters for remote execution
* Connect via SSH to detect remote user home directory
* Connect via SSH to create temporary work directory
* Connect via SSH to upload Python-file via SFTP
* Connect via SSH to execute Python-file and cleanup temp dir
* Get module's result from SSH standard output

---
Tags: ansible
---


title: Escaping curly braces in Bamboo

text:
# Question
Title: Escaping curly braces in Bamboo
In Atlassian Bamboo, in script tasks you can use special Bamboo environment variables e.g. for current build number

```
${bamboo.buildNumber}

```

Now I have the impression if I have a code piece like `${VAR}` which I want to pass to other context, Bamboo's templating logic will kick in and render this part as an empty string.

Proof:

```
echo "my foo ${VAR}!"

```

Output:

```
01-Feb-2018 20:47:58    my foo !

```

Is there a way to escape curly braces in this context?

What I have tried:

* googling it: nope
* `${{VAR}}` \- bad substution error
* `$\{VAR\}` leads to exactly same output as it escaped which I do not want to have either.

*UPDATE from comments* \- making sure there is no existing variable being just empty.

Input:

```
echo "my foo random ${RANDOM_VAR_ABCFOO}!"
echo "my foo random with prefix ${bamboo.RANDOM_VAR_ABCFOO}!"

```

Output:

*Bamboo 5.13.1*

```
02-Feb-2018 10:26:11    my foo random !
02-Feb-2018 10:26:11    /home/bambooagent/temp/FOO-JOB1-4-ScriptBuildTask-3130782940072218698.sh: 2: /home/bambooagent/temp/FOO-JOB1-4-ScriptBuildTask-3130782940072218698.sh: Bad substitution

```

*Bamboo 6.3.0*

```
02-Feb-2018 10:22:02    my foo random !
02-Feb-2018 10:22:02    /data/bamboo-agent/temp/FOO-JOB1-24-ScriptBuildTask-7913400670965638044.sh: line 2: my foo random with prefix ${bamboo.RANDOM_VAR_ABCFOO}!: bad substitution

```

# Answer
> 1 votes
I don't know the inner of bamboo, but I assume it works like a bash script.

What happens is that variables are replaced before execution, in bash that would be `echo "echo $VAR" > test.sh` and test.sh will only contain "echo" because VAR is replaced by it's value before the command is executed.
If you want test.sh to contain `echo $VAR` you have to tell bash to ignore the replacement on the first call by escaping the $ sign:

`echo "echo \$var" > test.sh` will give `echo $VAR` in the file.

In the same note, the notation `${VAR}` and `$VAR` are the same, using braces is a good practice when you do concatenation like in `echo "Size is ${VAR}Kb"` as without the braces like this `$VARKb` bash would try to find a variable named `VARKb` and return an empty value.

To address PrestonM different behavior, I assume there's either \- the fact running under windows with the powershell interpreter doesn't behave the same \- or just that bamboo has a special case for variables prefixed with `bamboo.` and as such escape them.

If someone want to test you may try:

```
echo "${VAR}"
echo "${bamboo.VAR}"
echo "\${VAR}"

```

and edit the results in this answer.

# Answer
> 0 votes
If your variable is **not** declared in Bamboo (either by injections, build/deploy variables etc.) it will **not** inject the variable and will use the string as is. It should only render the variable as an empty string if that is the value you set for that particular variable. As an example, my build plan has the following variables:

```
Variable Name    Value
version.major    1
version.minor    12

```

In my build plan, I have the following script job:

```
echo "${bamboo.substituteVar}"
echo "${bamboo.version.major}"
echo "${bamboo.version.minor}"

```

In my logs, the output will be:

> C:\Users\TestUser\bamboo-home\xml-data\build-dir\12345678\MANG-TC-TBV\>echo "${bamboo.substituteVar}"
>
> "${bamboo.substituteVar}"
>
> C:\Users\TestUser\bamboo-home\xml-data\build-dir\12345678\MANG-TC-TBV\>echo "1"
>
> "1"
>
> C:\Users\TestUser\bamboo-home\xml-data\build-dir\12345678\MANG-TC-TBV\>echo "12"
>
> "12"

As long as your variables and namespaces are unique and descriptive for the plan you are working on, you should not have any issues with name conflicts. If it is replacing it with an empty string, my guess is that you may have that variable created somewhere in Bamboo, but no value assigned to it.

---
Tags: continuous-integration, bamboo
---


# Question
Title: Strategy for deploying/hosting javascript-based static websites in containers
This comes up from time to time in several of our dev teams, without us having figured out the "right" way:

We use a lot of react-based webapplications that "compile" into static websites that are just a few html, js and css files.

However, the "building" of these apps take a number of variables that enable/disable feature flags, configure backend urls, etc. This means that we cannot "build" a binary in the traditional sense and just apply a config file at deploy-time - the "build" itself needs to have these environment-specific variables set, and thus the only time we can "build" is at deploy-time.

For now we solve this by injecting the required environment variables into the Docker container and run a start cmd along the lines of

```
npm build && nginx run

```

This has a couple of disadvantages:

1. The build process takes a *lot* of cpu/memory relative to the run-time requirements of the container. That means we need to scale the container for the build process instead of the run-time requirements - which feels wrong
2. Build failures are hard to "track". We can use healthchecks in Kubernetes, but if a build takes 2 minutes we still have to wait for 3 mins (1 extra for safety) before we can start testing the container's healthcheck endpoint to see if its alive.
3. Deployments might take a long time: If we configure Kubernetes to do a "serial" deployment, it will start each pod and wait for the "initialDelay" period of 2-3 minutes before starting the next. This means that we're easily looking at a 10-min deploy time if the deployment is scaled to 3-4 pods.

This all feels very suboptimal to me. I'd be very interested in hearing how the community solves the "build at deploy-time" conundrum with modern javascript webapps.

I realize that for Kubernetes we could use "init-containers" that perform the build, places the artiacts in persistent storage and then have the app containers simply pull from persistent storage during startup, but this still feels more like "bypassing" the problem than solving the root issue.

# Answer
From my standpoint the best approach would be to:

1. Separate build process using Jenkins that would build NodeJS project into distribution and wrap it into Docker image
2. Spin up Docker registry that would accumulate Docker images from Jenkins (this registry should be accessible from Kubernetes cluster)
3. Move environment variables to Jenkins secrets or use separate tool to collect and combine configurations from external Git repo (we use Spring Cloud Config through REST API to collect json/yml definitions for every application in every environment)

Using Jenkins you can configure continuous delivery system based on generic Pipelines. Possible flow would be:

1. Developers finish their work in correspondence to GitFlow (latest Pull Request merged into Release branch)
2. Webhook triggers Jenkins pipeline:
    * Stage 1: collect environment definition
    * Stage 2: build NodeJS app using npm
    * Stage 3: build Nginx Docker image with distribution
    * Stage 4: push Docker image to Docker registry
    * Stage 5: deploy/update service in Kubernetes using standard definitions
3. Notifications send in regard of results

This process might be visualized using Rancher. I can answer your questions in chat.
> 5 votes

---
Tags: docker, deployment, kubernetes, javascript
---