# Question
Title: Is it common for DevOps engineers to write code for user requirements?
After reading some articles about DevOps, I understand that DevOps engineers do write code. However, it's a bit unclear to me about what kind of code a DevOps engineer write.

Assume that there are (very roughly) 2 categories of code:

1. **For system operation**: like scripting for deployment, configuration of Puppet or Ansible, etc.
2. **For user requirement**: e.g. allow user to upload avatar images, add an "export" button to download as .csv files.

I found that some DevOps engineers only do the first category, while some others do both.

My question: is it common for DevOps engineers to write code for user requirements?

# Answer
# Short Answer

It depends on the organisation if you are a dedicated infrastructure specialist without any software development experience, you are unlikely to be writing feature code. However, if you are a Software Engineer who also knows Terraform, you might be called on to do both on the same day.

# Long Answer

DevOps is not strictly a job title; in the most general sense, the "DevOps Mindset" forms part of an agile or lean operating model. In this kind of model, DevOps is a way of working for everyone that focuses on the Three Ways:

1. **Systems Thinking** \- understanding the creating value is the product of the system as a whole, not any one part of the system. When I say systems, I am talking about systems that include humans and technical components, i.e. people writing automation or features in code and the technology that executes that code.
2. **Amplification of Feedback Loops** \- creation and tightening of information flowing between the consumer and the parts of the system.
3. **Culture of Continuous Experimentation and Learning** \- enabling a data-driven culture that encourages agility and learning through experimentation.

## Adoption Patterns

The three ways don't specify how the organisation should be structured; they just provide the foundational principles of DevOps. Coupled with the fact that most companies don't change quickly, this means there are several DevOps "Adoption Patterns":

1. **DevOps Bridge** \- where a team of DevOps engineers write the code to automate deployment and operations and often operate that.
2. **DevOps Operations Teams** \- where a team writes code to operate the software, but the development team writes the code, infrastructure and deployment automation.
3. **Tools Teams** \- where a group of DevOps engineers write tools that developers use.
4. **Embedded DevOps** \- where a DevOps engineer is embedded in the team to write infrastructure and deployment automation and built out the operational capability.
5. **Ubiquitous DevOps** \- where the group as a whole is accountable for feature development and infrastructure/deployment automation. They might have a DevOps Engineer or developers might "do DevOps".

## Google and Amazon

There are also two general approaches described by Amazon and Google:

1. **"You build it, you run it"** \- this is a mantra that Werner Vogels, Amazon's CTO, used to describe how Amazon scaled Development and Operations teams. i.e. the group as a whole is responsible for building features, delivering it into Production and operating it while it is there.
2. **Site Reliability Engineering** \- this is Google's (and many other organisations) technology operating model that does split Development and Operations, but in such a way that there is shared accountability for the quality and warranty of code.

## Amazon's "You Build It; You Run It" Model

Teams within Amazon are accountable for delivering a service, not software or infrastructure. Therefore they need to have people in the group who are capable of fulfilling every technical role: Software Architects, Back-end Developers, Front-end Engineers, Infrastructure Engineers, Automated Testers, Operations Engineers, and so on. Some people in those teams can fulfil more than one role; others may be much more specialist.

Within Amazon's model, it's the team as a whole's responsibility both implement features and write automation (amongst other things) they may:

1. Choose to hire a Software Engineer to write the code for the features; plus a DevOps Engineer to write the code for the automation, or.
2. Hire two engineers who can both do both roles within the team.

## Google's Site Reliability Engineering Model

Google did their own thing for about eight years and came up with Site Reliability Engineering, SRE. Broadly Google employs Software Engineers (SWEs) and Site Reliability Engineers (SREs), Software Engineers are responsible for building code, automating infrastructure and automated deployment with the promise that if the project is important enough and they have proven they can operate it in Production themselves that Site Reliability Engineers will take over the operation of the project.

SRE gives Google the best of both worlds; teams are accountable for getting a product live and making sure it's reliable. Site Reliability Engineers can then take over up to ten live products meaning Google reduce their on-call costs and free up Software Engineers to concentrate on developing new features or products.

If you refer to Google's Site Reliability Engineering triangle:

* **Product** \- is what you think of as high-level "feature" development, i.e. understanding what the user wants and implementing it.
* **Release Processes** \- is what you probably think of as code that improves the flow of features into Production which could be deployment code or infrastructure.
* **Development** \- is a bit of all three worlds: code that implements a feature, deployment code or infrastructure as code.

## My Journey

My journey to where I am now (heading up Managed Services) was first as a Network Technician and IT Manager, then becoming a .NET Developer, and then a DevOps consultant, finally taking up my current role. I'm at the point in my career where I can turn my hand to more or less any task.

When I hire, I look for T-Shaped individuals, i.e. people who are experts in one or two disciplines but are moderately capable in the full range of disciplines we need to be successful.

# Summing Up

When you're just starting, start with the end in mind:

* If the goal is a team that is cross-functional and capable of building and running a product - then work out what that team needs to do and make sure that there is more than one person that is proficient in each capability.
* If the goal is to become a DevOps engineer, start with the basics of Infrastructure of Code, Continous Deployment and Operations. As you learn, teach those in your team how to do those things themselves and ask them how they do their role (this includes QA, Front-end, Backend and Hardware engineers) eventually you become that T-Shaped individual.
> 5 votes

# Answer
As already mentioned, "It Dependes" and more importantly who are your "Users",

1. is it the dev/test team for whom you are creating an infrastructure -\> sure.
2. is it the staging server/deployment server -\> ofcourese.
3. is it end users, yes. In many cases there are scenarios like encrypitng data on transport or at encryption at rest, a devops professional do write code to fulfill those requirements.
> 0 votes

---
Tags: culture, terminology, organization
---


# Question
Title: Jenkins cron works only for the 1st line
I'm trying to use Jenkins to run a small monitoring script every minute on two separate ECS clusters. In the text box that opens when you enable "Build periodically with parameters" I wrote:

> * * * * * %CLUSTER=Cluster1;

> * * * * * %CLUSTER=Cluster2;

This works fine for Cluster1 but it does not run for Cluster2. It does not fail, it just doesn't run. If you switch the lines it works for Cluster2 (which is now at the top) but not for Cluster1. I made sure the node it runs on has enough runners, and that "Execute concurrent builds if necessary" is enabled. The closest thing I found while googling is this, and I checked but there is a blank line after these and adding another or removing it or adding a space between them does not help.

I have a bad workaround which is just squeezing both runs in an unparametrized build and that works but it is inflexible.

So to reiterate the question, why does Jenkins not run concurrent builds from cron, and what can I do to change that?

Thanks.

**EDIT:**

When you insert a cron schedule of * * * * \*, Jenkins posts the following:

> Do you really mean "every minute" when you say "\* * * * \*"? Perhaps you meant "H * * * \*" to poll once per hour.

As I need it to be on a per minute basis this is (for the 1st one, but not for the 2nd) the desired result. Based on Ian W's suggestion to do replace the schedule from "\* * * * * " to "H/2 * * * \*". I checked it but this still led to only the 1st one being run, and would have caused it to run every other minute and not every minute as intended. H/1 doesn't run at all, and H/0.5 returns an error of

> Invalid input: "H/0.5 * * * * %CLUSTER=Cluster1;": line 2:3: step must be positive, but found 0

I also tried H/1.1 but that returned:

> Invalid input: "H/1.1 * * * * %CLUSTER=Cluster1;": line 2:4: unexpected char: '.'

The closest I got so far was staggering the cron jobs like so:

> H/2 * * * * %CLUSTER=Cluster1;

> * * * * * %CLUSTER=Cluster2;

Which caused both jobs to run every 2 minutes.

BTW, the Jenkins version is 2.204.1, the Parametrized Scheduler plugin is on version 0.8, if any other plugins are relevant I can post their versions as well.

# Answer
I believe your issue is the use of * * * * \*.

The parameterized-scheduler-plugin example has two different times specified.

```
#lets run against the integration environment at 15 past the hour
15 * * * * % env=int
#run QA too
30 * * * % env=qa

```

So one job runs `@15` and one `@30`.

Yours specify the same time, so only the first is executed.

You could try the random time parameter `"H"` or fix some different interval on each.

Or, it does say "Leave a space. Put in a %. Then add the name=value pairs". So maybe it's `cronexp space percent space name=value` ?

OR you could just copy the job job and schedule two separate jobs w/fixed parameter. But Jenkins does sound like the wrong tool for this.

The above will address why only the 1st line was executed.

**NOTE** :There is a known issue attemptimg to to every minute, see JENKINS-22129 H/2 will run every 2 minutes, but H/1 will run every hour.
> 2 votes

---
Tags: jenkins
---


# Question
Title: How does Kubernetes terminate an instance after helm upgrade
I have a node.js application that is running on a Kubernetes cluster, I am using helm to manage it currently. I have ran into a bit of an issue, when the application starts it connects to a service discovery tool, when it is to shut down it should disconnect from said tool. If I do it locally then it works fine, however when I do a helm upgrade it goes sideways and doesn't disconnect which then results in the api gateway having a fit.

Is there something different than a basic helm template and using upgrade? The application supports what I need just a matter of having Kubernetes do it too.

Here is the express server and consul code, it goes at the bottom of my app.js file.

```
const port = 8081;
app.listen(port, () => {
    const CONSUL_ID = require('uuid').v4();
    const ip = require('ip');
    const my_IP = ip.address();
    let options = {
      name: 'auth',
      address: `${my_IP}`,
      port: port,
      id: CONSUL_ID,
      check: {
        ttl: '10s',
        deregister_critical_service_after: '1m'
      }
    };
    consul.agent.service.register(options, function(err) {
      if (err) throw err;
      console.log(`Registered service with ID of ${CONSUL_ID}`);
    });

    setInterval(() => {
      consul.agent.check.pass({id:`service:${CONSUL_ID}`}, err => {
        if (err) throw new Error(err);
      });
    }, 5 * 1000);

    process.on('SIGINT', () => {
      console.log(`SIGINT. De-Registering service with ID of ${CONSUL_ID}`);
      consul.agent.service.deregister(CONSUL_ID, (err) => {
        if(err) console.log(`Error de-registering service from consul, with error of : ${err}`);
        if(!err) console.log(`De-registered service with ID of ${CONSUL_ID}`);
        process.exit();
      });
    });
    console.log(`Auth server running express started on port ${port}.`);
});

```

# Answer
> 1 votes
Suppposing that you are using typical Deployments, a new replicasets is created and the old one shrunk.

Looking at the docs for 1.16, it seems that TERM is sent, and then finally KILL.

> When a user requests deletion of a Pod, the system records the intended grace period before the Pod is allowed to be forcefully killed, and a TERM signal is sent to the main process in each container. Once the grace period has expired, the KILL signal is sent to those processes, and the Pod is then deleted from the API server. If the Kubelet or the container manager is restarted while waiting for processes to terminate, the termination will be retried with the full grace period.

https://v1-16.docs.kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods

Therefore, you'll want to accept TERM signals, not just KILL. Also, you'll need to ensure the signal is properly transmitted from the surrounding docker container into your server process.

---
Tags: docker, kubernetes, helm
---


# Question
Title: What is a reasonable level of coding expertise expected of an SRE?
SREs and coding is a contentious topic. In most situations, SREs end up spending a lot of time on operational tasks, even while being efficiency and automation focused. This means that SREs don't spend as much time coding as a full time developer, which puts them in a very disadvantageous position as far as programming rigor is concerned.

Also, SREs are typically responsible for multiple services or applications, unlike developers who may be more focused on a small number of applications. Gaining deep knowledge of large codebases can be a daunting task especially if the focus is diffused by interrupt-driven nature of SRE work.

To be very good in coding and be able match a developer's level of expertise in programming, SREs need to spend more time coding on a continuous basis, which may not be possible.

Also, the question arises - if an engineer is a great developer, why would he or she opt for an SRE role? The pure development roles seem to have a better career path.

Also, the argument that SREs should spend most of their time coding and not in operations seems a little disconnected from reality. Not all operations is evil and not all code is an angel either. As someone said, "the best code is the one that was never written". Running operations efficiently and reliably is an important goal for SREs and coding can be helpful in achieving that goal, though it is not the only means to get there.

So, what is a reasonable expectation for an SRE when it comes to coding? Is it practical to have two cadres of SREs -- development focused ones or "SWE SREs" and operations focused ones or "Operation SREs"? Could they be a part of the same team? Is it practical to rotate people between these two roles?

I hope this question is not dismissed as opinion-based.

# Answer
> 10 votes
**If an engineer is a great developer, why would he or she opt for an SRE role?**

Here is a caricature of being a developer: Working in Sprints, getting a ticket with all of the BA work completed, spending 3-4 days on that problem, marking it done, getting the next ticket and so on. This is a really boring work environment. Yes there are better environments to work in, and not all development roles fall into the scrum mediocrity trap, but it happens frequently.

DevOps is new enough that it's not yet been "solved" as a problem. It certainly hasn't yet had all of the engineering challenge managed into separate roles, so it allows a certain type of developer a bit more freedom, working using multiple skills, in a fast paced environment. It's not for the faint hearted.

**What is a reasonable expectation for an SRE when it comes to coding?**

That they are a programmer that you would want on the development team. SRE is not a place for substandard developers. The development challenges in SRE are as great as that in the development space, but the space for errors is less.

**Is it practical to have two cadres of SREs -- development focused ones or "SWE SREs" and operations focused ones or "Operation SREs"?**

Not really. To be in SRE you must understand Dev and Ops. If thats not the case, then you are not doing SRE, you are doing something else. It doesn't mean it's wrong, or not working for you - it could be just what you need for your environment, and you have the people that make it work. It is possible to have a specialist who isn't really a developer on the team, IF they bring an important skill to the team that the developers are not able to cover (e.g. low level networking or linux kernel knowledge), but this is done rarely.

**Could they be a part of the same team?**

You can place Dev and Ops into the same team. It would make for some interesting management challenges. Please run the team in a more kanban style, ops doesn't work in sprints

**Is it practical to rotate people between these two roles?**

Most developers benefit from an exposure to the land of DevOps. However, if it is a true ops environment, that timescale is probably 2-3 days at most. Not sure most ops people would be able to assist meaningfully as a developer, but they would often make a good tester. Most ops people would not want to be in a testing role for long.

# Answer
> 4 votes
I think that if an SRE is spending “a lot of time on operational tasks” then they are missing an opportunity to improve their situation through code. Not all code is managed, but one of the driving tenets of the DevOps and SRE models is to approach operational problems as if they are software problems.

As far as rotating roles within and SRE team, you do what it takes to deliver value for your organization. I don’t think there’s a single right answer; I do think that the goal is to minimize manual intervention as much as possible, as well as doing everything you can to identify and resolve reliability issues. Whether you fulfill this as a traditional developer or from a zero-code approach doesn’t matter.

# Answer
> 0 votes
> So, what is a reasonable expectation for an SRE when it comes to coding? Is it practical to have two cadres of SREs -- development focused ones or "SWE SREs" and operations focused ones or "Operation SREs"? Could they be a part of the same team? Is it practical to rotate people between these two roles?

`class SRE implements DevOps` as they say. There are other models.

If you classify SRE as *Software Engineers tasked with operations and asked to deliver software engineering oriented solutions*, then you will want to hire SWEs with operational knowledge, and ask them to build a team to inculculate that knowledge. They should be coding first, but oriented on systems and infrastructure. SRE is where generalists who deal with high levels of ambiguity and ignorance thrive; there's always something new.

Concretely: A SRE should be the kind of person who looks at Kubernetes and is happy to write code against the APIs directly, rather than routing through YAMLs as the interface.

---
Tags: culture, sre
---


# Question
Title: Is it possible to view all changes made on a Git feature branch in Azure Devops?
Azure DevOps offers the overview of changes made in a branch when a PR is created. However, sometimes I'd like to have access to that overview when viewing a branch without the need to create a PR (or check out the branch locally). In the 'branches' section it's possible to view the changes on per commit basis, but the overview of all changes seems to be missing.

Is it possible to have this kind of overview of changes made in one branch in Azure DevOps without creating a PR?

# Answer
> 7 votes
You're looking for the branch compare functionality. In the list of branches, click the overflow ellipses on your feature branch and select compare branches.

Azure DevOps will diff your branch with the default branch which is changeable if necessary.

---
Tags: git, azure-devops
---


# Question
Title: Any site to host my little utility script?
I have a small node js script running in my aws ec2. It is just one file. If I hit the server with a query param, it will do some utility job (like hashing) and return the result in html.

Since its a small script I wondered if there is any site that can host small code for free. I can create a path/subdomain for my script and upload my script. so whenever I hit the my url the should just feed the url to my script and return whatevr the script returns as http response.

Is there any such site there?

If node js is too much, I can make it as a shell script too.

# Answer
Have you considered running this on AWS Lambda. It will probably forever free under free tier. There may be a couple of cents cost on API Gateway.

Otherwise you can rent outdated servers for cents per month, but I am unsure what the best search term is for that.

Finally, email addresses are free to generate and you can invest the time to set up a fresh AWS account once per year and run on free tier.
> 2 votes

---
Tags: cloud, shell-script, servers
---


# Question
Title: How to create a second IP just for dockers
**Problem:** my server is already using port 80 and I would like launch another service that uses port 80 on another address (I don't want to remember ports or force my wife to use non-default ports).

I've created a secondary IP address for all of my dockers (the dockers themselves will only map their ports to one address) but my server grabs port 80 for the default service on both addresses.

Is there a docker way to create a second IP address and avoid this problem?

Thanks

# Answer
If you pass the listen address as the very first component of `-p`, Docker will only listen on that address:

```
$ docker run -p 127.0.1.2:8080:80 -d nginx`
$ ss -ltp | grep 127.0.1.2
LISTEN  0       4096         127.0.1.2:http-alt          0.0.0.0:*

```
> 1 votes

---
Tags: docker, networking
---


# Question
Title: Is there a way to make Prometheus to scrape only the metrics which are present in Grafana?
As a customer of Grafna Cloud, I need to minimize the number of DataSeries that are being sent (remote\_write) to Grafana Cloud, to avoid being overcharged for metrics that I don't really use.

This can be done by filtering out the metrics that I don't use in my Grafana dashboards, is there a way to do that automatically? Without having to write down all the `metric_relabel_configs` rules manually?

# Answer
I haven't found any way to do that, so I created a simple CLI based on Python, which can assist with the task - make sure you don’t scrape metrics in Prometheus, which you don’t present in Grafana dashboards.

frigga - https://github.com/unfor19/frigga
> 1 votes

---
Tags: monitoring, prometheus, metrics, grafana
---


# Question
Title: What do I need to do as to prevent docker from downloading when I `run`?
If I don’t want anything to download when I run

```
 docker run -ti --rm \
            -v /home/sable/heudiconvTest/BIDS:ro \
            -v /home/sable/heudiconvTest/qsipOutput:/out \
            pennbbl/qsiprep:latest \
            /data /out participant

```

What do I need to do? Do I just do `pull pennbbl/qsiprep:latest` and that’s it?

# Answer
Provided that the commands run on the same machine, yes, that should work.

To be even safer and to make it clearer, I would use an explicit version tag, such as `0.9.0beta1`, or `0.8.0`, instead of `latest`.
> 1 votes

---
Tags: docker
---


# Question
Title: A tool to orchestrate cloud server (virtual machine) instances
Whenever I use search keywords such as "orchestration" or "server clusters" or "iaas" I'll always end up in hundreds of results that are going to talk about containers only.

My case is to be able to deploy, scale and monitor VM instances such as DO's droplets or AWS' EC2s. Currently I'm using a CD tool that will use Ansible to spawn (using just REST API) and provision newly created machines. Although neither of these two tools will have capability to monitor created instances, check up on their resources usage, scale them on demand or map them with respective HTTP endpoints (assume 1 VM hosts n webservices). So to assist CD+Ansible in their jobs I need custom tools that would: keep track of number of instances, their status, DNS server, integration with ingress loadbalancer (haproxy) to keep track of which servers are up and when, what are the IP addresses and/or networks of the newly created droplets and more.

I don't quite understand if terraform is even helpful in this case. From what I understood it'll merely help me with spawning a new droplet which I can as well do with single HTTP request. If I want to have more I need to go with paid cloud?

What would be a helpful stack to aid my case?

# Answer
> 3 votes
Is there a reason why you are trying to monitor it from the outside? Can an autoscaling group with a load balancer be a solution. If yes, then provisioning these resources, having the ability to track & update configuration is something that will be possible using a tool like terraform(multi-cloud support) or cloudformation(cloud-specific). Ansible might also be an alternative though it brings its fare share of pros & cons being stateless and relying on the actual cloud resources for tracking deviations

# Answer
> 2 votes
TL;DR; Terraform does allow you to get a lot of what you want done.

Also, you might benefit from a more managed solution like AWS Fargate (which you also declare with Terraform).

> Whenever I use search keywords such as "orchestration" or "server clusters" or "iaas" I'll always end up in hundreds of results that are going to talk about containers only.

That is because containers and serverless are the way of the future and most of us are moving our legacy workloads to those technologies. You should try and avoid starting new projects with an EC2 deployment pattern. In particular if you don't have the skillset to manage it all or the money to waste.

> My case is to be able to deploy, scale and monitor VM instances such as DO's droplets or AWS' EC2s.

Amazon provides auto scaling groups that allow to scale out EC2 instances.

For monitoring, you either use basic metrics (CPU, Network, etc) or you install an agent to monitor system internals (RAM and other application specific data). The installation of the agent (either a vendor like Datadog or AWS CloudWatch) is usually done through the instance's "User Data".

Terraform allows you to pass "User Data" scripts to your instances and has a nice templating syntax.

> Currently I'm using a CD tool that will use Ansible to spawn (using just REST API) and provision newly created machines. Although neither of these two tools will have capability to monitor created instances, check up on their resources usage, scale them on demand or map them with respective HTTP endpoints (assume 1 VM hosts n webservices).

Correct, as mentioned before Ansible is not the right tool for the job. I would say it is terrible in the space. The main aspect is the lack of state tracking and the fact that it is procedural instead of declarative (this is how I want something done vs this is what I want my end state to be go figure it out).

> So to assist CD+Ansible in their jobs I need custom tools that would: keep track of number of instances, their status, DNS server, integration with ingress loadbalancer (haproxy) to keep track of which servers are up and when, what are the IP addresses and/or networks of the newly created droplets and more.

If you are using an AutoScaling group, which seems to be what you are suggesting, you don't track individual instances. You define your minimum, maximum and your scaling criteria. Terraform can definitely declare and deploy the autoscaling group.

The autoscaling group allows you to define health checks to kill instances.

If you are in the autoscaling space you don't want to have individual DNS records for your instances (though for service discovery with HAProxy you might need to do SRV records or integrate with Cloud Map). Also, if you change load balancer technologies AWS ALB allows you to attach your instances to a target group without you having to do anything.

Both an AWS ALB or HAProxy would also make use of health checks so your orchestration tool doesn't need to know about the health of the instances.

> I don't quite understand if terraform is even helpful in this case. From what I understood it'll merely help me with spawning a new droplet which I can as well do with single HTTP request. If I want to have more I need to go with paid cloud?

We have thousands of instances and containers, hundreds of ALBs and a handful of lambdas all managed with Terraform. We let Fargate and other tools manage the blue/green aspects of the deployment process but Terraform defines the infra we want.

> What would be a helpful stack to aid my case?

Move to containers, use AWS Fargate so you don't have to manage the orchestration aspects of your autoscaling, health checks, rollbacks, etc.

Drop Ansible, it is not the right tool for this job. Definitely give Terraform a try.

---
Tags: ansible, amazon-ec2, orchestration
---


title: How to revert jenkins Safe restart?

text:
# Question
Title: How to revert jenkins Safe restart?
Jenkins was having issues and I safe rebooted it in order to solve the issue but the issue got solved the reboot. Some jobs have ours to run so is there way to cancel the reboot?

# Answer
I assume you talk about jenkins cli?

There is no command to abort the `safeRestart`.
Use `quiteDown`, `cancelQuietDown` instead and `safeRestart` only if needed.

> `safeRestart`: Puts Jenkins into the quiet mode, wait for existing builds to be completed, and then restart Jenkins
> `quietDown`: Put Jenkins in a Quiet mode, in preparation for a restart. In that mode Jenkins don’t start any build
> `cancelQuietDown`: Cancel the effect of the “quiet-down” command
> 2 votes

---
Tags: jenkins
---


# Question
Title: How to check available volume provisioners in Kubernetes cluster?
Is there a command or script that one can use to see or list enabled/available volume provisioners in a Kubernetes cluster?

We have an on-premise cluster and I came to know the hard way that 'local' provisioner has already been enabled in the cluster. So I am trying to see if there is something we can do upfront and set up a few of the provisioners for use by the containers/applications.

# Answer
Maybe storage classes?

`kubectl get sc`?
> 1 votes

# Answer
Came to know that there is no way to know this. Somewhere in the documentation I read that system will error out if the referenced provisioner is not present.
> -1 votes

---
Tags: kubernetes-volume
---


# Question
Title: AWS Temporary Permission Granting Tool for Account/Service "Firefights"
I'm looking for a service or third party solution that allows for granting temporary access to an IAM user (or group) to one or more services in an AWS account.

The example I have is there is an outage in a production account, and I would like to grant temporary access to developers/sys admins to resolve the issue. Under normal circumstances, they would not have access to this particular service/account.

Ideally, I would like this to me a managed off the shelf solution vs something like STS, which would require quite a bit of configuration. Example, a system outage occurs for unknown reasons in a production account that the on call SRE does not normally have access to. A manager or lead should quickly be able to temporarily escalate privileges for the user until the "firefight" is resolved.

# Answer
> 1 votes
1. Create an IAM role in prod with all the permissions you'd like the devs to have.
2. When you want devs to have prod access, add the `sts:AssumeRole` permission to their IAM group.
3. When it's time to revoke access, remove the `sts:AssumeRole` permission.

More details in my blog article here.

# Answer
> 0 votes
It would be better to use the official docs as trusted source :) AWS Official Docs

---
Tags: amazon-web-services, aws-iam, sysadmin
---


# Question
Title: How to trim spaces for parameter value in jenkins pipeline?
I am using jenkins pipeline job with parameters. when i had any trailing spaces in parameter value the job gets failed. is there any way to trim the spaces in pipeline job to avoid build failure?

# Answer
Use the Groovy `trim()` method. For instance, `def myTrimmedParam = params.myParam.trim()`.
> 6 votes

# Answer
You can use the trim parameter:

```
string(name: 'some parameter', trim: true)

```
> 4 votes

---
Tags: jenkins, jenkins-pipeline
---


# Question
Title: Are manual changes to an aws IAM policy detected by Terraform plan?
I know that Terraform dose some checks before the plan and can detect some changes but not others such as detecting changes to variables on a lambda but not detecting a new secret in a secret manager.

Are changes to the IAM policy something it detects?

# Answer
Yes, Terraform will detect drift in IAM policies.

Here is my starting point, a policy that allows full S3 and SQS access to specific resources.

```
data "aws_iam_policy_document" "task" {
  statement {
    effect  = "Allow"
    actions = ["sqs:*"]
    resources = [
      aws_sqs_queue.my-queue.arn
    ]
  }

  statement {
    effect  = "Allow"
    actions = ["s3:*"]
    resources = [
      "${aws_s3_bucket.my_bucket.arn}",
      "${aws_s3_bucket.my_bucket.arn}/*",
    ]
  }
}

resource "aws_iam_policy" "task_role" {
  name   = "my-task-policy"
  policy = data.aws_iam_policy_document.task.json
}

```

I then manually added `ec2:DescribeInstances` to this policy via the AWS management console.

To test whether it was detected or not, I ran `terraform apply -target aws_iam_policy.task_role`

```
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # aws_iam_policy.task_role will be updated in-place
  ~ resource "aws_iam_policy" "task_role" {
        arn    = "arn:aws:iam::111111111111:policy/my-task-policy"
        id     = "arn:aws:iam::111111111111:policy/my-task-policy"
        name   = "my-task-policy"
        path   = "/"
      ~ policy = jsonencode(
          ~ {
              ~ Statement = [
                  ... REDACTED...
                  - {
                      - Action   = "ec2:DescribeInstances"
                      - Effect   = "Allow"
                      - Resource = "*"
                      - Sid      = "VisualEditor0"
                    },
                ]
                Version   = "2012-10-17"
            }
        )
    }

Plan: 0 to add, 1 to change, 0 to destroy.

```

So Terraform correctly detected the drift.
> 3 votes

---
Tags: amazon-web-services, terraform
---


title: How not expose my backend

text:
# Question
Title: How not expose my backend
I'm studying Docker and how to properly publish my containers coding a very simple web application. It has only two containers, backend and frontend. I've published the source code here: https://github.com/sinayra/hello-private-world

If I run locally each node application, its works "fine" (everything is public, but this is not what I want in the end of the day). Then, I created Dockerfile for each container, to try to simulate a real application. For the frontend, I exposed the port 3000 to access from the host. However, I don't want to expose the port 3333 for the host, and, because of that, my front can't fetch the data from the back.

Here is my index.html from the frontend.

```
<html>
  <head>
    <title>Hello Public World</title>
  </head>
  <body>
    <h1>Message from backend:</h1>
    <p id="backend"></p>

    <script>
      const back = document.getElementById("backend");
      const url = "http://localhost:3333/"; //I tried backend:3333, but without success

      fetch(url)
        .then((resp) => resp.json()) // Transform the data into json
        .then(function (data) {
          back.innerHTML = data.message;
        });
    </script>
  </body>
</html>

```

Its Dockerfile:

```
# pull official base image
FROM node:lts

# set working directory
WORKDIR /usr/src/app

# install app dependencies
COPY package.json ./
RUN npm install

# add app
COPY . ./

EXPOSE 3000

# start app
CMD ["npm", "run", "start"]

```

My index.js from backend:

```
const express = require("express");
const app = express();
const cors = require("cors");
const port = 3333;

const allowlist = ["http://localhost:3000"];
const corsOptions = {
  origin: allowlist,
  methods: "GET",
  preflightContinue: false,
  optionsSuccessStatus: 204,
};
app.use(cors(corsOptions));

app.use(express.json());
app.get("/", (req, res) => {
  console.log("GET /");
  res.json({ message: "Hello Private World!" });
});

app.listen(port, () =>
  console.log(`Example app listening at http://localhost:${port}`)
);

```

Its Dockerfile:

```
# pull official base image
FROM node:lts

# set working directory
WORKDIR /usr/src/app

# install app dependencies
COPY package.json ./
RUN npm install

# add app
COPY . ./

#EXPOSE 3333

# start app
CMD ["npm", "run", "start"]

```

And the Docker compose file

```
version: "3.8"
services:

  backend:
    image: helloworld-private
    build:
      context: ./backend
      dockerfile: Dockerfile
    working_dir: /usr/src/app
    container_name: "express_container"
    volumes:
      - ./backend/:/usr/src/app
      - /usr/src/app/node_modules
    networks:
      - webappnetwork
    expose:
      - "3333"

  web:
    image: helloworld-public
    build:
      context: ./web
      dockerfile: Dockerfile
    working_dir: /usr/src/app
    container_name: "liveserver_container"
    links:
      - backend
    volumes:
      - ./web/:/usr/src/app
      - /usr/src/app/node_modules
    environment:
      - PORT=3000
    networks:
      - webappnetwork
    ports:
      - "3000:3000"

networks:
  webappnetwork:
    driver: bridge

```

Is there any abstraction layer that I'm missing?

# Answer
> 2 votes
The layer that you're missing is nginx proxy. See my toy Mafia Game project here how to do it:

The missing piece in your case is nginx proxy that you use in your UI.

So what you need to do is to define some location for your api - i.e. `/api`. Then your proxy it via nginx - see my nginx configuration for the project above here - https://github.com/taleodor/mafia-vue/tree/master/nginx. At the same time it's highly recommended for performance to deploy static resources on the same nginx.

Use Dockerfile to define environment variables and defaults, i.e. in my project - https://github.com/taleodor/mafia-vue/blob/master/Dockerfile

Finally, you can update configuration via environment variables in your docker-compose yaml or k8s definitions.

---
Tags: docker, docker-compose, containers, security, node.js
---


title: What's in THIS environment?

text:
# Question
Title: What's in THIS environment?
We work on multiple java web projects going on at the same time and those are all being tested on several different QA environments. I'd like to show on our support portal what projects are in what environments without having to manually update it all the time. I was thinking of some way to tag the builds so I could just query tomcat or those linux servers themselves and show the results. The tags would be something like "August Release", "Project 1", "Project 2", etc.. Has anyone done something similar? I'm looking for different options.

# Answer
> 3 votes
I've done this before, normally in the form of having a `/.well-known/info` endpoint available on every microservice that then returns a JSON payload with what's in it:

```
{
    "service_name": "token_exchange",
    "server_id": "RD1717628373",
    "commit_hash": "f72a28",
    "branch": "feature/content-security-policy",
    "version": "v2.8.12-f72a28-prerelease1",
    "dependencies": [
        {
            "id": "db",
            "type": "database",
            "endpoint": "db.mysite.com"
        },
        {
            "id": "service",
            "type": "microservice",
            "endpoint": "RD661872834"
        }
    ]
}

```

I then have a cron job and custom dashboard that queries all of the services that it knows about and then walks the dependency tree to add any newly added dependencies.

# Considerations

1. **Agree a Standard** \- The first step is to agree on a standard across services that you want to report on, then get them implemented. It's hard to show the value of this kind of thing when only 1% of your estate implements it.
2. **Start Small** \- I created a command-line interface that went and queried it at first.
3. **Security** \- It could be an attack surface from an information security perspective, so test it and possibly add some authentication component to it and turn it off in production.

---
Tags: jenkins, git, java, tomcat
---


title: auto delete aws S3 backups

text:
# Question
Title: auto delete aws S3 backups
I need to automate deletion of aws S3 backups ... evidently if I tag the backup with one of daily, weekly, monthly, yearly then aws will delete based on my desired retention counts per those periods however I see no easy way to determine which of those tags I give to my daily backup process which is

```
//   Upload backup file

aws s3 cp /tmp/backup_2007020644.20200703.1248_blobs.tar  s3://foo-bar-baz-someaccount-us-east-2/backup_2007020644.20200703.1248_blobs.tar --region  us-east-2 --only-show-errors

aws s3api put-object-tagging --bucket foo-bar-baz-someaccount-us-east-2  --key backup_2007020644.20200703.1248_blobs.tar --tagging  --region  us-east-2 TagSet=[{Key=backuptype,Value=blobs}]

//   now lets list this s3 bucket

aws s3 ls s3://foo-bar-baz-someaccount-us-east-2 --region  us-east-2

2020-07-01 22:55:57   31904428 backup_2007010938.20200701.2233_blobs.tar
2020-07-01 22:55:43     893239 backup_2007010938.20200701.2233_mongo.tar
2020-07-02 15:30:36   34343354 backup_2007010938.20200702.1508_blobs.tar
2020-07-02 15:30:22     893676 backup_2007010938.20200702.1508_mongo.tar
2020-07-03 01:20:04   30596405 backup_2007020644.20200703.0055_blobs.tar
2020-07-03 01:19:51     893741 backup_2007020644.20200703.0055_mongo.tar
2020-07-03 12:48:44   34658003 backup_2007020644.20200703.1226_blobs.tar
2020-07-03 12:48:30     895294 backup_2007020644.20200703.1226_mongo.tar
2020-07-03 15:05:00   34657972 backup_2007020644.20200703.1248_blobs.tar
2020-07-03 15:04:46     895279 backup_2007020644.20200703.1248_mongo.tar

```

Alternatively I can code up my own logic to parse above listing then issue the delete command per backup file after my code keeps track of my retention policy. There must be a better way, any advice?

# Answer
I think you're on the right track. Just need a tiny bit more logic on the backup process, then S3 can handle the rest.

Consider this:

1. A backup is uploaded every day.
2. If today is January 1st (for example), the backup is given a "yearly" tag.
3. Otherwise, if today is the 1st of the month, the backup is given a "monthly" tag.
4. Otherwise, if today is a Sunday, the backup is given a "weekly" tag.
5. Otherwise the backup is given a "daily" tag.

Next, set up S3 Bucket Lifecycle Policies with each tag as the filter.

1. Transition "yearly" backups to Glacier after 365 days (kept indefinitely)
2. Delete "monthly" backups after 365 days (1 year)
3. Delete "weekly" backups after 183 days (6 months)
4. Delete "daily" backups after 31 days (1 month)

References:
> 4 votes

---
Tags: amazon-s3
---


# Question
Title: GitHub Actions Runner execute commands with sudo GitHub Actions
I have a GitHub action set on self hosted Ubuntu VPS on Linode. I ran `sudo ./svc.sh install` and then `./svc.sh start`. The runner is up and working but I have a command in my yaml file that needs to run as sudo. I have this command in sudoers file to run without sudo `%sudo ALL= NOPASSWD: /bin/systemctl restart twentydb.service` But still when I push on the server the following error occurs. `sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper`

```
name: restart the service
run: |
sudo systemctl restart twentydb

```

I also need to set up 2 more runners on the same server and same setup.

# Answer
> 3 votes
airtower-luna from GitHub community answered it here and it worked for me Answer pasted below: The command in your workflow file is not the same as the one in your sudoers file, sudo doesn’t know they are functionally equivalent. Use twentydb.service in the workflow, too.

Also, is the user your runner runs as in the sudo group? If not, that entry won’t apply to it. However, I’d suggest using a separate entry for the user rather than adding the runner to the sudo group.

---
Tags: linux, continuous-deployment, github
---


# Question
Title: Jenkins Declarative Pipeline: Can Jenkins execute a dynamic list of jobs?
Context: I have a long dynamic list of builds calls that I am currently executing in parallel successfully. Example below.

Problem: In some specific moments it is required to run that list of builds but in a sequential fashion.

Question: Is there any way to reuse the list of jobs that I already have generated for the parallel execution, but to execute them in a sequential way?

```
stage ('Execute jobs') {
    steps {
        node('node1') {
            script {
                parallel_jobs = [:]
                if (params.includeJob1 == "true") {
                    parallel_jobs['ci'] = {
                        build job: job1, parameters: [], propagate: true
                    }
                }
                parallel_jobs['second'] = {
                    build job: job2, parameters: [], propagate: true
                }

                // ...
                // and more jobs dinamically added to the parallel_jobs list
                // ... parallel_jobs['etc'] = ...

                parallel parallel_jobs
            }
        }
    }
}

```

I am thinking to replace the 'parallel' call for any other Jenkins command, but cannot find the correct one at docs (https://www.jenkins.io/doc/book/pipeline/syntax/#parallel).

Thank you.

# Answer
> 2 votes
I'm not really sure I understand your question, but you can execute arbitrary Groovy inside of a `script` block, so you can do just about anything within `script`.

For example, here is my interpretation of what you are asking for:

```
stage ('Execute jobs') {
    steps {
        node('node1') {
            script {
                parallel_jobs = [:]
                if (params.includeJob1 == "true") {
                    parallel_jobs['ci'] = {
                        build job: job1, parameters: [], propagate: true
                    }
                }
                parallel_jobs['second'] = {
                    build job: job2, parameters: [], propagate: true
                }

                // ...
                // and more jobs dinamically added to the parallel_jobs list
                // ... parallel_jobs['etc'] = ...

                if (params.parallel) {
                    // execute jobs in parallel
                    parallel(parallel_jobs)
                } else {
                    // execute jobs in sequence
                    parallel_jobs.each { name, closure->
                        closure.call()
                    }
                }
            }
        }
    }
}

```

In Groovy, Maps (i.e. associative arrays) are ordered. So if you use this to execute the builds serially rather than in parallel, the builds will be executed in the order they are defined.

---
Tags: jenkins-pipeline, automation, job-dsl
---


# Question
Title: How can I expose Ingress service on 80 and 443 to the outer world
I used google compute engine to setup a Self Managed Kubernetes, While exposing ingress service via NodePort/LoadBalancer/EnternalIP, it is using default 30000 port range, But I want to expose it on 80 and 443 to the outer world. Please suggest How can I do that?

# Answer
> 6 votes
It's would be better if you will use `ClusterIP` type as a service than apply `Ingress` resource on it

https://kubernetes.io/docs/concepts/services-networking/ingress/#single-service-ingress

---
Tags: kubernetes, google-cloud-platform, ingress, google-kubernetes-engine
---


# Question
Title: how to set a variable from an ini file in ansible?
I would like to set a variable value in a playbook reading it from an ini file, possibly failing the play if the file is not found or the value is missing in the file.

I have found the ini lookup plugin to read values from ini files but it is not clear to me how to put that into a variable that I can reuse in other tasks without having to read it again.

I could think of something like:

```
- name: lookup variable from ini
  ini:
    name: myParam
    section: mySection
    file: /path_to/ini_file
  register: my_variable

```

But since ini is not a module this does not work.

Moreover, the ini file to be read from is located on the remote system, the ini plugin howhever, seems to look on the control host for it (from the warning "Unable to find '/path\_to/ini\_file' in expected paths).

# Answer
Since the ini lookup plugin only reads files local to the controlling host, the remote file needs to be fetched locally before it can be read and set\_fact has to be used to put its content into a variable, which results in the following:

```
- block:
  - name: "Retrieve remote ini file"
    fetch:
      src: /path_to/ini_file
      dest: /tmp/ansible
  - name: "Read and store my value"
    set_fact:
      my_variable: "{{ lookup( 'ini', 'myParam section=mySection file=/tmp/ansible/{{ inventory_hostname }}/path_to/ini_file' ) }}"

```

Then the parameter is available as variable and can be used further in the playbook

```
- debug:
    var: my_variable

```
> 7 votes

# Answer
You need to use `set_fact` when setting a variable (instead of `register` the outcome of a task):

```
- name: Set myParam from ini
  set_fact:
    myParam: "{{ lookup('ini', 'section=mySection, file=/path_to/ini_file') }}"

```

In the lookup:

* `ini` specifies what lookup plugin we are using
* `section` and `file` are arguments passed to the lookup, according to its documentation
> 1 votes

---
Tags: ansible
---


# Question
Title: How to have jenkins add a review to a pull request
I have a Jenkins instance, with credentials for a "robot" github account, which we use to interact with the GitHub API as a team. Typically, **jobs are defined as declarative pipelines**. I am looking for the correct syntax to perform the following steps:

1. At the start of the run, create a review
2. At the end of each stage, post a review comment, with the status of the stage, *e.g.* "build passed".
3. If all of the stages pass, approve the pull request.

I can allow scripted steps, but the pipeline as a whole should be declarative. No shared libraries, and preferably only using commonly-used plugins.

*(update with what I have tried)*

I have tried using the pipeline github plugin's review method:

```
pullRequest.review('APPROVE')

```

naively, but this throws errors.

What would be the correct syntax here?

# Answer
I guess the simplest and direct solution would be to use GitHub API: https://docs.github.com/en/rest/reference/pulls#create-a-review-for-a-pull-request

But for the requirement you have, PR checks is usually the way to go and you have a plugin that does the API communication for you: https://www.jenkins.io/doc/pipeline/steps/pipeline-githubnotify-step/
> 7 votes

---
Tags: jenkins, jenkins-plugins, jenkinsfile, github
---


# Question
Title: Vault `SSH-CA` Elliptic Curve rather than `RSA`
Is it possible to use `ECDSA`(Elliptic Curve Digital Signature Algorithm) or `EdDSA`(Edwards-curve Digital Signature Algorithm) instead of `RSA` when configuring `SSH-CA` on `HashiCorp Vault`?

# Answer
> 3 votes
I think the answer should be yes to ecdsa but no to ed25519, but that it is currently no to both.

You can externally generate an ecdsa keypair and load it into ssh/config/ca but vault-1.4.3 cannot sign using it. I haven't tried ed25519.

Golang's ssh code (which vault uses) doesn't appear to know how to sign with ed25519 but should be able to sign with ecdsa. However, I think there's an issue with an inappropriate default being set elsewhere that means it is always being told to use an rsa-based signing algorithm and hence doomed to fail for ecdsa keys. See:

https://github.com/hashicorp/vault/issues/8414#issuecomment-656145432

If I am right, it looks like an easy fix so it may be available soon.

This is possible as of August 2020, see https://github.com/hashicorp/vault/pull/9824, tested with vault 1.6.2

---
Tags: hashicorp-vault, encryption, ssh
---


# Question
Title: Control GKE CICD from a Jenkins in a lab with private network?
For a test purpose I need to use my locally provisioned Jenkins with Vagrant in order to connect to GKE and use pods to build. Is that possible, because from what I read K8s will need access to Jenkins as well. How can I achieve that?

Looks to be possible, but I am stuck on access rights for now:

```
o.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://xxxxxx/api/v1/namespaces/cicd/pods?labelSelector=jenkins%3Dslave. Message: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" in the namespace "cicd". Received status: Status(apiVersion=v1, code=403, details=StatusDetails(causes=[], group=null, kind=pods, name=null, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" in the namespace "cicd", metadata=ListMeta(_continue=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Forbidden, status=Failure, additionalProperties={}).

```

# Answer
> 2 votes
Taking the url from cluster-info and creating a config file with jenkins and using at as secret did the trick to pass the connection. I am still curious how to authenticate with a bearer token though. I have tried to use existing managed Jenkins secret, but it failed with it.

Update: The authentication happens, but after creating pods the job never runs. Update: It looks like using jenkinsci docker image as a hardcoded podtemplate works, I try to work with Groovy provisioned yaml template that calls GCR and there I got the firstly mentioned error.

---
Tags: jenkins, kubernetes
---


# Question
Title: Sending HTTP requests to private EC2 DNS from other EC2 host works, but not from container on EC2 host
I want to make a container hosted on one EC2 host communicate with another container hosted on another EC2 host.

Currently, if I send an HTTP request from host A to host B using host B's private DNS and the exposed container port, I receive a response correctly. However, if I try the same thing within the container on host B, I get a "Host is unreachable" error. To me, this would indicate it is not a host networking issue, which is configured in AWS, so I won't enter into detail on it.

What puzzles me is that I have done the exact same thing in our pre-production environment and it works perfectly.

What could be the issue and where do you think I should look to troubleshoot this?

# Answer
> 3 votes
There are a couple of things that you can rule out, thanks to the testing that you have already done. You know your container port has an active network socket, that is listening for and accepting connections. You also know that your EC2 instance has working networking.

I can think of three obvious problems:

1. Your host is using local DNS resolution to identify your hostname as 127.0.0.1 and your container port is not listening on the loopback adapter. you can test this on the server by pinging your DNS name and see which IP address it pings. You can also use telnet to connect to the port on 127.0.0.1 and see if that port accepts connections. To fix this, you may choose to make listening on the local loopback work, but you could also fix this by using a DNS entry other than the hosts AWS generated name.
2. You are resolving the hostname to its private IP address, which then is filtered by the AWS Security Group. On your security group you have enabled incoming connections to this port, but you have specified the source either as a cidr that doesn't include its own IP address, or as security group that is not attached to this EC2 instance. This is easy to fix in the security group editor. To prove that this is working successfully and allowing the connection, you can set up an AWS Flow log which will allow you to see all TCP connections on the Network interface.
3. The Container hosting which provides the port mapping also has a firewall element that restricts incoming connections to only some IP addresses, and may also provide some diagnostics to assist in understanding whether it receives the connection, and whether it decides to route or reject the connection to the container.

If it is less obvious, i think you may be in the world of using IP Tables to generate log messages in the syslog as your connection routes though the IP stack. This will at least allow you to confirm that at a network level you are generating a TCP connection and receiving the connection within the OS.

---
Tags: docker, amazon-web-services, amazon-ec2, networking
---


# Question
Title: How to Save and Restore the Iptables rule and configuration from file?
I want to save the current iptables configuration to a file and restore it from a file. I have a lot of rules and modifying the rules really hard for me. So, how can i do that?!

# Answer
If you want to modify the configuration and rules, so you must save the current configuration to a file. So step one is to save the rules configuration by typing the following commands:

```
$ sudo iptables-save > /root/my-iptables.rules

```

To restore it just use the command iptables-restore:

```
$ sudo iptables-restore < /root/my-iptables.rules

```

⚠ When you changed the existing rules after saving the file, You should save it again or change the existing file and restore it from the file.
> 2 votes

# Answer
The first thing to point out, is that you should already be writing IP tables rules to a file, and restoring them from that file at start up - otherwise you will lose all of the rules when the server reboots.

The standard commands to do this on Debian are:

```
sudo iptables-save > /etc/iptables.up.rules
sudo iptables-restore < /etc/iptables.up.rules

```

**DO NOT EDIT THIS FILE**

> If you make a change that breaks your networking, killing your access to the server, then you can only recover by walking to the computer's console, and logging in there to fix the problem. If thats a Cloud server, or in a different city / country / continent you have a bigger problem.

The next issue that i want to cover, is if writing individual rules is hard, the config files just capture the same command that you would type into the terminal window. Using a configuration file does not reduce the complexity here.

Personally, I would use the existing file to help with the syntax (e.g. find something that already does something that i want to do), and then run that command manually in the shell, and verify that it does what i need it to do by testing the network connections carefully, and thoroughly. I would then save rules, knowing that they are already working and tested.

If i messed up, a server reboot, using the management panel for the server would at least restore the previous IP table rules.
> 0 votes

---
Tags: linux, networking
---


# Question
Title: Accessing a AWS docker instance via localhost only?
Suppose I have two apps launched via the AWS ECS cluster (using Docker containers).

I want to expose one app to the world via a public IP (and I do it via the AWS load balancer) but the other one I want to be able to access only internally, so that it would not have any public IPs and would only be accessible internally.

Is this possible to do that at all? I suppose it should be easier via docker containers because I could possibly make them communicate to each other by exposing a localhost via

```
--network="host" in docker run

```

But that would work if I run the two apps on the same EC2 instance.

What if I run them on separate instances but they are using the same load balancer or — separate instances but in the same AWS zone?

What setting would I use in ECS to expose this app only via the localhost?

# Answer
> 3 votes
Have you tried using `awsvpc` mode? https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-networking.html

You can follow this tutorial - https://docs.aws.amazon.com/AmazonECS/latest/userguide/create-public-private-vpc.html

You need a VPC, with at least two subnets, one private and one public. After that the sky is the limit ...

If you need internet access from the private subnet, create a NAT Gateway in the public subnet, and route traffic from the private subnet to 0.0.0.0, via the NAT Gateway - https://docs.aws.amazon.com/vpc/latest/userguide/VPC\_Scenario2.html

P.S - I never use network mode `host` with ECS, you should read more about it here - https://docs.docker.com/network/host/ I didn't find a strong use case for using it, except for testing purposes

# Answer
> 2 votes
First, @Meir beat me to the punch on `awsvpc` network mode, so give him a +1 and read his linked document about task networking.

---

I'm going to expand on that and include an description of how to route to your containers from both a public-facing ALB, as well as from another container.

To simplify things, we're going to pretend we're running on ECS Fargate to start, and at the end I'll point how that would be different for EC2.

You described two ECS services: one that is public-facing behind the ALB, which we'll call **frontend**, and the other, which communicates to other containers internally, we will call **backend**.

Here's what **frontend** that might look like in Terraform:

```
resource "aws_ecs_task_definition" "frontend" {
  family                   = "frontend"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  execution_role_arn       = "" # generic

  container_definitions = "[]" # not included in example
}

resource "aws_ecs_service" "frontend" {
  name            = "frontend"
  cluster         = "my-ecs-cluster"
  task_definition = aws_ecs_task_definition.frontend.arn
  desired_count   = 2 # high availability
  launch_type     = "FARGATE"

  network_configuration {
    security_groups = [] # SG allowing traffic from the loadbalancer to port 8080
    subnets = [] # list of private subnets
  }

  load_balancer {
    target_group_arn = "" # an aws_alb_target_group
    container_name   = "frontend"
    container_port   = 8080
  }
}

```

**backend** would look similar, except instead of a `load_balancer` block it would have this:

```
  service_registries {
    registry_arn = aws_service_discovery_service.internal.arn
  }

```

And these new resources:

```
resource "aws_service_discovery_private_dns_namespace" "internal" {
  name = "internal.dev"
  vpc  = "" # a vpc id
}

resource "aws_service_discovery_service" "backend" {
  name = "backend"

  dns_config {
    namespace_id = aws_service_discovery_private_dns_namespace.internal.id
    dns_records {
      ttl  = 10
      type = "A"
    }
  }
}

```

Service Discovery creates and manages DNS records for the private IPs (ENIs) of your **backend** tasks automatically. That means `backend.internal.dev` now resolves to one of those IPs in your VPC CIDRs, something like `10.10.0.60`.

Now when **frontend** wants to connect to **backend**, it can do so via `backend.internal.dev`. No internal ALB required! Your entire VPC can resolve that domain.

---

*"Okay, but how does this work on EC2?"* you might ask. Well, that's the beauty of `awsvpc` network mode... it works the same way! These task ENIs are completely different network devices than your EC2 instance's ENI. You will need to modify the Terraform slightly, and perhaps review your security groups, but ultimately nothing has changed: the ALB still connects to the **frontend** task ENI, and the **backend** task ENI is still registered in service discovery. You can run multiples of the same container on a single EC2 instance without port collision because they are using different network interfaces.

---

References:

https://aws.amazon.com/blogs/aws/amazon-ecs-service-discovery/ https://aws.amazon.com/about-aws/whats-new/2019/06/Amazon-ECS-Improves-ENI-Density-Limits-for-awsvpc-Networking-Mode/

# Answer
> 0 votes
Regardless of instance count, you can publish only the port of publicly available app by adding security rule which will allow external traffic into that port's bound host's port, and for the second one just bridge to the first app

---
Tags: docker, amazon-web-services, amazon-ecs
---


# Question
Title: Amazon EKS from local kubectl does not cache MFA token/session
I have set up an EKS cluster in AWS. My user-account there is secured with MFA. And I have an `assumerole`, that is allowed to manage my infrastructure.

So I installed and configured my system as stated in the AWS userguide: https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html

I have `kubectl` in version 1.11.x installed, so I can use that binary to communicate with aws (and I do not need the patched amazon version). I then installed `aws-cli` and `aws-iam-authenticator`.

```
user@notebook [04:44:51 PM] [~]
-> % kubectl version --short --client
Client Version: v1.11.0
user@notebook [04:45:13 PM] [~]
-> % aws-iam-authenticator help
A tool to authenticate to Kubernetes using AWS IAM credentials
...
user@notebook [04:45:25 PM] [~]
-> % aws --version
aws-cli/1.15.51 Python/3.6.6 Linux/4.17.8-1-ARCH botocore/1.10.50
user@notebook [04:45:30 PM] [~]
-> %

```

I then created the configuration files from the how-to `~/.aws/credentials`, `~/.aws/config`, `~/.kube_aws/config` and created an alias for myself to use `kubectl` with the new config (since I have more clusters to manage): `alias kubectlaws='KUBECONFIG=/home/user/.kube_aws/config kubectl $@'`.

So here comes my problem: When I connect to the cluster using the `aws-cli` binary, I have to enter my MFA token and then the *session* is cached (I read for about 15 minutes).

```
user@notebook [04:58:52 PM] [~]
-> % aws eks describe-cluster --profile my_profile --name ClusterName  --query cluster.status
Enter MFA code for arn:aws:iam::<MY_MFA_ARN>:
"ACTIVE"
user@notebook [04:59:09 PM] [~]
-> % aws eks describe-cluster --profile my_profile --name ClusterName  --query cluster.endpoint
"https://..."
user@notebook [04:59:30 PM] [~]
-> %

```

But when I do this with my `kubectl` alias, I have to enter the token every time I run a command! Furthermore, I cannot re-use a code, so I have to wait about 30 seconds, until the next one is provided.

```
user@notebook [05:03:56 PM] [~]
-> % kubectlaws get nodes
Assume Role MFA token code: 123456
NAME                             STATUS    ROLES     AGE       VERSION
...
user@notebook [05:04:05 PM] [~]
-> % kubectlaws get nodes
Assume Role MFA token code: 123456
could not get token: AccessDenied: MultiFactorAuthentication failed with invalid MFA one time pass code.
        status code: 403, request id: <SOME_ID>
No resources found.
Unable to connect to the server: getting credentials: exec: exit status 1
user@notebook [05:04:14 PM] [~]
-> %

```

This is really annoying, as you might imagine. Has anyone any idea what I did wrong there? Is this a bug? Or is this some kind of mis-configuration?

Thanks in advance!

# Answer
Indeed, this is a pain. I've been fighting with it for the whole day yesterday, but managed to get it working, let's see if our cases are similar enough.

My guess is that the *usual* `aws configure` configuration with a MFA ARN + the AWS SDK used to develop *kubectl* are not entirely compatible: I have two profiles set for this, one that has its *mfa\_serial\_arn* and *role\_arn* that works as follows:

* First **aws** command I type, I get prompted for MFA token
* From there onwards, while session is active, I can work without typing in more tokens.
* *kubectl* asks for a token EVERY SINGLE TIME.

This is bad.

Now, I set a different profile, without any of the above, and I do the following:

\`\`\`

```
keys=($(aws sts assume-role --role-arn <ARN_OF_DELEGATED_ROLE> \
   --role-session-name  <RANDOM_UNIQUE_NAME> \
   --serial-number <ARN_OF_MFA_DEVICE>\
   --token-code  \
    --query 'Credentials.[AccessKeyId,SecretAccessKey,SessionToken]' --output text))

# profile used here will be configured in ~/.aws/credentials
export AWS_PROFILE=<PROFILE YOU WANT>

aws configure set aws_access_key_id ${temp_keys[0]}
aws configure set aws_secret_access_key ${temp_keys[1]}
aws configure set aws_session_token ${temp_keys[2]}

```

\`\`\`

This creates a **temporary valid** aws profile, that can be used **BOTH** with the aws cli, and kubectl.
> 2 votes

# Answer
Btw: I managed to fix this problem by using `aws-mfa` (https://github.com/broamski/aws-mfa), which I have configured to cache my aws credentials for an hour.
> 2 votes

# Answer
For anyone came here to solve this problem without any workarounds - the solution is simple.

We need to add switch `--cache` to the kubeconfig for a user used to access a Kubernetes cluster.

```
users:
- name: <redacted>
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      args:
      - token
      - --cache
      - -i
      - prod

```

Tested on:

```
$ aws-iam-authenticator version
{"Version":"v0.5.1","Commit":"d7c0b2e9131faabb2b09dd804a35ee03822f8447"}

```

Hope this will help someone.
> 1 votes

---
Tags: amazon-web-services, kubernetes, aws-cli
---


# Question
Title: is it possible to connect an AWS autoscaling group with an application load balancer?
I'm working with AWS cloudformation I have an autoscaling group for a couple of machines running an API server, and I would like to set up an application load balancer as a single entry point, so that client application only sees one URL. I was able to create the load balancer and the autoscaling group but I don't know how to connect them.

```
Resources:
  NodeLaunchConfig:
    Type: AWS::AutoScaling::LaunchConfiguration
    Properties:
      AssociatePublicIpAddress: false
      # IamInstanceProfile: !Ref NodeInstanceProfile
      ImageId: !Ref NodeImageId
      InstanceType: !Ref NodeInstanceType
      KeyName: !Ref KeyName
      # SecurityGroups is like this on purpuse, this work, IGNORE the error message if any
      SecurityGroups: !Ref NodeSecurityGroup
      BlockDeviceMappings:
        - DeviceName: /dev/xvda
          Ebs:
            VolumeSize: 200
            VolumeType: gp2
            DeleteOnTermination: true

  NodeGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    DependsOn: LoadBalancer
    Properties:
      LaunchConfigurationName: !Ref NodeLaunchConfig
      DesiredCapacity: "1"
      MinSize: "0"
      MaxSize: "10"
      # TargetGroupARNs:
      #   - !Ref LoadBalancer
      TerminationPolicies:
        - NewestInstance
      # VPCZoneIdentifier is like this on purpuse, this work, IGNORE the error message if any
      VPCZoneIdentifier: !Ref Subnets
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-node
          PropagateAtLaunch: true
        - Key: Owner
          Value: !Ref Owner
          PropagateAtLaunch: true
        - Key: Department
          Value: AI
          PropagateAtLaunch: true
        - Key: Env
          Value: !Ref Environment
          PropagateAtLaunch: true
        - Key: Job
          Value: KubernetesGroupNode
          PropagateAtLaunch: true
    UpdatePolicy:
      AutoScalingRollingUpdate:
        MaxBatchSize: 1
        MinInstancesInService: 1
        PauseTime: PT5M

  LoadBalancer:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Name: decrypt-loadbalancer
      # SecurityGroups is like this on purpuse, this work, IGNORE the error message if any
      SecurityGroups: !Ref NodeSecurityGroup
      # Subnets is like this on purpuse, this work, IGNORE the error message if any
      Subnets: !Ref Subnets
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-node
        - Key: Owner
          Value: !Ref Owner
        - Key: Department
          Value: AI
        - Key: Env
          Value: !Ref Environment
        - Key: Job
          Value: KubernetesGroupNode
      Type: application

```

# Answer
> 5 votes
Your CloudFormation template currently has the following resources:

* A Launch Configuration
* An Auto Scaling Group
* A Load Balancer

Follow these steps in sequence to attach the auto scaling group to the load balancer:

1. Add a listener resource to your CloudFormation template, as described here. Set its `LoadBalancerArn` property to the load balancer's ARN.
2. Add a target group resource to your CloudFormation template, as described here.
3. Add a "listener rule" resource to your CloudFormation template, as described here. Set its `ListenerArn` property to the listener's ARN. In its `Actions` property, add a `forward` action & provide the target group's ARN like this:

```
Actions:
- Type: forward
  TargetGroupArn: !Ref MyTargetGroup

```
4. Add the target group's ARN to the `TargetGroupARNs` property of the auto scaling group, as described here.

If you prefer to visualize a chain of CloudFormation resources linking the load balancer to the auto scaling group, it'll look like this:

Here's a complete example: LB to ASG Example.

---
Tags: amazon-web-services
---


# Question
Title: Deploying a single-page web application with a java backend on kubernetes and haproxy
My team (full stack) team develops single-page web applications using react as a client and spring-boot java apps as back-end.

We deploy the applications on kubernetes. However, we have troubles making rolling updates of the kubernetes pods seamless. Here's our packaging / deployment process:

1. The CI pipeline builds a docker images packaging the java application as an executable jar file and the compile javascript/css assets that form the react client.
2. That image has 2 entry points: one for starting the jar file (java back-end) and one which starts an nginx that serves the client assets (js/css).
3. A haproxy cluster routes requests based on URL paths to either nginx or the java back-end.

This approach works fine so far.

However, during a rolling update, there will be a number of pods with an "old" java back-end. This leads to the situation that clients with a newly deployed version of the react client make requests to old java back-ends, because haproxy simply routes api traffic to any available java backend.

As i understand haproxy seems to have support for sticky-backends based on some rules, but this defeats the purpose of having a cluster of backends for load-balancing.

How do others tackle this problem?

# Answer
> 2 votes
Modern way to achieve seamless roll-out in this scenario is not on the load balancing (i.e. haproxy) side, but instead via proper deployment planning, backward compatibility and/or feature flags.

The issue with load-balancer tinkering is that in the end of day even if you get sticky sessions right it's not going to solve your problem. The reason for that - imagine a user that opened your app in a browser with old react. Underneath, you deploy new java back-end. If the user doesn't reload her browser - she's still going to use old react app with new back-end no matter what (just because javascript is already loaded to the browser).

Instead, look at following example how you can achieve it using API versioning:

* Let's assume that data model on one of your api end-points is going to change
* Let's say your old api was /v1 (or even un-versioned)

Here is how you can solve it:

1. You create /v2 version of your api, register /v2 version in your react app
2. Your latest java back-end needs to still have both /v1 and /v2
3. You first deploy all java back-ends to your back-end deployment with /v2 support (all your UI is still /v1 - but this is fine, because latest back-end supports it)
4. Once all back-ends are deployed, you can start roll-out of your UI deployment with latest image (again, experience is seamless since all back-ends support both /v1 and /v2)

As you can see, in the above approach, version conflict is avoided and experience is seamless.

P.s. You might want to consider splitting your docker image in 2 - it's a little odd to have a multi-purpose image with 2 entrypoints. There may be little justification for this design.

---
Tags: kubernetes, continuous-deployment
---


# Question
Title: Unable to send cloudwatch logs to loggly using a blueprint lambda - Invalid ciphertext
I am trying to use the loggly Lambda blueprint to send cloudwatch logs to loggly. I am doing this because I have a bunch of lambdas that I need to have a single point of viewing logs. I've followed this guide and have run into an issue when I check the lambda logs after a test

Command used to do ciphertext

```
aws kms encrypt --key-id alias/logglyCustomerToken --plaintext "<my token>"

```

KMS key was created using symetric and default settings, can't delete key to try again or at least I don't know how to delete it and thinking I shouldn't change the code to match a new one

Lambda log message when failed test

```
INFO    InvalidCiphertextException: null
    at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/json.js:51:27)
    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:683:14)
    at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:685:12)
    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:116:18) {
  code: 'InvalidCiphertextException',
  time: 2020-07-08T02:48:55.536Z,
  requestId: 'cf0df165-5a59-4db0-beef-5d1ae3619c37',
  statusCode: 400,
  retryable: false,
  retryDelay: 79.27899980360486
}

```

# Answer
Follow these steps to fix `InvalidCiphertextException`:

1. Open the Lambda function in AWS console.
2. Scroll down to the `Environment Variables` section & click `Edit`:

3. Delete the value of `kmsEncryptedCustomerToken` & replace it with the customer token you got from `https://<your-company>.loggly.com/tokens`. The token looks like this: `72cf6d64-256e-449d-aabd-49e1f422ed58`.
4. Expand the `Encryption Configuration` section at the bottom.
5. Select `Enable helpers for encryption in transit`.
6. Click the `Encrypt` button that appears next to `kmsEncryptedCustomerToken`.

7. Select the `logglyCustomerToken` KMS key (CMK) & `Encrypt`.
8. Finally, `Save` the environment variables.

The Lambda function test should succeed now. If you face any other issues or need a complete end-to-end screenshot-guided tutorial for the entire process of sending CloudWatch Logs to Loggly, please see my blog post.
> 2 votes

---
Tags: amazon-web-services, aws-lambda, logging, amazon-cloudwatch, serverless
---


# Question
Title: AWS subnets for lambda synchronous invocations
Am I right in thinking giving a lambda using an API gateway 3 subnets each with a range of 16 would result in 2 subnets being used at once and 1 IP used for the API gateway that is in the same subnets resulting in a max of 31 invocations before the error code 429 is returned due to Too Many Requests. This is my thinking as there should be no other resources using an ip address that will be in these subnets but I can see testing it as being complicated and taking a very significant amount of time with my current knowledge.

# Answer
I don't entirely understand your use-case, but it sounds like you are trying to limit the number of Lambda invocations by restricting its IP range.

There are two better ways to rate limit your application. Option one would be to use API Gateway **request throttling**:

https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-request-throttling.html

Option two, Lambdas have a **concurrency limit** that can be set to achieve a similar behaviour to your approach:

https://aws.amazon.com/about-aws/whats-new/2017/11/set-concurrency-limits-on-individual-aws-lambda-functions

Hope that helps with your design!
> 0 votes

---
Tags: aws-lambda
---


# Question
Title: Improving deployment process of agent software
I'm a developer on an agent software team, researching ways to streamline the mass deployment of our agent on clients' machines. Currently the process involves running a script to determine the exact package needed, downloading a bsx from our company's FTP and running it, followed by running an executable with flags for registration.

I've looked into Ansible as a possible improvement, however I'm unclear on the value it provides in such a case. What's more, I'm not sure I'm the right person to be authoring the clients' playbooks for such a thing. At this point it would appear to me that developing our own script to further automate the process would be a better solution, and let the customer DevOps team deploy it in whichever way they see fit.

Before I present these findings to my managers, I would appreciate any opinions on the matter the community could provide.

# Answer
> 1 votes
ansible-pull would work well for this, though it makes the assumption that your clients are using ansible. That's not an assumption I would be comfortable making in your shoes.

> At this point it would appear to me that developing our own script to further automate the process would be a better solution

I agree with that. Since you already have a script to figure out what needs to be downloaded, I'd extend that script to download the needed file(s) and trigger the installation/registration.

---
Tags: ansible, deployment, practices
---


# Question
Title: Ansible - Handler not found on first attempt
Command run: `sudo ansible-playbook site.yml --connection=local`

```
Distributor ID: Ubuntu
Description:    Ubuntu 18.04 LTS
Release:        18.04
Codename:       bionic

ansible 2.5.5

```

I'm encountering this error only on my first run through after I change the following variable in the following file:

./group\_vars/all

```
ntpserver: pool.ntp.org

```

`TASK [common : Configure NTP file] **************************************************************************************************************
ERROR! The requested handler 'restart ntp' was not found in either the main handlers list nor in the listening handlers list`

On my second run through, I'm having no issues:

`TASK [common : Configure NTP file] **************************************************************************************************************
ok: [127.0.0.1] => {"changed": false, "checksum": "cc9e1a2d8777588fa4675e52f7ca7e352b23188a", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 139, "state": "file", "uid": 0}`

The folder and file structure I'm using is as follows:

https://github.com/ansible/ansible-examples/tree/master/lamp\_simple\_rhel7

./roles/common/handlers

```
- name: Restart NTP
  service: name=ntpd state=restarted`

```

./roles/common/tasks

```
---
- name: Install NTP
  apt: name=ntp state=present
  tags: ntp

- name: Configure NTP file
  template: src=ntp.conf.j2 dest=/etc/ntp.conf
  tags: ntp
  notify: restart ntp

- name: Install common packages
  apt: name={{ item }} state=present
  with_items:
    - fail2ban
    - ranger

- name: Start the NTP service
  service: name=ntp state=started enabled=yes
  tags: ntp

- name: Upgrade all packages to the latest version
  become: true
  apt:
    upgrade: yes
    update_cache: yes
    cache_valid_time: 86400 # one-day

```

# Answer
The name of the handler "Restart NTP" does not match the handler notified "restart ntp". This is the reason of the error.

On the second run the configuration file /etc/ntp.conf did not change ("changed": false). Therefor the handler was not notified, was not looked for, and no error was reported.
> 5 votes

# Answer
I just found out the hard way. handlers only works with include\_tasks not import\_tasks .

The error message is rather missdirecting. I went all in adjusting indentation, syntax check and all . Finally I recreate the whole playbook but still fail. Finally switch to include\_tasks then it works.

Rgrds
> 1 votes

# Answer
Ansible is case sensitive, hence a != A

Therefore, the correct code should look like this:

```
- name: Configure NTP file
  template: src=ntp.conf.j2 dest=/etc/ntp.conf
  tags: ntp
  notify: Restart NTP

```
> 0 votes

---
Tags: ansible
---


# Question
Title: `UnauthorizedResourceAccessException` when calling `aws workdocs create-user`
When calling `aws workdocs create-user`, I'm getting `UnauthorizedResourceAccessException` or `UnauthorizedOperationException`. I believe I have all of the permissions required. What could be causing this?

# Answer
While the WorkDocs API describes `create-user` as creating a user in the Directory Service (and does), it also activates the user as a WorkDocs user. As a result, it can only be called when the directory is associated with WorkDocs (i.e. when the directory is "enabled" in WorkDocs). These errors can occur when WorkDocs is not enabled.

These errors can arise, for example, if you're trying to create users for WorkSpaces using the WorkDocs API since a comparable API isn't available for WorkSpaces. To enable WorkDocs, do one of the following:

* Use `EnableWorkDocs` when registering the Directory with WorkSpaces
* Manually create a site on the WorkDocs page

WARNING: Even if you allocate no storage, it appears these users will incur WorkDocs charges unless you also `deactivate_user`.
WARNING: `EnableWorkDocs` is not fully reversible from the API i.e. you cannot delete the Directory Service from the API if a WorkDocs site exists and the site cannot be removed from the API (likely for data protection reasons)
> 1 votes

---
Tags: aws-cli, aws-cloudformation
---


# Question
Title: Running AWS lambda in serial (for a given parameter)
I'm not sure if this is the best place to ask this. But we've currently got something in Microsoft Azure where we have their serverless functions (Azure Functions) running in serial based off a particular key, say a user ID.

So I want these functions to be called in parallel for different users, but in serial for each individual user. I spent ages trying to do this with locking behaviour in the function logic but it was always causing issues until I discovered with Azure, I simply make my Azure Function trigger from Service Bus, and use the `userId` as the Service Bus session id. Then Azure makes sure only one 'session' is running at once.

We're looking at moving stuff over to AWS and Lambda functions, but there's so many services - AWS Firehose, Simple Queue, CloudWatch Events, Step Functions, Kinesis Data Stream. In my initial searching I couldn't find any references to something similar (but everything is called different things, and I can spend hours reading about a service and find it's totally nothing to do with what I'm trying to accomplish)

Does anyone know if any of them offer that kind of "serial for a given property" kind of behaviour out of the box?

# Answer
> 2 votes
Use SQS with Lambda to accomplish this.

SQS supports 2 types of queues: Standard & FIFO. FIFO is appropriate for your use case. Create a FIFO queue & attach a Lambda function trigger to it. This causes the Lambda function to be triggered when a message arrives in the SQS queue. When you send a message to a FIFO queue, set the `userId` as the message group ID. This will ensure that messages from an individual user are only processed serially but messages from different users are processed in parallel.

Here's the relevant section from AWS Lambda documentation:

> For FIFO queues, Lambda sends messages to your function in the order that it receives them. When you send a message to a FIFO queue, you specify a message group ID. Amazon SQS ensures that messages in the same group are delivered to Lambda in order. Lambda sorts the messages into groups and sends only one batch at a time for a group. If the function returns an error, all retries are attempted on the affected messages before Lambda receives additional messages from the same group. Your function can scale in concurrency to the number of active message groups.

---
Tags: amazon-web-services
---


# Question
Title: How to programmatically enable and disable SQS triggering AWS Lambda?
I have a requirement where I want to stop my lambda from receiving messages from SQS. And I want to implement this like an API which can toggle between the disabling and enabling of the AWS Lambda getting triggered by SQS. I tried looking for options but there were suggestions telling to invoke the permission which I don't find feasible.

# Answer
> 2 votes
Call Lambda's UpdateEventSourceMapping API with the `{"Enabled":true/false}` request body.

If, for some reason, you want your own API for this, you can create one in API Gateway as described below.

Before you begin, you need the UUID of the event source mapping you wish to enable/disable. Use the AWS CLI to get this (it isn’t shown on the UI):

```
aws lambda list-event-source-mappings --function-name my-func

```

```
{
    "EventSourceMappings": [
        {
            "UUID": "b962dcee-173c-47r1-9600-5c6d81catf85",
            "BatchSize": 1,
            "EventSourceArn": "arn:aws:sqs:us-east-1:123456789012:my-queue",
            "FunctionArn": "arn:aws:lambda:us-east-1:123456789012:function:my-func",
            "LastModified": 1595058588.141,
            "State": "Enabled",
            "StateTransitionReason": "USER_INITIATED"
        }
    ]
}

```

Next, either create a new API in API Gateway or add a resource & a GET method to your existing API. Configure it as shown below. The UUID you got above is highlighted below:

The execution role shown above must allow API Gateway to invoke Lambda APIs.

Scroll further down & provide this mapping template:

```
Content-Type: application/json

```

```
{
    "Enabled" : $input.params("Enabled")
}

```

Save & deploy to a stage. Now you can hit this API with the query param `Enabled=true` / `Enabled=false` to enable/disable your Lambda's SQS trigger. Here's a test done from the API Gateway console:

For a detailed screenshot-guided walkthrough of this entire process, see my blog post.

---
Tags: amazon-web-services, aws-lambda, aws-iam, serverless, amazon-sqs
---


title: SSH connection to jenkins server

text:
# Question
Title: SSH connection to jenkins server
I have a absh script which connects and exexutes programs from my local to remote server and now this should be executed from jenkins build so for this jenkins is taking continous changes from git for this script but my build is failing with below error:

Warning: Identity file /c/Users/D35N/Downloads/GitHubProjeckt/Automation/GitHubProjeckt/newharbortestkey not accessible: No such file or directory. Host key verification failed. Build step 'Execute shell' marked build as failure

I have tried to set ssh connectivity in my jenkins server for this remote server incredentials and ssh plugins but using my key i am getting an error cant connect to the server.

when i run the script from local it works fine and my ssh connection gets established properly using SSH key.

jenkins server is also integrated or installed on my kuberenets cluster which is my remote server in this case and bash script should execute on this kuberenets cluster.

Can anyone please suggest how this problem can be solved?

# Answer
> 1 votes
Your configuration refers to a file that is local on your system, but does not exist on the jenkins server. The error message is trying to tell you this. `/c/Users/D35N/Downloads/GitHubProjeckt/Automation/GitHubProjeckt/newharbortestkey` is not on the jenkins server. You need to make this file available on the jenkins server by copying it up there, and it probably shouldn't be in your personal `Downloads` folder.

You may want to configure your script to look for a series of files and use the first one that exists. This will let you have the file at different locations on your development box versus the server and it will still work in either.

---
Tags: jenkins, kubernetes, github, shell-script, bash
---