# Question
Title: How to find logs when submitting resource type to Cloudformation Registry?
I am developing a private resource type for AWS Cloudformation Registry. I have designed my model schema and developed my handler code, submitted it, and even successfully deployed a stack with my very own private resource type. Yay.

What i need to do now is inspect the logging thereof. As I had generated the scaffolding using the cfn init command, i merely added logging entries to the existing logger object.

e.g.

```
# Use this logger to forward log messages to CloudWatch Logs.
LOG = logging.getLogger(__name__)
TYPE_NAME = "Myself::Test::Resourceful"

resource = Resource(TYPE_NAME, ResourceModel)
test_entrypoint = resource.test_entrypoint

@resource.handler(Action.CREATE)
def create_handler(
        session: Optional[SessionProxy],
        request: ResourceHandlerRequest,
        callback_context: MutableMapping[str, Any],
) -> ProgressEvent:
    model = request.desiredResourceState
    progress: ProgressEvent = ProgressEvent(
        status=OperationStatus.IN_PROGRESS,
        resourceModel=model,
    )
    # TODO: put code here

    LOG.info('Creating....')

```

According to the literature,

> When you register a resource type using cfn submit, CloudFormation creates a CloudWatch log group for the resource type in your account. This enables you to access the logs for your resource to help you diagnose any faults. The log group is named according to the following pattern:
>
> /my-resource-type-stack-ResourceHandler-string
>
> Now, when you initiate stack operations for stacks that contain the resource type, CloudFormation delivers log events emitted by the resource type to this log group.

When submitting my resource type however (and even deploying it), a cannot see any LogGroup created in CloudWatch whatsoever. There is clearly something i am missing here.

**Please help me understand how to find the logging for my private Cloudformation registry resource types.**

Of course, i will be happy to provide any additional info needed. Thank you!

# Answer
Got it.

Thanks to @maslick, all i had to do was **explicitly set an appropriate logging level**.

```
# Use this logger to forward log messages to CloudWatch Logs.

LOG = logging.getLogger(__name__)
LOG.setLevel(logging.INFO)            # <- this line was missing

```

When deploying stack, log groups appears. Yay.
> 0 votes

---
Tags: amazon-web-services, aws-cloudformation
---


# Question
Title: How to curl elastic or kibana api for alerts?
I am learning to use the ELK stack. Both kibana and elasticsearch are installed on my localhost. I just learnt how to install metricbeat and how to set up alerts. When setting up the alerts, I used the index connector, and called my index testconnector.

I see the alerts showing up in my web browser when i go to `http://localhost:5601/app/observability/alerts`.

Is there a way for me to get the same information via REST API? i tried all these endpoints but they all say "no handler found for uri"

```
curl -X GET -k -u elasticuser:elasticpass "http://localhost:9200/api/index_management/indices"
curl -X GET -k -u elasticuser:elasticpass "http://localhost:9200/api/alert"
curl -X GET -k -u elasticuser:elasticpass "http://localhost:9200/api/alert/_search"
curl -X GET -k -u elasticuser:elasticpass "http://localhost:9200/api/alert/_find"
curl -X GET -k -u elasticuser:elasticpass "http://localhost:9200/alert/_search"
curl -X GET -k -u elasticuser:elasticpass "http://localhost:9200/alert/_find"
curl -X GET -k -u elasticuser:elasticpass "http://localhost:9200/kibana/api/alerting"
curl -X GET -k -u elasticuser:elasticpass "http://localhost:9200/testconnector/_search"

```

If anyone can tell me how to get the alerts (not the rules) through a REST API, that will be great!

# Answer
I figured it out. First, you can get a list of all the rules from **kibana api** (as opposed to elastic api) like this:

```
curl -X GET -k -u elasticuser:elasticpass "http://localhost:5601/api/alerting/rules/_find"

```

The list of rules also contains some execution status of each rule. Meaning some are active, some are ok, some are recovered etc... So right off the bat, I have what I need related to the alerts associated with each rule.

If I need more details, I can grab the rule id and get more details with this curl

```
curl -X GET -k -u elasticuser:elasticpass "http://localhost:5601/api/alerting/rule/<rule id>"

```

All the API endpoints for rule and alert management can be found here: https://www.elastic.co/guide/en/kibana/current/alerting-apis.html
> 1 votes

---
Tags: elasticsearch, elk
---


# Question
Title: WinRM / WSMan - getting SSL handshake error / TCP RST when connecting with IP address, correct connection when connecting with hostname
While using Ansible to manage some AWS based Windows machines, when connecting using the IP as destination, on some machines I'm getting an SSL Error

> credssp: HTTPSConnectionPool(host='18.217.233.26', port=5986): Max retries exceeded with url: /wsman (Caused by SSLError(SSLError(\\"bad handshake: SysCallError(-1, 'Unexpected EOF')\\",),))", "unreachable": true}

In wireshark it looks like TCP RST is sent by server

When connecting using hostname or FQDN the connection succeeds.

I don't see any relevant log in the event logs on WSMan side.

Anybody have any experience with setting up winrm for Ansible and can assist?

# Answer
I faced the same error .. You probably want to run ConfigureRemotingForAnsible.ps1 script with `-ForceNewSSLCert` option on your remote machines.

```
.\ConfigureRemotingForAnsible.ps1 -ForceNewSSLCert

```
> 1 votes

# Answer
If that doesnt work, check your TLS1.3 registry key `Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.3\Server`

The actual error for me was TLS 1.3 was set enabled (1), in registry, but was not set DisabledByDefault. setting that to 1 resolved the long looming ansible ssl connection error.

You can also delete the TLS1.3 key, so that its not explicitly set.

```
ansible_connection: winrm
ansible_winrm_scheme: https
ansible_port: 5986 #didnt break it, leaving this enabled
ansible_winrm_cert_pem: ..
ansible_winrm_cert_key_pem: ...
ansible_winrm_transport: certificate
ansible_winrm_server_cert_validation: ignore # maybe because i WAS USING IP, not hostname, also the cert  CN is "ansible" anyway, not matching any hostname.

```

https://community.tenable.com/s/article/How-to-view-and-change-the-Windows-Registry-Settings-for-the-SSL-TLS-Protocols-on-a-Windows-Host
> 0 votes

---
Tags: ansible, windows
---


# Question
Title: Docker port mapping across several IPs on same NIC results in error
I have docker stack running on a Raspberry Pi with docker-compose. It’s been going nicely for about a year now, but now I want to bring it to the next level.

The issue is, the stack contains a few services with website, which I access from the network. Obviously, that means I always have to remember the ports of each of those services, which has become tiresome after a while. And I do this from several devices in the network and sometimes several browsers on each device (I have my reasons :) ) so creating bookmarks isn’t an option.

So I thought I would solve this via DNS, as I already have pi-hole running in a container nicely as both a DNS and a DHCP server. For this to work I need, among others, the containers to listen to port 80.

Unfortunately, this is where the trouble shows.

The first thing I tried was to use ipvlan and macvlan networks. And it kind of works. But they still don’t solve the issue of remembering the ports, as the containers keep their current one. Port mapping doesn’t help here because of the reason below.

Another issue with this is that, as the containers also communicate to each other and there is a lot of existing configuration there, with IPs and ports, I have to connect them to 2 networks, a bridge for the inter-container communication, and an ipvlan for the outside communication. However, the port mapping is only one, and I don’t understand how it works when a container is connected to 2 networks. I thought this should have worked, but it didn’t.

So the next idea I had was to create several IPs on the host, then use the syntax “host\_IP:port:container\_port” to map as necessary.

So, for example, I have on the host

```
eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether dc:a6:32:07:f8:85 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.10/24 brd 10.0.0.255 scope global noprefixroute eth0
valid_lft forever preferred_lft forever
inet 10.0.0.50/24 scope global secondary eth0
valid_lft forever preferred_lft forever
inet 10.0.0.51/24 scope global secondary eth0
valid_lft forever preferred_lft forever
inet 10.0.0.52/24 scope global secondary eth0
valid_lft forever preferred_lft forever
inet6 fe80::185a:8f0e:78b0:a110/64 scope link
valid_lft forever preferred_lft forever

```

and then in docker-compose.yml I have (only relevant part):

```
grafana:
ports:
- “3000:3000”
- “10.0.0.50:80:3000”

nodered:
ports:
- “1880:1880”
- “10.0.0.51:80:1880”

```

Unfortunately, this throws a bunch of errors when I run docker-compose up -d

```
ERROR: for grafana Cannot start service grafana: driver failed programming external connectivity on endpoint grafana (b70ca36d6f0953ca16e046c7af561a638ce2fe537417600e6c7f7796fbf228c0): Error starting userland proxy: listen tcp4 10.0.0.50Recreating 922d1912d260_nodered … error

ERROR: for 922d1912d260_nodered Cannot start service nodered: driver failed programming external connectivity on endpoint nodered (6676c921fc83a1a521db50bce6ef128ec7811b4dae4d69cf0597b9f118199bfc): Error starting userland proxy: listen tcp4 10.0.0.51:80: bind: address already in use

ERROR: for grafana Cannot start service grafana: driver failed programming external connectivity on endpoint grafana (b70ca36d6f0953ca16e046c7af561a638ce2fe537417600e6c7f7796fbf228c0): Error starting userland proxy: listen tcp4 10.0.0.50:80: bind: address already in use

ERROR: for nodered Cannot start service nodered: driver failed programming external connectivity on endpoint nodered (6676c921fc83a1a521db50bce6ef128ec7811b4dae4d69cf0597b9f118199bfc): Error starting userland proxy: listen tcp4 10.0.0.51:80: bind: address already in use
ERROR: Encountered errors while bringing up the project.

```

It sounds a bit like docker can not share the same port even when it belongs to several IPs on the host. I have a limited understanding of this, but from what I read, they shouldn’t conflict. A NIC with several IPs assigned can handle all ports on each IP, as the port belongs to TCP and UDP. I googled this a lot and haven't been able to find any solution

I will appreciate any suggestions.

# Answer
> 1 votes
There are several ways to address this particular problem.

## Using a dynamic proxy

Using a dynamic proxy is nice because you only need a single ip address and port. Incoming requests are directed to the appropriate container based on metadata about the request (such as the hostname, or particular headers, or the request path, etc).

This is actually what I'm doing at home: I have a number of web services running on my local system and I want to access them by name. I'm handling this by using Traefik, a dynamic proxy designed to work well with systems like Docker. Configuration is primarily via Docker labels.

So for example, if I start the Traefik container like this:

```
docker run -d --name traefik \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 80:80 \
    -p 127.0.0.1:8080:8080 \
    docker.io/traefik:v2.8 \
    --api.insecure=true \
    --providers.docker

```

(That `-p 127.0.0.1:8080:8080` is exposing the Traefik dashboard on http://localhost:8080.)

Then I can make a web service available at `grafana.local` by starting it like this:

```
docker run -d --name grafana \
    -l 'traefik.http.routers.grafana.rule=Host(`grafana.local`)' \
  docker.io/nginx:mainline

```

Now any host on my network can access that service at http://grafana.local (assuming I have arranged for the hostname "grafana.local" to map to the address of the host running the traefik proxy).

I've used `docker run` commands here, but you can obviously set this all up using `docker-compose` instead.

## Using multiple ip addresses

> It sounds a bit like docker can not share the same port even when it belongs to several IPs on the host.

That is in fact not true. Consider this; on my host I have the following addresses assigned on `eth0`:

```
$ ip -4 addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    altname eno2
    altname enp0s31f6
    inet 192.168.1.175/24 brd 192.168.1.255 scope global dynamic noprefixroute eth0
       valid_lft 83583sec preferred_lft 83583sec
    inet 192.168.1.180/24 scope global secondary eth0
       valid_lft forever preferred_lft forever
    inet 192.168.1.181/24 scope global secondary eth0
       valid_lft forever preferred_lft forever
    inet 192.168.1.182/24 scope global secondary eth0
       valid_lft forever preferred_lft forever

```

Let's start up a few containers:

```
docker run -d --name darkhttpd  -p 192.168.1.180:80:8080 docker.io/alpinelinux/darkhttpd
docker run -d --name nginx -p 192.168.1.181:80:80 docker.io/nginx:mainline
docker run -d --name httpd -p 192.168.1.182:80:80 docker.io/httpd:2.4

```

We can see that the containers are all running successfully:

```
$ docker ps
CONTAINER ID   IMAGE                   COMMAND                  CREATED          STATUS          PORTS                        NAMES
283c8e061ebc   httpd:2.4               "httpd-foreground"       16 seconds ago   Up 15 seconds   192.168.1.182:80->80/tcp     httpd
0d2960c07212   nginx:mainline          "/docker-entrypoint.…"   18 seconds ago   Up 18 seconds   192.168.1.181:80->80/tcp     nginx
68e56ed8c180   alpinelinux/darkhttpd   "darkhttpd /var/www/…"   18 seconds ago   Up 18 seconds   192.168.1.180:80->8080/tcp   darkhttpd

```

And I get different services depending on which IP address I use:

```
$ curl 192.168.1.180
<html>
<head>
 <title>/</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
</head>
...
$ curl 192.168.1.181
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
...
$ curl 192.168.1.182
<html><body><h1>It works!</h1></body></html>

```

---

Both options work. I like the proxy model, because it gives me a lot of flexibility (if instead of separate hostnames I want to mount services on different paths -- e.g., I want `monitoring.local/grafana` to go to grafana and `monitoring.local/prometheus` to go to prometheus -- I can do that). It also means I don't need to allocate a new ip address every time I have a new service.

Using multiple IP addresses is arguably simpler to set up, and as long as your configuration isn't changing all that often it doesn't really introduce much overhead.

---

It's hard to tell exactly what's going on with your specific configuration because you haven't really provided enough information to reproduce the behavior you're asking about. If the information in this answer isn't enough to solve your problem, I'd be happy to take a closer look if you can update your question to include an \[mcve\]: ideally a simplified `docker-compose.yaml` file that we can copy and run locally to reproduce the problem.

---
Tags: docker, docker-compose, networking
---


# Question
Title: Gather kubectl logs data to an external service
When I run `kubectl logs MyPodNameHere` I get back the standard-out of that pod.

My company has a central logging service for all our applications and I would like to send the `logs` output to that service. (Via an `HTTPS` endpoint.)

But I can't seem to find a way to hook into a Kubernetes API to get that working. **Is there a Kubernetes API or other integration point to do this?**

NOTE: I am also using Istio. I had hopped that I could use that for this, but I can only see stuff about sending "access logs" in the documnation on Istio. But it seems to me that it should be possible via Istio as well.

**Anyone know a way to get this `logs` data sent to an HTTPS endpoint?**

# Answer
> 2 votes
There are many products that will ship Kubernetes logs off to a central log server. The kubernetes documentation has a section on logging that is a good place to start.

There are a number of existing projects that are designed to ship your local off to a central location:

Etc.

And following some of the suggestions in the Kubernetes documentation I linked to earlier you could probably write your own (or reconfigure your existing host-based solutions to support your Kubernetes environment as well).

---
Tags: kubernetes, istio, kubernetes-on-premises
---


# Question
Title: How to access elements of a variable in ansible
I'm trying to access a specific value in a list of data and it fails, can't find the ipa variable.

Here is a short bit of the boulder.yaml vars file, yes, dual homed. Network configurations require it:

```
service:
  control:
  - name: "bldr0cuomdev01"
    ipa: 192.168.101.81
  - name: "bldr0cuomdev02"
    ipa: 192.168.101.82
management:
  control:
  - name: "bldr0cuomdev11"
    ipa: 10.100.78.81
  - name: "bldr0cuomdev12"
    ipa: 10.100.78.82

```

And I'm trying to use the following (much shorter) jinja2 template (nodes.j2) to create a file:

```
{{ service.control.ipa[0] }},{{ management.control.ipa[0] }}

```

I'm basically getting the error that ipa is not found.

Since there are two sections, having it be a loop isn't going to work but if I test just the service section as a loop, it does work. So it's able to find ipa, but my method isn't valid for some reason.

```
{% for s in service.control %}
{{ s.ipa }}
{% endfor %}

```

I saw another suggestion of using .0 instead of a bracketed array

```
{{ service.control.ipa.0 }},{{ management.control.ipa.0 }}

```

And also tried using a quoted zero

```
{{ service.control.ipa['0'] }},{{ management.control.ipa['0'] }}

```

But neither work. Searching on the 'net is common enough that my search is not finding anything relevant.

# Answer
> 2 votes
For example,

```
result: "{{ service.control.0.ipa }},{{ management.control.0.ipa }}"

```

gives

```
result: 192.168.101.81,10.100.78.81

```

---
Tags: ansible, jinja
---


# Question
Title: How to tell helm not to deploy a resource or remove it if a value is set to "false"?
I am working on an HPA template that will be applied only if the `enabled` value is set to true. Currently when setting `enabled` to false, it will create an empty object in yaml. This is then applied with an error stating that there is no `apiVersion` defined. How can I tell helm to not apply the HPA template if the value is set to `false` our skip the resource templating?

values.yaml:

```
# hpa
hpa:
  enabled: false
  maxReplicas: 10
  minReplicas: 2
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 70

```

hpa.yaml:

```
{{- template "common.hpa" (list . "service.deployment") -}}
{{- define "service.deployment" -}}
{{- end -}}

```

\_hpa.yaml:

```
{{- define "common.hpa.tpl" -}}
{{ if .Values.hpa.enabled }}
---
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
  creationTimestamp: null
  name: {{ required "serviceName value is required" $.Values.serviceName }}
  namespace: {{ required "namespace value is required" $.Values.namespace }}
spec:
  maxReplicas: {{ .Values.hpa.maxReplicas }}
  minReplicas: {{ .Values.hpa.minReplicas }}
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: {{ required "serviceName value is required" $.Values.serviceName }}
  metrics:
{{ toYaml .Values.hpa.metrics | indent 4 }}
{{- end -}}
{{- end -}}

{{- define "common.hpa" -}}
{{- include "common.util.merge" (append . "common.hpa.tpl") -}}
{{- end -}}

```

\_util.yaml

```
{{- /*
common.util.merge will merge two YAML templates and output the result.
This takes an array of three values:
- the top context
- the template name of the overrides (destination)
- the template name of the base (source)
*/}}
{{- define "common.util.merge" -}}
{{- $top := first . -}}
{{- $overrides := fromYaml (include (index . 1) $top) | default (dict ) -}}
{{- $tpl := fromYaml (include (index . 2) $top) | default (dict ) -}}
{{- toYaml (merge $overrides $tpl) -}}
{{- end -}}

```

output from running `helm template`

```
---
# Source: service/templates/hpa.yaml
{}

```

error message when doing a `helm install`:

```
Error: UPGRADE FAILED: error validating "": error validating data: [apiVersion not set, kind not set]
helm.go:84: [debug] error validating "": error validating data: [apiVersion not set, kind not set]

```

# Answer
You've specified:

```
{{ if .Values.hpa }}
---
...

```

To skip that code when enabled is false, you'd need:

```
{{ if .Values.hpa.enabled }}
---
...

```
> 1 votes

---
Tags: kubernetes, google-cloud-platform, helm, yaml
---


# Question
Title: Why is AWS ALB not talking to an ingress controller?
I'm taking TechnoWorld Kubernetes Administration class, where we learn to create a Kubernetes cluster from scratch using one Ubuntu EC2 for master node, and two Ubuntu EC2s for worker nodes. The course guides us into installing the different Kubernetes components - containerd, kubeadm, kubelet, kubectl, Weavnet etc. - on each pertinent EC2. We use NGINX as our simple application that are running on our worker node pods. I have the infrastructure set up where I can actually, for example, do a `curl <pod IP address>` from my master node and get the "Welcome to nginx" greeting as a response. So far so good. I even created a Terraform script to set this up. I even setup test Kubernetes LoadBalancer service that's accessible to the outside world, put it's `IP:port` on the browser and see "Welcome to nginx" proving that the networking (VPC, subnet, security group) is correct.

Here's the traffic flow of the **fundamental** infrastructure we're trying to build.

Internet-facing AWS ALB(port 80) -\> ingress controller LoadBalancer service(port 80:32111 mapping) -\> ingress -\> ClusterIP service -\> EC2 worker with NGINX pod

So a user puts an ALBs DNS name on the browser and gets the "Welcome to NGINX" greeting.

However with my setup, when I put the ALB's DNS name, I get a "This site can't be reached" message.

I set up my internet-facing ALB to serve port 80, forward to a target group on port 32111 (ingress controller port) that has the 2 EC2 worker nodes registered as targets.

Any pointers on how I can debug this seemingly simple/fundamental setup?

TIA

# Answer
Assuming everything is running and working fine on AWS, the key pieces of information you mentioned above are the DNS record and the Ingress.

I would assume you have defined the Ingress Name based virtual hosting.

Then you'd also need to add, in AWS Route53, the appropriate DNS records to point the the ALB.
> 0 votes

---
Tags: kubernetes, amazon-web-services, amazon-ec2
---


# Question
Title: Publish python package into private repository behind VPN
I'm using Github Actions to publish my python packages automatically. When working with PyPI everything works fine and providing different config to point on different repository is not an issue. But when repository (JFrong artifactory) is located behind VPN GH Actions is not able to push/fetch from there.

What is the right approach to deal with such kind of situation? Is there any kind of existing solution for this issue?

# Answer
So I found similar question (regarding installing, but works for publishing as well) https://stackoverflow.com/questions/73624685/install-python-package-from-private-artifactory-behind-vpn

Basically, you can either setup GH Actions runner within VPN or use proxy to access your resorces.

I have ended up using answer from comments

> my strategy would be to create a source code release zip file and publish it on GitHub and then download it from within the VPN to push it to the intern repository. github.com/marketplace/actions/create-github-release

and still run separate job for publishing into internal repository.
> 0 votes

---
Tags: python, aws-vpc, artifactory, github-actions, vpn
---


title: Azure Pipeline Parallelism option

text:
# Question
Title: Azure Pipeline Parallelism option
I am new to azure pipelines, started learning & in the process of creating my very 1st yaml pipeline.

My project is private, I am using a multi-stage templated pipeline, self-hosted as need to concurrently deploy a java web application to 7 VMs using mvn tomcat7 plugin run: command so as to run selenium automation tests in parallel across all the VMs. A template pipeline which is called 7 times to deploy to all the VMs is such that it needs to stay running as necessitated by the embedded tomcat instance on each of the VMs which in turn requires the ability to have parallelism.

Is it possible to have several parallel jobs running without turning my project from a private one to public ?

# Answer
> 1 votes
### Public project

Up to 10 free Microsoft-hosted parallel jobs that can run for up to 360 minutes (6 hours) each time No overall time limit per month

### Private project

One free job that can run for up to 60 minutes each time 1,800 minutes (30 hours) per mont. You can pay extra to get parallel jobs.

### Self Hosted Agent

You can just setup your own self hosted agents, it just a script you run on a VM, takes about 10min per agent plus of course you need to install the software you need on the agent(s) and then you don't have those restrictions.

---
Tags: azure, pipeline
---


title: How to test pipeline changes in Azure?

text:
# Question
Title: How to test pipeline changes in Azure?
I currently have a pipeline in Azure which consists of a bunch of YAML which eventually calls some terraform script to handle the deployment side.

What is the best way to handle dev testing of changes to the pipeline?

For example, i want to add a new task in the YAML. There doesnt seem to be a way to test this without actually performing and completing a pull request. I the cases where i do something wrong and the pipeline (say an invalid parameter value) fails i need to do a whole new checkin, push, pull request, merge and then wait for the pipeline to complete or fail.

There has to be a better way?

# Answer
> 1 votes
Not sure if this helps you? but if you edit it from the portal you can select the branch you are working on and there is a validate option

I also add a parameter to my pipelines,

```
 parameters:
  - name: DebugMode
    displayName: Skip Deployments and run in DebugMode
    type: string
    default: false
    values:
    - false
    - true

```

Which adds a radio button when you run the Pipeline

and then I have a condition to skip the actual deployment task for the debug mode so I can test the logic of my pipelines running them from teh branch I am working on before PR the change in to main.

```
         steps:
      - ${{ if eq(parameters.DebugMode, false) }}:
        - task: Deploy
          inputs:
            ....
      - ${{ if eq(parameters.DebugMode, true) }}:
        - task: PowerShell@2
          name: DummyDeployment
          displayName: 'Dummy Do Nothing'
          continueOnError: false
          inputs:
            targetType: Inline
            script: |
              echo "In Debug mode running Dummy Deployment"
              echo $buildNumber

```

---
Tags: terraform, git, azure, yaml, pipeline
---


# Question
Title: In jenkins how to restrict users to select first default element with other options in extended choice parameter
I have a jenkins job with multi select extended choice parameter. There are list of elements in a parameter. So, my requirement is I want to allow users to select multiple parameters excluding first element in a parameter. Means user should not able to select first element with other elements in a parameters. I am using jenkinsfile to create parameter.

Like shown above, users should not able to select 'None' with any other element in a parameter. Does anyone know how to do this?

# Answer
It's probably easier to create two separate parameters. E.g.:

* Run tests? (boolean parameter)
* If running tests, select tests (multi-select choice parameter)

Then you could code your job to skip all tests if the first parameter is false, otherwise it would run the selected tests.

If you want to get really fancy, you can use the Active Choices plugin to dynamically show the list of tests parameter only if the "run tests" parameter is also selected.
> 0 votes

---
Tags: jenkins, jenkins-pipeline, groovy
---


title: cni0 already has an ip address

text:
# Question
Title: cni0 already has an ip address
I've got a k8s (on-premise) cluster with 1 master and 3 workers, all ubuntu, latest LTS version.

2 workers are running fine, but pods getting scheduled to one stay in the ContainerCreating state with the error.

> Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "1234" network for pod "clean-updater-hxknv": networkPlugin cni failed to set up pod "clean-updater-hxknv\_default" network: failed to set bridge addr: "cni0" already has an IP address different from 10.244.6.1/24

cni0 has an ip of:

cni0: \<BROADCAST,MULTICAST,UP\> mtu 1500 qdisc noqueue state UP group default qlen 1000 inet 10.244.3.1/24 brd 10.244.3.255 scope global cni0

I've tried kubeadm reset and join again on the node but it didnt help. How to fix this error?

# Answer
I had this happen to me when my flannel and cni0 inet addresses were out of sync. Bringing down the cni0 bridge made it resync for me:

```
ifconfig cni0 down
brctl delbr cni0

```

Or if that doesn't work:

```
sudo ip link delete cni0 type bridge

```

Hope that helps.
> 2 votes

---
Tags: docker, kubernetes
---


# Question
Title: failed to acquire lease: subnet "10.244.0.0/16" specified in the flannel net config doesn't contain "192.168.0.0/24"
I'm trying to configure Kubernetes with web dashboard. I use these steps:

```
swapoff -a

```

Remove following line from `/etc/fstab`

```
/swap.img       none    swap    sw      0       0

```

then run these shell commands:

```
sudo apt update
sudo apt install docker.io
sudo systemctl start docker
sudo systemctl enable docker
sudo apt install apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" >> ~/kubernetes.list
sudo mv ~/kubernetes.list /etc/apt/sources.list.d
sudo apt update
sudo apt install kubeadm kubelet kubectl kubernetes-cni

sudo kubeadm init --pod-network-cidr=192.168.0.0/16

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

```

I applied this configuration:

```
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: flannel-fix
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: flannel
subjects:
  - kind: ServiceAccount
    name: flannel
    namespace: kube-flannel

```

```
kubectl create -f <file>

```

But I get:

```
Error registering network: failed to acquire lease: subnet "10.244.0.0/16" specified in the flannel net config doesn't contain "192.168.0.0/24" PodCIDR of the "ubuntukubernetis1" node

```

Do you know what should be the correct configuration?

# Answer
I figured it out by reading the documentation:

> Take care that your Pod network must not overlap with any of the host networks: you are likely to see problems if there is any overlap.

Flannel uses 10.244.0.0/16 and can't run on the same as your host... it creates new interfaces when implemented. So when you run this init command it should always be using 10.244.0.0/16.

```
sudo kubeadm init --pod-network-cidr=10.244.0.0/16

```

Once I did this, things started working as expected.
> 1 votes

---
Tags: kubernetes
---


# Question
Title: Is it possible to trigger an AWS Lambda function directly from CloudWatch alarm?
Currently, for triggering a lambda function in response to a CloudWatch alarm, I need to route the alarm via SNS, like this:

```
AWS CloudWatch Alarm --> Send to a topic in SNS --> SNS topic triggers the lambda function

```

**So, is there a way I can do it directly, without SNS in the middle?** \[Reason: Less touch points and a bit of cost optimization\].

CloudWatch events trigger lambda but there is no way I can trigger alarms via it. The only options available in CloudWatch events for SQS are:

>

# Answer
As far as I know, it is not possible. The the offcial AWS Lambda documentation says it support CloudWatch **Events**, but no mention of CloudWatch **Alarms** (and CW Events != CW Alarms)

What you are currently doing, sending the alarms on a SNS and using listening on a topic seems the way to go. In fact, CloudWatch Alarms only outputs to SNS so far.
> 10 votes

# Answer
It is possible to trigger Lambda function from CloudWatch alarm, by using EventBridge https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch-and-eventbridge.html
> 1 votes

---
Tags: aws-lambda, amazon-cloudwatch, amazon-sqs
---


# Question
Title: How to put production-like data into version control
One goal of DevOps is to create reproducible, production-like environments from source-code repositories. To achieve this, I believe it's necessary to also restore all the **data** that is needed to setup the environment, so that automated & manual tests can be run there.

For example, let's take a CMS product that provides content (labels, images, rich-text) to a user-facing web application. We want to run integrations tests or explorative UI tests against that web application using production-like data from the CMS. Due to the nature of the product, the data itself can be manipulated at any time by users using the UI of the CMS.

There are two main use cases

* Developers create example content or add required label translations for new features
* Content managers publish new content for production, which may include static content like label translations

In a typical setup, there are at least two environments, let's say `staging` and `production`. Due to the use cases described above, the CMS databases where the contents are stored will deviate quite quickly between these environments.

Without taking any measures here, after a while, the two databases will be completely out-of-sync and the `staging` database will not be "production-like" anymore. Additionally, each release of a new feature will require manual steps to add data like label translations into the `production` database, a step that is neither verified nor tested before the release.

So my question is: What is a good way to bring new "baseline" data from a `staging` database into a `production` database without overriding the changes there. And was is a good way to bring back data from `production` into the `staging` environment, to provide production-like data for testing purposes. Ideally, all automated and all part of version-control.

Side Note: If we say "different CMS content doesn't change anything about testing", let's think of another use-case where the data behaves even more like configuration. For example, payment or delivery options that are offered on an e-Commerce site.

# Answer
it depends a bit what kind of cms you use, most cms offer a way to package data in to packages that you can check in to git and deploy to databases almost like code to webapps.

For example, we work with Sitecore and it lets you use TDS or Unicorn, other big CMS use different tools but mostly the same concept.

The tools let you pick items from your Database and serialise them in to code/text files that are packaged and check in to Git, lets say you have a copy of the database on your Development Workstation, you make some updates to the database and you serialise and package the change with TDS in to a package that you check in to your Development branch.

When you deploy the development branch to your Staging environment the TDS package is applied and makes the same updates to the database of your Staging environments.
> 1 votes

---
Tags: versioning, environments, data
---


# Question
Title: can't run default/hello-minikube application in browser
I am new to devops stuff and just started learning Kubernetes. I installed minikube and kubernetes-cli with homebrew and then setup docker drivers

```
minikube start --driver=docker
minikube config set driver docker

```

I created hello-minikube sample application

```
$kubectl create deployment hello-minikube --image=k8s.gcr.io/echoserver:1.4
$ kubectl expose deployment hello-minikube --type=NodePort --port=8080
Error from server (AlreadyExists): services "hello-minikube" already exists

```

and then tried to start it

```
❯ minikube service hello-minikube
|-----------|----------------|-------------|---------------------------|
| NAMESPACE |      NAME      | TARGET PORT |            URL            |
|-----------|----------------|-------------|---------------------------|
| default   | hello-minikube |        8080 | http://192.168.49.2:31510 |
|-----------|----------------|-------------|---------------------------|
  Starting tunnel for service hello-minikube.
|-----------|----------------|-------------|------------------------|
| NAMESPACE |      NAME      | TARGET PORT |          URL           |
|-----------|----------------|-------------|------------------------|
| default   | hello-minikube |             | http://127.0.0.1:52658 |
|-----------|----------------|-------------|------------------------|
  Opening service default/hello-minikube in default browser...
❗  Because you are using a Docker driver on darwin, the terminal needs to be open to run it.

```

But in browser it just see **The connection was reset** error so it doesn't load the application. What do you think went wrong?

# Answer
> 0 votes
On inspection

```
$ kubectl describe pod hello-minikube

$ kubectl get nodes

```

I found that I was running x86 container on ARM Macbook M1. Installed on x86 Linux and everything works as expected

https://github.com/kubernetes/minikube/issues/13129

---
Tags: docker, kubernetes, minikube
---


# Question
Title: How should I organize a large number of unrelated Infrastructure as Code projects?
I work at a municipal government with many departments. Teams from these departments request new servers for specific uses/applications. Our environment is currently using VMware VSphere (~1500 VMs, 2 datacenters, 50 hosts/blades) and we'll be migrating gradually to Azure over the next few years.

I'm just starting with Terraform and can build VMs with it, one at a time. Should I create one Terraform project per server? Or is it worth it to group them somehow? Most of the requests are for one or two servers at a time; sometimes up to four at a time. Each request tends to be unrelated to any previous requests.

Is there a source that covers the theory of organizing something on this scale? Or should I just accept that we'll have to maintain 1500 sets of TF project files?

# Answer
I assume this is what Terragrunt would help to solve. It adds a new layer where you can easily manage complex infrastructure or keep your work DRY.
> 0 votes

# Answer
I'll give this a shot because I think it's a very common situation which could use a bit of analysis.

I imagine that due to the large number of internal customers, you don't want to treat all of them as special cases, but instead provide a kind of front-office which they can interact with and order the components they need to build out the service that they want.

Let's start at the end:

> Is there a source that covers the theory of organizing something on this scale?

I think what you need here is to apply the concept of a Service Portfolio. Instead of offering "VM's", try to understand what **components** your internal customers will need, and focus on building an inventory of that instead. Doing this will allow you to focus on the quality of the components, rather than the quality of the finished product. This hearkens back to the Deming principle of "build quality in".

In a well-designed scenario, you will end up having to maintain a handful of components, which will be mapped to a handful of well-architected patterns, instead of a few hundred custom solutions.

> Our environment is currently using VMware VSphere (~1500 VMs, 2 datacenters, 50 hosts/blades) and we'll be migrating gradually to Azure over the next few years.

When it comes time to evolve the environment, from VMWare to a cloud, you have to build new modules for the new environment, which do the same thing as their cousins in the old environment. This will allow you to build common interfaces to underlying components so that the users of these components don't have to care about the difference. So, you could have a Terraform module for "A VM with a private network and firewall" which is implemented one way for VMWare and another for Azure, but provides the same functionality to the user. Hashicorp's page on "module composition" gives a good overview of the considerations when combining modules and how to design them for re-usability.

The next concept you could use is that of **self-service**.

With a set of architectures and components, application owners can pick them the shelf and deploy them with a high-degree of certainty that they will work, and you will be able to provide support for them, because they will fit into well-known categories (or be declared out of scope for support -- similar to voiding the warranty).

With that out of the way, let me take a stab at answering the actual questions:

> Should I create one Terraform project per server?

You should create a Terraform module for a pattern, and allow your customers to Terraform your datacentre with them. If you do this on behalf of your customers, then the Terraform definition can reside alongside the code for the actual application in the same repo, or in a sister repo containing only the Terraform definition. This will allow you to create pipelines for specific applications and destroy them cleanly when it comes time to decommission.

> Or is it worth it to group them somehow? Most of the requests are for one or two servers at a time; sometimes up to four at a time. Each request tends to be unrelated to any previous requests.

The "grouping" here I interpret as "instantiations of a similar pattern" and it is the reason for making Terraform modules, or components, as I referred to them above. So yes, group them into abstract components, and provide as specific an interface as necessary, via the variables that the module exposes, for them to be useful in well-defined cases, which you then test rigorously.

> Or should I just accept that we'll have to maintain 1500 sets of TF project files?

This depends on their lifecycle. You may end up having to write many Terraform definitions, which all consume the same modules -- but do they change at a similar rate? Are they all created and destroyed at the same time? These are considerations for how to create deployment pipelines, if that's what you're doing.

If you are Terraforming by hand, and all of the environments have a similar lifecycle (*e.g.* every tuesday you have a maintenance window and pull in / apply changes), then you could start considering using some form of wrapper.

Personally, I have avoided Terragrunt because it is an extra layer over something which can already do the job, but your situation might be different. The DRYness in my experience is best introduced via paralellism in CI/CD pipelines, and with appropriate automation at the repo level. Again -this depends very much on your operating environment, and the existing tools you use, so be intentional about the approach.
> 0 votes

---
Tags: terraform, infrastructure-as-code
---


# Question
Title: How can I use rewrite-target on the root path with Azure Kubernetes Service?
I want to have an API service that I want to reach under `/api`, and other requests to fall back to a frontend service.

Peculiar is that this worked locally with minikube, but now that I deployed on Azure Kubernetes Service (I used this official MS tutorial to add an ingress controller), I can not get it to work.

```
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  rules:
  - http:
      paths:
      - path: /api/(.*)
        pathType: Prefix
        backend:
          service:
            name: api-service
            port:
              number: 9002
      - path: /(.*)
        pathType: Prefix
        backend:
          service:
            name: frontend-service
            port:
              number: 9001

```

That worked just fine on minikube, but on AKS, this causes **a timeout** when I try to open the publicly available IP. I have tried the following things:

* Change the frontend path to `(/|$)(.*)`
Together with changing the rewrite-target to `/$2`. `kubectl` **will not even deploy** this because the path needs to be absolute.
* Change the frontend path to `/`
Now **everything returns the frontend index.html**, even if the request is for `/style.css` or `favicon.ico` etc. (unless it starts with `/api`).
* Change the frontend path to `/` and remove the rewrite-target
Then the frontend works fine, but **the api service doesn't "fit"** as intended. The api service has a call `/get-employee`, which I want to call under `/api/get-employee`, but I can't do that unless the api service itself would already have it under `/api/get-employee`.

How can I add a "root" or "fallback" service while still using rewrite-target?

# Answer
I had the same issue when trying Microsofts tutorial. Adding `host` under the `rules` section worked for me.

**Example:**

```
spec:
  ingressClassName: nginx
  rules:
  - host: {your_app}.northeurope.cloudapp.azure.com
    http:
      paths:
      - path: /test(/|$)(.*)
        pathType: Prefix
        backend:
          service:
            name: mockserver-service
            port:
              number: 1234

```
> 1 votes

---
Tags: kubernetes, ingress, azure-kubernetes-service, ingress-nginx
---


# Question
Title: Escape quotes and commas in Docker volume paths using bind-mount syntax
In order for me to write the most bullet-proof scripts as possible that don't break with possible edge cases (no matter how unlikely), how do I escape commas and quotes in Docker bind-mount paths using `--mount` syntax?

Note that Docker docs on bind mounting say everyone is "encouraged to use" `--mount` instead of `-v`, but the case of `-v`, I have a similar question for how to escape `:` in paths.

For example,

```
cd /Users/name
mkdir te,\"st
touch te,\"st/file.txt
docker run -it --rm --mount 'type=bind,source=/Users/name/te??st,target=/usr/test' alpine ash

```

Wher e`??` needs to be the comma `,` and double quote `"` from the `te,"st` directory created above.

* I tried wrapping the value after `source=` in double quotes just to test if it will accept a comma, but get error `bare " in non-quoted-field`. This attempt was based on the info-box "Escape values from outer CSV parser" in the Choose the -v or --mount flag section of Docker storage volumes docs. But this field clearly doesn't like double quotes, at all, ever.
* Single quotes, double commas, and backslashes in the volume path don't seem to work, with errors such as `invalid field 'st' must be a key=value pair`.
* With this, I am out of guesses. I tried a few random stabs in the dark to see if Docker would do environment variable expansion instead of the shell, but nothing is working.

Is it simply impossible to work with paths or volumes that contain a comma or a double quote in their names? Can I instruct Docker to use environment variables somehow so it does the environment variable expansion instead of the shell?

Or, do I need to use a `compose.yaml` file to get around this limitation so I can present the parts/pieces of the mounts in a different encoding/format?

# Answer
If you have a directory named `te,"st`, it's trivial to work with using `-v`: just quote it, keeping in mind that if you have a double quote inside a double quote you need to escape it. So this works:

```
$ mkdir te,\"st
$ touch te,\"st/file1
$ docker run --rm -v "$PWD/te,\"st:/data" alpine ls /data
file1

```

For `--mount` it's a little trickier. From https://github.com/docker/cli/issues/1480 we know that the argument to `--mount` is parsed using CSV syntax, so we can use typical CSV escaping to take of things (quote fields that contain commas, and double-quote to escape quotes (`""`)):

```
$ docker run --rm --mount "type=bind,\"src=$PWD/te,\"\"st\",target=/data" alpine ls /data
file1

```
> 2 votes

---
Tags: docker
---


# Question
Title: Can the status be running after applying the yaml file?
This is an interview question, It asks apply the following yaml file into your k8s cluster:

```
apiVersion: V1
Kind: pod
Metadata:
  Name: freebox
  Spec:
    Containers:
    - Name: busybox
      Image: busybox:latest
      Imagepullpolicy: IfNotPresent

```

When running the kubectl get pod freebox, could the status be running? Why?

# Answer
> 0 votes
Running means that container(s) in your Pod is/are not crashed. Entrypoint started, and command is still running.

Creating that yaml (assuming we fixed case / caps characters aren't at the right place, ...). Pod will be Running. Why: because process has no reason to crash. Because if I inspect that image it uses, entrypoint/cmd is just some /bin/sh -c /bin/sh. Shell will start and keep running.

---
Tags: kubernetes
---


title: How Docker volume works?

text:
# Question
Title: How Docker volume works?
Somebody creates a php and Apache project, with docker, using volumes to persist the files that this project generated (plain text files saved in a folder).

This person built the image and ran the project normally, with the exception of one detail, (s)he didn't link the folder where (s)he saved the messages (www/site/messages) (s)he simply used the /data folder, in which the project had no connection. And it worked.

How does the volume really work? Because to me it doesn't make sense.

This is the project:

https://github.com/matheusbattisti/curso\_docker/tree/main/2\_volumes

# Answer
Dockerfile's `VOLUME` does not allow you to specify a host path.
On the host-side, the volumes are created with a very long ID-like name, these volumes are often referred to as unnamed/anonymous volumes.

Given this Dockerfile:

```
FROM php7:latest
VOLUME /var/www

```

Build it:

```
docker build -t myTest

```

Run it :

```
docker run --rm -it myTest

```

Inside the container, run `ls` and you'll notice the directory exists; `/var/www`

Running the container also creates a directory on the host-side.

While having the container running, execute `docker volume ls` on the host machine and you'll see something like this

```
DRIVER    VOLUME NAME
local     c984..e4fc

```

Back in the *container*, execute `touch /var/www/myFile`

This file is now available on the host machine, in one of the unnamed volumes.

```
ls /var/lib/docker/volumes/c984..e4fc/_data

```

Similarly, you can try to delete this file on the host and it will be deleted in the container as well.

Now run a new container, but specify a volume using `-v`:

```
docker run --rm -it -v /myVolume myTest

```

This *adds* a second volume and the whole system ends up having two unnamed volumes. On the host-side, the new second volume is anonymous and resides together with the other volume in `/var/lib/docker/volumes/`.

It was stated earlier that the `Dockerfile` can not map to a host path which sort of pose a problem for us when trying to bring files in from the host to the container during runtime. A different `-v` syntax solves this problem.

Imagine I have a subfolder in my project directory `./src` that I wish to sync to `/src` inside the container. This command does the trick:

```
docker run -it -v $(pwd)/src:/src myTest

```

Both sides of the `:` character expects an absolute path. Left side being an absolute path on the host machine, right side being an absolute path inside the container.

We run this command:

```
docker run -v $(pwd)/src:/src myTest

```

**Is the specification purely informational?**

No, it does mount a volume, it's similar to `-v /var/www` you just don't specify a mount point in the host machine, so docker will take care of it.

**What are the (dis)advantages of (not) specifying volumes?**
**Why and in which cases should I use VOLUME and when shouldn't I?**

It's probably a best practice to never use `VOLUME`.
The first reason we have already identified: We can not specify the host path.
The second reason is people might forget to use the `--rm` option when running the container, you will end up with a couple of unused volumes and it might be a daunting task to figure out which of all anonymous volumes are safe to remove

**What about files that may optionally be mounted? Do they count as volumes in that case, too?**

In this case you don't need volumes; mapping a single file does not make sense, I would use `ADD` or `COPY` inside the dockerfile.
> 3 votes

# Answer
Say we have an application running on a docker container that generates some kind of daily reports. The daily reports that our application generates are stored somewhere within the container. One fine day we entered the office to realize the docker container was deleted because of some internal issues and that’s it, we lost all the reports which were stored within the container.

To tackle such issues we should have some way to persist that valuable data even if the container is removed. This is possible with the help of the volume and the bind mount feature in the docker.

Docker volume is a storage area on our local host system which is completely managed by docker. This is the area where our data will be persisted. In case our container is removed still we can refer to this volume and get back our data.

> > docker volume create storage

* Here we are creating a volume named storage

> > docker container run --name ubuntu-container-1 -d -it -v storage:/opt/test ubuntu

-v is use to map the volume to a container path
> 1 votes

---
Tags: docker, volume
---


# Question
Title: Ansible / Jinja2 Unexpected templating type error
I am trying to take a dictionary from one or more async tasks "register: task\_variable" then collecting it as a list based on its id's and running a wait task using async\_status on the resulting list generated from the id's

But i am getting an "Unexpected templating type error" with the Jinja "sync\_do\_list() takes 1 positional argument but 2 were given"

I've been trying various things for hours too many to remember and i just cant get past this

Here is the code in it's current form:

```
loop: "{{ (task_variable.results|default({}, true)) | selectattr('ansible_job_id') | map(attribute='ansible_job_id') | list([]) }}"

```

Some of the wait tasks add together multiple test\_variables if multiple asyncs need to complete to proceed

Some of the tasks that generate the task\_variables are wrapped in whens so may never generate a variable hence why i am doing a loop like that to be able to handle them all as appropriate

The code is on my work laptop so hard to get it over without manually typing it, but this is the only piece that really matters, any help would be appreciated. thanks.

Edit:

So to fix that error I had to change list(\[\]) to list

The end result however is has been changed to

```
loop: "{{ [task1, task2, task3, task4] | selectattr('results', 'defined') | map(attribute='results') | selectattr('ansible_job_id', 'defined') | map(attribute='ansible_job_id') | list }}"

```

Where any of the tasks can be empty or undefined and they will get filtered out correctly but must remain as a list even if only checking one task

The resulting {{ item }} can be passed into an async\_status and avoids having to check what is defined and what isn't via when statements and such.

# Answer
Changing `list([])` to `list` resolves that specific error.
> 0 votes

---
Tags: ansible, yaml, jinja
---


# Question
Title: How do I make my AMD GPU available within a docker image based on python:3.9.10
I'd like to do some machine learning on my AMD 6800 XT gpu within a python image based on `python:3.9.10`. I can confirm that the GPU is available outside of the image (in a wsl2 instance). However, if I do `docker run -it python:3.9.10 /bin/bash` and then complete the same tutorial (https://docs.microsoft.com/en-us/windows/ai/directml/gpu-tensorflow-wsl#install-the-tensorflow-with-directml-package) it doesn't work:

```
(directml) root@8a8274e5337f:/# python
Python 3.6.13 |Anaconda, Inc.| (default, Jun  4 2021, 14:25:59)
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow.compat.v1 as tf
>>> tf.enable_eager_execution(tf.ConfigProto(log_device_placement=True))
>>> print(tf.add([1.0, 2.0], [3.0, 4.0]))
2022-08-18 12:29:39.540717: W tensorflow/stream_executor/platform/default/dso_loader.cc:108] Could not load dynamic library 'libdirectml.0de2b4431c6572ee74152a7ee0cd3fb1534e4a95.so'; dlerror: libd3d12.so: cannot open shared object file: No such file or directory
2022-08-18 12:29:39.540760: W tensorflow/core/common_runtime/dml/dml_device_cache.cc:137] Could not load DirectML.
2022-08-18 12:29:39.540793: I tensorflow/core/common_runtime/dml/dml_device_cache.cc:250] DirectML device enumeration: found 0 compatible adapters.
2022-08-18 12:29:39.541010: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2022-08-18 12:29:39.545145: I tensorflow/core/common_runtime/eager/execute.cc:571] Executing op Add in device /job:localhost/replica:0/task:0/device:CPU:0
tf.Tensor([4. 6.], shape=(2,), dtype=float32)

```

This article has led me to think that perhaps docker doesn't support AMD GPUs at all: https://docs.docker.com/desktop/windows/wsl/

Can anyone suggest what I might be able to do to get this to work?

Note that the reason I have picked this image is that my environment is based on a rather lengthy Dockerfile inheriting from `python:3.9.10`, and I'd like to keep using that image on the PC with the GPU as well as other (nvidia) environments, so I'm after a portable solution as far as the image is concerned, although I'd be grateful for any solution at this point.

# Answer
Tensorflow doesn't have access to libd3d12.so file:

> 2022-08-18 12:29:39.540717: \>tensorflow/stream\_executor/platform/default/dso\_loader.cc:108\] Could not load dynamic library 'libdirectml.0de2b4431c6572ee74152a7ee0cd3fb1534e4a95.so'; dlerror: libd3d12.so: cannot open shared object file: No such file or directory

Here is an article solving this issue: enter link description here

Shortly said, use this line instead:

```
docker run -it --rm --device /dev/dxg --mount type=bind,src=/usr/lib/wsl,dst=/usr/lib/wsl -e LD_LIBRARY_PATH=/usr/lib/wsl/lib python:3.9.10

```
> 0 votes

---
Tags: docker
---


# Question
Title: How to set reserved concurrency of a lambda function
I have serveral lambda functions (all in the same region) and I want to limit the concurrency of one of them to a very small value. Specifically, I do not want more than 3 instances of this function running concurrently. This function is not expected to receive a lot of traffic so setting its `Reserve concurrency` to a low value makes a lot of sense.

When I try to set this function's `Reserve concurrency` I get a rejection message (screeshot below) as follows: `The unreserved account concurrency can't go below 100`.

What should I do to be able to restrict the concurrency of this function?

(FWIW, the `Unreserved account concurrency` of my account is `100`)

# Answer
> 2 votes
Functions that don't have reserved concurrency defined need to be able to scale when necessary. AWS insists that you leave 100 invocations for such functions.

you can use upto Unreserved account concurrency minus 100 when defining reserved concurrency for your functions.

> You can reserve up to the Unreserved account concurrency value that is shown, minus 100 for functions that don't have reserved concurrency. To throttle a function, set the reserved concurrency to zero. This stops any events from being processed until you remove the limit.
>
> Use GetAccountSettings to see your Regional concurrency limit. You can reserve concurrency for as many functions as you like, as long as you leave at least 100 simultaneous executions unreserved for functions that aren't configured with a per-function limit.

https://docs.aws.amazon.com/lambda/latest/dg/configuration-concurrency.html

---
Tags: amazon-web-services, aws-lambda
---


# Question
Title: How to identify the physical IP of where Kubernetes API server runs
# Question

In a Kubernetes cluster, there is a Docker network IP which looks to be the API server. However, not sure how to identify the physical IP and location of the box in which it is working.

If there is a guidance how to proceed, it would be appreciated. I have been handed over the system but no information.

```
$ curl https://172.30.0.1
{
  "paths": [
    "/api",
    "/api/v1",
    "/apis",
    "/apis/apps",
    "/apis/apps/v1beta1",
    ...
  ]
}

$ kubectl get pods -o wide
NAME                    READY     STATUS    RESTARTS   AGE       IP
logging-fluentd-2vc4v   1/1       Running   0          1d        10.1.13.155
logging-fluentd-bvdtz   1/1       Running   0          1d        10.1.3.3
logging-fluentd-xvxrk   1/1       Running   0          1d        10.1.10.71

$ sudo iptables -t nat -L -n
Chain KUBE-SERVICES (2 references)
target                     prot opt source               destination
....
KUBE-SVC-NPX46M4PTMTKRN6Y  tcp  --  0.0.0.0/0            172.30.0.1           /* default/kubernetes:https cluster IP */ tcp dpt:443
KUBE-SVC-BA6I5HTZKAAAJT56  tcp  --  0.0.0.0/0            172.30.0.1           /* default/kubernetes:dns-tcp cluster IP */ tcp dpt:53
....

$ ip route
default via 10.0.4.1 dev eth0 proto static metric 100
10.0.4.0/22 dev eth0 proto kernel scope link src 10.0.4.21 metric 100
10.1.0.0/16 dev tun0 proto kernel scope link
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1
172.30.0.0/16 dev tun0 scope link

```

# Answer
The apiserver runs on master nodes. To get a list of all the master node IP's, this should help-

```
kubectl -n kube-system get po | grep -i apiserver | cut -f1 -d" " | xargs kubectl -n kube-system get po $1 -o=jsonpath="{.items[*].status.hostIP}"

```

Based on the version of Kubernetes you are running, you might need to update the location of the `hostIP` in the `jsonPath` parameter.
> 1 votes

# Answer
This on-liner work on k8s 1.20:

```
kubectl -n kube-system get pod -l component=kube-apiserver -o=jsonpath="{.items[0].metadata.annotations.kubeadm\.kubernetes\.io/kube-apiserver\.advertise-address\.endpoint}"

```
> 1 votes

---
Tags: kubernetes, networking
---


# Question
Title: The best practice to set up cpanel with mongoDB on a cloud server!
I'm trying to set up my cloud server so i can run an app made with nodeJs and mongoDB, now for nodeJs my setup is to deploy it using cpanel on a cenos 7 os and since cpanel does not really support mongoDB i wonder what's the best practice to deploy mongoDB alongside of it!

1. should i simply install and run the mongoDB on my centos 7?
2. should i install docker alongside cpanel and run mongoDB inside a docker container? i prefer this choice since docker has an official mongoDB Image but what i'm concerned about whether it would be okay if my cpanel is not actually in a docker container itself?

# Answer
At a minimum you need access to the shell. Without the shell, you will work as if you were 20 years ago or more. If you will host just one web application, at least you also need a public ip or domain preferable with https

Also the usage of environment variables to avoid hardcoded values are required.

The following strategies assume that you have shell access and a public ip or domain.

## let's not reinvent the wheel

**Dokku**

Dokku is an open-source platform-as-a-service (PaaS). If you're familiar with Heroku, you can consider Dokku a private Heroku that you manage.

Basically, after the installation, you will have a private git url in which if you push some standard code, the app will be deployed.

I don't know any other similar open source platform.

**Buddy**

https://buddy.works/pricing

In this platform you could automate some tasks. It is like a mini jenkins

## Minimal manual implementation

* Create a github repository
* Add a Dockerfile to your source code
* Use environment variables instead hardcoded values of database connection in the source code
* Install docker and git in the server
* Start a mongodb using docker and volumes
* After tests on your localhost, push to github
* Enter to the cloud and clone the git repository
* Execute the `docker build` to generate the image
* Execute `docker run -p 80:xyz`. **xyz*** is the port of your application
> 0 votes

---
Tags: docker, node.js, mongodb
---


# Question
Title: What is an example of an error that tflint will catch that `terraform init` will not error on?
I am trying to test a pipeline change that will run tflint automatically, but I'm struggling to find an example of an error that won't prevent `terraform init` from completing without errors but will be caught by `tflint --module`. My goal is to prove that tflint is running is running in the pipeline and will fail any pipeline that finds linting errors. Any ideas?

I know there must be lots of examples, or tflint wouldn't exist - but I'm not running across them right now.

Examples of things I've tried so far: tflint doesn't catch white space or using dashes when underlines are more 'correct' (or is this a config issue on my part?), `terraform init` doesn't tolerate single quotes or misnamed modules.

# Answer
These are the default rules for `tflint`. Most of these rules will have an example of how to trigger it and remediate it. A simple one you could use is the `terraform_comment_syntax` rule.

You can trigger it by using a `// comment` inside a `*.tf` file.
> 2 votes

---
Tags: terraform
---


# Question
Title: Can not authenticate to DockerHub docker.io with ctr; works fine with crictl
Authentication works on `crictl`, when I use `--creds`

```
$ sudo crictl pull --creds "evancarroll:$TOKEN" docker.io/alpine:3

```

But when I try the same command `ctr`, I get an error:

```
$ sudo ctr images pull --user "evancarroll:$TOKEN" docker.io/alpine:3
docker.io/alpine:3: resolving
INFO[0000] trying next host error="pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed" host=registry-1.docker.io
ctr: failed to resolve reference "docker.io/alpine:3": pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

```

---

You can see the `--http-dump --http-trace`,

```
$ sudo ctr images pull --http-dump --http-trace --user "evancarroll:$TOKEN" docker.io/alpine:3
INFO[0000] HEAD /v2/alpine/manifests/3 HTTP/1.1
INFO[0000] Host: registry-1.docker.io
INFO[0000] Accept: application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*
INFO[0000] User-Agent: containerd/v1.5.11-k3s2
INFO[0000]
docker.io/alpine:3: resolving      |--------------------------------------|
elapsed: 0.1 s      total:   0.0 B (0.0 B/s)
INFO[0000] HTTP/1.1 401 Unauthorized
INFO[0000] Content-Length: 149
INFO[0000] Content-Type: application/json
INFO[0000] Date: Wed, 15 Jun 2022 16:00:59 GMT
INFO[0000] Docker-Distribution-Api-Version: registry/2.0
INFO[0000] Strict-Transport-Security: max-age=31536000
INFO[0000] Www-Authenticate: Bearer realm="https://auth.docker.io/token",service="registry.docker.io",scope="repository:alpine:pull"
INFO[0000]
INFO[0000] POST /token HTTP/1.1
INFO[0000] Host: auth.docker.io
INFO[0000] Content-Type: application/x-www-form-urlencoded; charset=utf-8
docker.io/alpine:3: resolving      |--------------------------------------|
elapsed: 0.2 s      total:   0.0 B (0.0 B/s)
INFO[0000] client_id=containerd-client&grant_type=password&password=HIDDEN&scope=repository%3Aalpine%3Apull&service=registry.docker.io&username=evancarrollHTTP/1.1 200 OK
INFO[0000] Transfer-Encoding: chunked
INFO[0000] Content-Type: application/json; charset=utf-8
INFO[0000] Date: Wed, 15 Jun 2022 16:00:59 GMT
INFO[0000] Strict-Transport-Security: max-age=31536000
INFO[0000] X-Trace-Id: 5da4dacdeb208ead79e15a59f83499b2
INFO[0000]
INFO[0000] 833
INFO[0000] {"access_token":"HIDDEN","scope":"","expires_in":300,"issued_at":"2022-06-15T16:00:59.464114622Z"}
INFO[0000]
INFO[0000] 0
INFO[0000]
INFO[0000] HEAD /v2/alpine/manifests/3 HTTP/1.1
INFO[0000] Host: registry-1.docker.io
INFO[0000] Accept: application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*
INFO[0000] Authorization: Bearer HIDDEN
docker.io/alpine:3: resolving      |--------------------------------------|
elapsed: 0.3 s      total:   0.0 B (0.0 B/s)
INFO[0000] HTTP/1.1 401 Unauthorized
INFO[0000] Content-Length: 149
INFO[0000] Content-Type: application/json
INFO[0000] Date: Wed, 15 Jun 2022 16:00:59 GMT
INFO[0000] Docker-Distribution-Api-Version: registry/2.0
INFO[0000] Strict-Transport-Security: max-age=31536000
INFO[0000] Www-Authenticate: Bearer realm="https://auth.docker.io/token",service="registry.docker.io",scope="repository:alpine:pull",error="insufficient_scope"
INFO[0000]
INFO[0000] trying next host error="pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed" host=registry-1.docker.io
ctr: failed to resolve reference "docker.io/alpine:3": pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

```

# Answer
> 3 votes
# `docker.io/library/$IMAGE:$TAG`

The problem here is simple the right syntax for `ctr` isn't `docker.io/alpine:latest` but `docker.io/library/alpine:latest` that should now work with `--creds "evancarroll:$TOKEN"` too.

```
ctr images pull docker.io/library/alpine:latest

```

## IRC logs

Thanks goes out to `larks` and `ada` on IRC `#docker` for helping out with this,

> larsks\> EvanCarroll: you probably need to include the tag there (`ctr images pull docker.io/library/alpine:latest`). The `/library` is there because that's the actual path to the image repository. There's some magic somewhere that transforms `docker.io/alpine` into `docker.io/library/alpine`; I don't know if that's client side or server side; ada will know much more about that than I do.

And thanks to @ada for showing where this is documented in the code, and clarifying

> library/ is the namespace for all the top-level images on dockerhub with docker, that namespace is implied if you leave off the registry & namespace part of the tag : `docker pull alpine` == `ctr images pull docker.io/library/alpine`

# Answer
> 1 votes
This is actually how I got it working with "ctr containerd.io 1.4.6".

**ctr image pull --user "myusername:mypasswd" myurl.dot.com/docker\_image:1.0.3**

Hope it helps someone.

---
Tags: dockerhub, authentication, containerd
---


# Question
Title: How do I list pods sorted by label version in Kubernetes?
I need to list some pods sorted by version and get the last index of it.

I'm trying to do like:

```
kubectl get pods
--namespace my-namespace
--selector "app.kubernetes.io/name=my-cool-pod"
--sort-by='.items[*].metadata.labels["app.kubernetes.io/version"]'
--output jsonpath="{.items[-1:].metadata.name}

```

But it is not working.

Also, my pod labels are:

```
    labels:
      app.kubernetes.io/instance: my-instance
      app.kubernetes.io/managed-by: my-manager
      app.kubernetes.io/name: my-name
      app.kubernetes.io/version: x.y.z

```

How to I list pods sorted my version?

Thanks

# Answer
> 1 votes
I've figured it out

Lables in K8S are for filtering, and not sorting.

I've just added a version field outside labels and it worked.

---
Tags: kubernetes, kubectl
---


# Question
Title: How to disable some branches in Multibranch Jenkins Builds?
I have a multibranch job set to run any branch with a Jenkinsfile.

I have some options I can think of if I want to remove a branch from the list of jobs running for the multi-branch pipeline.

1. I can delete the branch
2. I can delete the Jenkinsfile in that branch

The second solution is good, except I need to commit and push that to the git repo for my branch, and if that branch is merged into another branch it blows away the Jenkinsfile.

What is the best way to disable only some branches of a multibranch pipeline?

# Answer
Jenkins can filter branches in a multibranch pipeline by name using a wildcard or regular expression.
> 29 votes

# Answer
Simply delete the Jenkinsfile on the branch(es) you don't want to have corresponding branch jobs for. This will delete the branch job (of course, iff you have set your "Orphaned Item Strategy" appropriately).

From the perspective of a Jenkins Multibranch Pipeline Project, this has the same effect as deleting the branch. This is because it is simply scanning for branches that contain Jenkinsfiles as its criterion for when to create (or delete) a corresponding branch job.

Revert the commit to restore the Jenkinsfile if/when you need the branch job again.
> 1 votes

# Answer
Instead of configuring it in the UI (as perfectly described by casey vega's answer, you can also do this within your Jenkinsfile:

## Declarative Pipeline

Add the following condition within any stage directive:

```
when { anyOf { branch 'master'; branch 'release/**' } }

```

As you can see, it's possible to use exact branch names as well as some wildcards (ANT style path glob pattern). It's also possible to use regex and some more fancy conditions and combinations. Please have a look at Jenkins' offical docs on the `when` directive for reference and detailed information.

## Scripted Pipeline

Just enclose the part you want to have run conditionally in the following code:

```
if (env.BRANCH_NAME == 'master' || (env.BRANCH_NAME).startsWith('release/')) {
  // Your Pipeline goes here
}

```

This can be used almost everywhere within your (scripted) Jenkinsfile. As you can see, it's just some simple string comparison. For more complex Regex comparisons, take a look at Groovy's Find and Match operators, which also work fine here.

## Conclusion

I would prefer using this method (although I recommended it otherwise some time ago), as you can then check-in your pipeline configuration into your source control and you're more on the Configuration-as-Code approach as when you configure it in your Jenkins UI.
> 1 votes

---
Tags: jenkins-pipeline
---


# Question
Title: How to properly achieve dynamic parallel action with a declarative pipeline?
Currently, I'm going to need an implementation that must find all files within a directory and start a parallel task for every file found.

Is it possible to achieve this using declarative pipelines?

```
pipeline {
    agent any
    stages {
        stage("test") {
            steps {
                dir ("file_path") {
                    // find all files with complete path
                    parallel (
                        // execute parallel tasks for each file found.
                        // this must be dynamic
                        }
                    }
                }
            }
        }
    }
}

```

# Answer
> 40 votes
Managed to solve it with the following code:

```
pipeline {
    agent { label "master"}
    stages {
        stage('1') {
            steps {
                script {
                    def tests = [:]
                    for (f in findFiles(glob: '**/html/*.html')) {
                        tests["${f}"] = {
                            node {
                                stage("${f}") {
                                    echo '${f}'
                                }
                            }
                        }
                    }
                    parallel tests
                }
            }
        }
    }
}

```

# Answer
> 14 votes
This also works, if you want to stay within the `Declarative Pipeline` space

```
// declare our vars outside the pipeline
def tests = [:]
def files

pipeline {
    agent any
    stages {
        stage('1') {
            steps {
                script {
                    // we've declared the variable, now we give it the values
                    files = findFiles(glob: '**/html/*.html')
                    // Loop through them
                    files.each { f ->
                        // add each object from the 'files' loop to the 'tests' array
                        tests[f] = {
                            // we're already in the script{} block, so do our advanced stuff here
                            echo f.toString()
                        }
                    }
                    // Still within the 'Script' block, run the parallel array object
                    parallel tests
                }
            }
        }
    }
}

```

# Answer
> 7 votes
Be aware, that dynamic build steps could cause some problems in some build steps, e.g. when you call an other job:

```
pipeline {
    stages {
        stage('Test') {
            steps {
                script {
                    def tests = [:]
                    for (f in findFiles(glob: '**/html/*.html')) {
                        // Create temp variable, otherwise the name will be the last value of the for loop
                        def name = f
                        tests["${name}"] = {
                            build job: "${name}"
                        }
                    }
                    parallel tests
                }
            }
        }
    }
}

```

# Answer
> 2 votes
It's much easier to use scripted Pipelines to do this since you can use arbitrary Groovy, but you should still be able to do this with declarative Pipelines using the `findFiles` step.

# Answer
> 0 votes
Here is a solution where you can setup the maximum number of concurrent tasks running at once with `collate(n)`.

```
pipeline {
  stages {
    stage("Parallel") {
      steps {
        script {
          def list = ["alpha", "beta", "gamma", "delta", "epsilon", "zeta", "eta", "theta", "iota", "kappa"]

          // Split the list into batches of 4
          def parallelBatches = list.collate(4);
          // [["alpha", "beta", "gamma", "delta"], ["epsilon", "zeta", "eta", "theta"], ["iota", "kappa"]]

          // This allows to run a maximum of 4 tasks in parallel (at once)
          for (parallelBatch in parallelBatches) {
            def tasks = [:]

            for (task in parallelBatch) {
              tasks["${task}"] = {
                stage("${task}") {
                  echo '${task}'
                }
              }
            }
            parallel tasks

            // Once the batch is over, the outer loop move onto the next batch
          }
        }
      }
    }
  }
}

```

---
Tags: jenkins, jenkinsfile
---


# Question
Title: How to keep the overview over the entire lifecycle of backlog items
I'm working as a product owner in a software house. For the project I'm taking care of, we're using Azure DevOps to manage our backlog. The team consists of 3 developers + me as product owner / part time developer.

By default, Azure DevOps offers the following status values for Bugs and Product Backlog Items:

* New
* Approved
* Committed
* Done

For me as product owner it turns out that these status values are not enough to keep the overview over the entire lifecycle of the bugs / product backlog items, because I'm missing an overview of features / backlog items / bugs

* whose quote must still be approved from the customer
* that have been released to the integration environment, but must still be approved
* that have been approved
* that have been released to the production environment.

Long story short, the status values in Azure DevOps are covering only a part of the lifecycle of the backlog items.

To have more overview, I'd need additional values, like

**Statuses before the development starts**

* Customer requires a feature
* We sent the quote for the feature to the customer
* Customer approved the quote

**Statuses after the development done**

* Feature / bugfix is released to the integreation environment
* Customer approved the feature / bugfix
* Feature / bugfix is released to the integreation environment

I've read about several possibilitites to achieve this, e.g.

* Introducing additional status values (some people dissuaded me from this option)
* Working with Tags
* Working with Area paths
* Working with Custom Fields

I strongly assume that I'm not the only one who's facing this issue. What would you recommend considering your experience so far?

# Answer
> 0 votes
There is no single, perfect solution that I've found. The default work item statuses are pretty sparse, and ill-defined, but they are meant to be super generic to apply in the maximum number of use cases. Yes, people say you shouldn't add statuses. I think those people are wrong, and partially right.

Each work item broadly goes through these phases:

1. New: someone just threw it in the backlog. Needs lots of definition.
2. Design/Analysis: business and technical requirements get hashed out. Bugs get investigated. Mockups get created. Scrum teams are doing backlog grooming sessions for these items. Clients need to approve stuff.
3. Estimated: the team has provided an estimate good enough to start work.
4. Ready: the customer approves of the scope and estimate, and can be started on at the team's convenience.
5. Active: the team is actively developing or testing an item.
6. QA Complete: the QA team verified to the best of their ability that the work items meet the acceptance criteria.
7. In User Acceptance Testing: a customer can start using this new thing.
8. Approved for Release: the customer wants to proceed with a production deployment.
9. Released: the team deployed and the customer ensured it made it out.

Work item states are great for many of these points in the lifecycle, whether doing agile, scrum, kanban, or waterfall. Feel free to name them appropriately for your team's workflow. I have found the Design/Analysis phase has a lot of nuance that isn't easily captured in the "state" of an item.

Client approvals seems to be a missing "thing" in Azure DevOps. Rational Team Concert by IBM has an Approvals feature for their work items, which is nice. For Azure DevOps, configure the Kanban board for your backlog to add columns for these approvals. Azure DevOps tracks these column assignments in the work item history, so provide meaningful names and descriptions for these columns.

Many times, columns will map directly to work item states. Other times, it won't. This is similar to how I set up our Kanban board. The table header shows the Kanban column names, and underneath shows a corresponding work item state:

| **New** | **Design/Analysis** | **Example Mapping** | **Backlog Grooming** | **Estimated** | **Ready** | **Active** | **QA Complete** | **UAT** | **UAT Complete** | **Done** |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| New | Design/Analysis | Design/Analysis | Design/Analysis | Estimated | Ready | Active | QA Complete | UAT | UAT | Done |

The Design/Analysis part has a number of columns to represent that back-and-forth process of defining and refining requirements. Finally, the user acceptance testing side of things needed a little more explicit naming, but from a process perspective we didn't need a separate status. This was arbitrary, and simply worked for us.

When things like mock ups, UML diagrams, or other design documents must be produced, consider adding Task work items under the relevant story to encompass this work.

If work item states, Kanban board columns, and Task work items are not sufficient to capture the nuance of your process, you can always resort to tagging work items and creating work item queries.

To summarize:

* Add work item states to capture the main points of your lifecycle.
    + Litmus test: switch to the backlog view, highlight items, right-click, copy to clipboard. Paste into a spreadsheet. If the work item states are not sufficient to communicate where the item is in your process **to management** then add a work item state. Ignore people who recommend using a Kanban board for this.
* Configure a Kanban board for your backlog to add columns beyond what work item states can communicate. This can work well for customer approvals, as long as they are they ones to change the column.
* Use Task work items when doing things like mock ups and other design documentation.
* Any other non-standard or wonky edge cases can be captured by tagging work items and developing work item queries.

---
Tags: azure-devops
---


# Question
Title: How to write bash or shell script in SSM run command and execute in linux ec2 instance?
I want to run the bash or shell script in Linux using the SSM run command. But I don't want to write it on the Linux server and execute it remotely using the SSM command. I want to write a script in the run command itself.

Is there any way to run a script from the AWS console? Apart from the run command is there any other way to run the whole script?

I am referring to the multiline script not only one command.

# Answer
First option using aws "CloudShell" service (available only in specific region).

Alternately, you can execute shell script using AWS Systems Manager \> Run Command \> Run a command, Select the type of command that you want to run & Search by keyword " AWS-RunShellScript " (Linux platform only).

In Command Parameters, specify a shell script or a command to run.

PS : For both Windows & Linux platform, search by keyword " AWS-RunPowerShellScript "
> 0 votes

# Answer
You can write a bash or shell script in SSM run command and execute it in a linux ec2 instance using the following command:

aws ssm run-command --document-name "MyShellScript" --instance-ids "i-1234567890" --parameters "MyScriptParameter=Value"

This will execute the "MyShellScript" script on the instance with ID "i-1234567890" and pass in the parameter "MyScriptParameter" with value "Value".
> -1 votes

---
Tags: amazon-web-services, amazon-ssm
---


# Question
Title: forward http request to pod running on worker node
Context: I am trying to setup a kubernetes cluster on my PC using virtual box. Here's the setup - In this setup i am able to launch pods from the control plane, as well as able to send http requests to pods

here CP01: master/control plane, W01 - worker1, W02 - worker2 node.

I had initialized control plane using -

```
master] kubeadm init --apiserver-advertise-address 10.5.5.1 --pod-network-cidr 10.244.0.0/16 --cri-socket unix:///var/run/cri-dockerd.sock
worker] kubeadm join 10.5.5.1:6443  --token jv5pxe.t07snw8ewrbejn6i   --cri-socket  unix:///var/run/cri-dockerd.sock      --discovery-token-ca-cert-hash sha256:10fc6e3fdc2085085f1ea1a75c9eb4316f13759b0d3773377db86baa30d8b972

```

I am able to create deployment as -

## run.yaml

```
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

```

Here's the load balancer -

## serv.yaml

```
apiVersion: v1
kind: Service
metadata:
  name: hello-world
spec:
  type: LoadBalancer
  selector:
    app: nginx
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 80

```

and from the cp01 node , i am able to hit both the pods via load balancer

```
[root@cp01 ~]# kubectl get services
NAME          TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
hello-world   LoadBalancer   10.96.81.128   <pending>     80:31785/TCP   3m56s
[root@cp01 ~]# curl 10.96.81.128|grep Welcome
   <title>Welcome to nginx!</title>
   <h1>Welcome to nginx!</h1>

```

I have following 2 Queries -

Q1: What settings to i need to carry out to make the flannel work without enp0s3 interface? I don't need external connectivity on w01 and w02. So if i disable the enp0s3 on w01 and w02, the flannel pods on worker starts failing. Here's is how i tried reproducing issue on w01 -

```
[root@cp01 ~]# kubectl get pods -A
NAMESPACE      NAME                           READY   STATUS    RESTARTS        AGE
kube-flannel   kube-flannel-ds-mp8zs          1/1     Running   8 (31m ago)     41m
kube-flannel   kube-flannel-ds-p5kwj          1/1     Running   2 (12h ago)     3d1h
kube-flannel   kube-flannel-ds-wqpwl          1/1     Running   0               24m
kube-system    coredns-565d847f94-xddkq       1/1     Running   1 (12h ago)     15h
kube-system    coredns-565d847f94-xl7pj       1/1     Running   1 (12h ago)     15h
kube-system    etcd-cp01                      1/1     Running   2 (12h ago)     3d1h
kube-system    kube-apiserver-cp01            1/1     Running   2 (12h ago)     3d1h
kube-system    kube-controller-manager-cp01   1/1     Running   2 (12h ago)     3d1h
kube-system    kube-proxy-9f4xm               1/1     Running   2 (12h ago)     3d1h
kube-system    kube-proxy-dhhqc               1/1     Running   2 (12h ago)     3d1h
kube-system    kube-proxy-w64gc               1/1     Running   1 (2d16h ago)   3d1h
kube-system    kube-scheduler-cp01            1/1     Running   2 (12h ago)     3d1h
[root@cp01 ~]# ssh w01 'nmcli con down enp0s3'
[root@cp01 ~]# kubectl delete pod -n kube-flannel kube-flannel-ds-mp8zs
pod "kube-flannel-ds-mp8zs" deleted
[root@cp01 ~]# kubectl delete pod -n kube-flannel kube-flannel-ds-wqpwl
pod "kube-flannel-ds-wqpwl" deleted
[root@cp01 ~]# kubectl get pods -A
NAMESPACE      NAME                           READY   STATUS             RESTARTS        AGE
kube-flannel   kube-flannel-ds-2kqq5          0/1     CrashLoopBackOff   2 (25s ago)     45s
kube-flannel   kube-flannel-ds-kcwk6          1/1     Running            0               49s
kube-flannel   kube-flannel-ds-p5kwj          1/1     Running            2 (12h ago)     3d1h
kube-system    coredns-565d847f94-xddkq       1/1     Running            1 (12h ago)     15h
kube-system    coredns-565d847f94-xl7pj       1/1     Running            1 (12h ago)     15h
kube-system    etcd-cp01                      1/1     Running            2 (12h ago)     3d1h
kube-system    kube-apiserver-cp01            1/1     Running            2 (12h ago)     3d1h
kube-system    kube-controller-manager-cp01   1/1     Running            2 (12h ago)     3d1h
kube-system    kube-proxy-9f4xm               1/1     Running            2 (12h ago)     3d1h
kube-system    kube-proxy-dhhqc               1/1     Running            2 (12h ago)     3d1h
kube-system    kube-proxy-w64gc               1/1     Running            1 (2d16h ago)   3d1h
kube-system    kube-scheduler-cp01            1/1     Running            2 (12h ago)     3d1h

```

here's the reason -

```
[root@cp01 ~]# kubectl logs -n kube-flannel kube-flannel-ds-2kqq5
Defaulted container "kube-flannel" out of: kube-flannel, install-cni-plugin (init), install-cni (init)
I1005 08:17:59.211331       1 main.go:207] CLI flags config: {etcdEndpoints:http://127.0.0.1:4001,http://127.0.0.1:2379 etcdPrefix:/coreos.com/network etcdKeyfile: etcdCertfile: etcdCAFile: etcdUsername: etcdPassword: version:false kubeSubnetMgr:true kubeApiUrl: kubeAnnotationPrefix:flannel.alpha.coreos.com kubeConfigFile: iface:[] ifaceRegex:[] ipMasq:true ifaceCanReach: subnetFile:/run/flannel/subnet.env publicIP: publicIPv6: subnetLeaseRenewMargin:60 healthzIP:0.0.0.0 healthzPort:0 iptablesResyncSeconds:5 iptablesForwardRules:true netConfPath:/etc/kube-flannel/net-conf.json setNodeNetworkUnavailable:true}
W1005 08:17:59.211537       1 client_config.go:614] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
E1005 08:17:59.213916       1 main.go:224] Failed to create SubnetManager: error retrieving pod spec for 'kube-flannel/kube-flannel-ds-2kqq5': Get "https://10.96.0.1:443/api/v1/namespaces/kube-flannel/pods/kube-flannel-ds-2kqq5": dial tcp 10.96.0.1:443: connect: network is unreachable

```

Q2: I plan to send http request to the load balancer via enp0s3 interface on cp01 node. do i need to :

* reset the cluster and do the kube init again using 0.0.0.0 ip or,
* is there a way to accomplish it without disturbing existing configurations/setup (using ingress?) ?

Please advice. I have started learning kubernetes recently, so please excuse if i missed out on some basic concepts of the kubernetes world while framing these queries. Please do let me know if something is unclear.

UPDATE:

Q2: i tried initializing kubeadm node via -

```
[root@cp01 ~]# kubeadm init --apiserver-advertise-address 0.0.0.0  --pod-network-cidr 10.244.0.0/16 --cri-socket unix:///var/run/cri-dockerd.sock

```

but with this method, worker nodes are unable to join cluster from 10.5.5.0/24 (enp0s8) network. they are able to join cluster as -

```
kubeadm join 192.168.29.73:6443 --cri-socket  unix:///var/run/cri-dockerd.sock   --token 6srhn0.1lyiffiml08qcnfw --discovery-token-ca-cert-hash sha256:ab87e3e04da65176725776c08e0f924bbc07b26d0f8e2501793067e477ab6379

```

# Answer
Kubelet would usually bind to whichever interface holds your default gateway. I suspect your SDN pods are failing to start up due to this. At that stage, I would check for `netstat`, `ip routes` and iptables rules.

You may want to remove the internal network thing. Deploying Kubernetes, it is recommended for your nodes to run in a LAN, using loadbalancers to expose your ingresses.

Otherwise, you need your default gateway to point to your internal network. And then, you'll need some kind of router, NAT-ing traffic while sending it back to your bridged network.
> 1 votes

---
Tags: kubernetes
---


# Question
Title: Is there aws-vault kind of tool for GCP?
I would like to keep my use access tokens to GCP encrypted in my operating system’s keychain and use them easily with gcloud CLI.

So, does aws-vault for gcp exist to secure my own tokesn when accessing gcp?

This is unrelated to AWS Secret Manager

# Answer
It sounds like you want Google Secret manager.

From the documentation, it claims to:

> Store API keys, passwords, certificates, and other sensitive data.

which sounds like it does what you want it to do.

However, you state

> I would like to keep my tokens encrypted in my operating system’s keychain

-- I don't see why you would want to do that though, if you have secrets manager. The secrets manager should be the source of truth, and the machine should be authorised via IAM to access specific secrets on demand, not persisted in the OS keychain. If the OS is compromised, the secrets will be exposed, whilst the attacker would need to break secrets manager or IAM in order to access them if they are consumed uniquely from secrets manager.
> 0 votes

---
Tags: amazon-web-services, gcp, encryption
---


title: How do I install BlackDuck on mac?

text:
# Question
Title: How do I install BlackDuck on mac?
I am trying to scan container images using the blackdock scanning tool. Unfortunately, I couldn't find any free version to play around with; please provide guidance and the simplest way to scan the docker images.

I have tried using this resource - https://github.com/blackducksoftware?q=docker&type=all&language=&sort=. Not able to install it.

# Answer
> 0 votes
Your link directs to a public GitHub org. It would also appear that their repositories are licensed under Apache 2.0 so you should be able to use these more or less how you see fit (obviously within Apache 2.0, but that is quite permissible). This is further bolstered on their website by their statement here Introduction to Detect where they state:

> "...It is available through GitHub, under the permissive Apache License, Version 2.0 and does not require pre-installation or configuration."

Looks like you should be able to just pull down the repo and get cracking.

---
Tags: docker, dockerfile, devsecops
---