<p>We are planning to run our Azure Devops build agents in a Kubernetes pods.But going through the internet, couldn't find any recommended approach to follow.</p>
<p>Details:</p>
<ul>
<li>Azure Devops Server</li>
<li>AKS- 1.19.11</li>
</ul>
<p>Looking for</p>
<ul>
<li>AKS kubernetes cluster where ADO can trigger its pipeline with the  dependencies.</li>
<li>The scaling of pods should happen as the load from the ADO will be initiating</li>
<li>Is there any default MS provided image available currently for the build agents?</li>
<li>The image should be light weight with BuildAgents and the zulu jdk debian as we are running java based apps.</li>
</ul>
<p>Any suggestions highly appreciated</p>

<p><a href="https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/docker?view=azure-devops" rel="nofollow noreferrer">This article</a> provides instructions for running your Azure Pipelines agent in Docker. You can set up a self-hosted agent in Azure Pipelines to run inside a Windows Server Core (for Windows hosts), or Ubuntu container (for Linux hosts) with Docker.</p>
<blockquote>
<p>The image should be light weight with BuildAgents and the zulu jdk debian as we are running java based apps.</p>
</blockquote>
<h5><a href="https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/docker?view=azure-devops#add-tools-and-customize-the-container" rel="nofollow noreferrer">Add tools and customize the container</a></h5>
<p>Once you have created a basic build agent, you can extend the Dockerfile to include additional tools and their dependencies, or build your own container by using this one as a base layer. Just make sure that the following are left untouched:</p>
<ul>
<li>The <code>start.sh</code> script is called by the Dockerfile.</li>
<li>The <code>start.sh</code> script is the last command in the Dockerfile.</li>
<li>Ensure that derivative containers don't remove any of the dependencies stated by the Dockerfile.</li>
</ul>
<blockquote>
<p><strong>Note:</strong> <a href="https://learn.microsoft.com/en-us/azure/aks/cluster-configuration#container-runtime-configuration" rel="nofollow noreferrer">Docker was replaced with containerd</a> in Kubernetes 1.19, and <strong>Docker-in-Docker</strong> became unavailable. A few use cases to run docker inside a docker container:</p>
<ul>
<li>One potential use case for docker in docker is for the CI pipeline, where you need to build and push docker images to a container registry after a successful code build.</li>
<li>Building Docker images with a VM is pretty straightforward. However, when you plan to use Jenkins <a href="https://devopscube.com/docker-containers-as-build-slaves-jenkins/" rel="nofollow noreferrer">Docker-based dynamic agents</a> for your CI/CD pipelines, docker in docker comes as a must-have functionality.</li>
<li>Sandboxed environments.</li>
<li>For experimental purposes on your local development workstation.</li>
</ul>
</blockquote>
<p>If your use case requires running docker inside a container then, you must use Kubernetes with version &lt;= 1.18.x (currently not supported on Azure) as shown <a href="https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/docker?view=azure-devops#configure-secrets-and-deploy-a-replica-set" rel="nofollow noreferrer">here</a> or run the agent in an alternative <strong>docker</strong> environment as shown <a href="https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/docker?view=azure-devops#start-the-image-1" rel="nofollow noreferrer">here</a>.</p>
<p>Else if you are <a href="https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/docker?view=azure-devops#use-azure-kubernetes-service-cluster" rel="nofollow noreferrer">deploying the self hosted agent on AKS</a>, the <code>azdevops-deployment</code> <em>Deployment</em> at step 4, <a href="https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/docker?view=azure-devops#configure-secrets-and-deploy-a-replica-set" rel="nofollow noreferrer">here</a>, must be changed to:</p>
<pre><code>apiVersion: apps/v1
kind: Deployment
metadata:
  name: azdevops-deployment
  labels:
    app: azdevops-agent
spec:
  replicas: 1 #here is the configuration for the actual agent always running
  selector:
    matchLabels:
      app: azdevops-agent
  template:
    metadata:
      labels:
        app: azdevops-agent
    spec:
      containers:
      - name: azdevops-agent
        image: &lt;acr-server&gt;/dockeragent:latest
        env:
          - name: AZP_URL
            valueFrom:
              secretKeyRef:
                name: azdevops
                key: AZP_URL
          - name: AZP_TOKEN
            valueFrom:
              secretKeyRef:
                name: azdevops
                key: AZP_TOKEN
          - name: AZP_POOL
            valueFrom:
              secretKeyRef:
                name: azdevops
                key: AZP_POOL
</code></pre>
<blockquote>
<p>The scaling of pods should happen as the load from the ADO will be initiating</p>
</blockquote>
<p>You can use cluster-autoscaler and <a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/" rel="nofollow noreferrer">horizontal pod autoscaler</a>.  When combined, the horizontal pod autoscaler is focused on running the number of pods required to meet application demand. The cluster autoscaler is focused on running the number of nodes required to support the scheduled pods. [<a href="https://learn.microsoft.com/en-us/azure/aks/cluster-autoscaler#about-the-cluster-autoscaler" rel="nofollow noreferrer">Reference</a>]</p>
