<p>I read through the karpenter document at <a href="https://karpenter.sh/v0.16.1/getting-started/getting-started-with-terraform/#install-karpenter-helm-chart" rel="nofollow noreferrer">https://karpenter.sh/v0.16.1/getting-started/getting-started-with-terraform/#install-karpenter-helm-chart</a>. I followed instructions step by step. I got errors at the end.</p>
<p>kubectl logs -f -n karpenter -l app.kubernetes.io/name=karpenter -c controller</p>
<p>DEBUG   controller.provisioning Relaxing soft constraints for pod since it previously failed to schedule, removing: spec.topologySpreadConstraints = {&quot;maxSkew&quot;:1,&quot;topologyKey&quot;:&quot;topology.kubernetes.io/zone&quot;,&quot;whenUnsatisfiable&quot;:&quot;ScheduleAnyway&quot;,&quot;labelSelector&quot;:{&quot;matchLabels&quot;:{&quot;app.kubernetes.io/instance&quot;:&quot;karpenter&quot;,&quot;app.kubernetes.io/name&quot;:&quot;karpenter&quot;}}} {&quot;commit&quot;: &quot;b157d45&quot;, &quot;pod&quot;: &quot;karpenter/karpenter-5755bb5b54-rh65t&quot;}
2022-09-10T00:13:13.122Z</p>
<p>ERROR   controller.provisioning Could not schedule pod, incompatible with provisioner &quot;default&quot;, incompatible requirements, key karpenter.sh/provisioner-name, karpenter.sh/provisioner-name DoesNotExist not in karpenter.sh/provisioner-name In [default] {&quot;commit&quot;: &quot;b157d45&quot;, &quot;pod&quot;: &quot;karpenter/karpenter-5755bb5b54-rh65t&quot;}</p>
<p>Below is the source code:</p>
<pre><code>cat main.tf
terraform {
  required_version = &quot;~&gt; 1.0&quot;

  required_providers {
    aws = {
      source  = &quot;hashicorp/aws&quot;
      version = &quot;~&gt; 4.0&quot;
    }
    helm = {
      source  = &quot;hashicorp/helm&quot;
      version = &quot;~&gt; 2.5&quot;
    }
    kubectl = {
      source  = &quot;gavinbunney/kubectl&quot;
      version = &quot;~&gt; 1.14&quot;
    }
  }
}

provider &quot;aws&quot; {
  region = &quot;us-east-1&quot;
}

locals {
  cluster_name = &quot;karpenter-demo&quot;

  # Used to determine correct partition (i.e. - `aws`, `aws-gov`, `aws-cn`, etc.)
  partition = data.aws_partition.current.partition
}

data &quot;aws_partition&quot; &quot;current&quot; {}

module &quot;vpc&quot; {
  # https://registry.terraform.io/modules/terraform-aws-modules/vpc/aws/latest
  source  = &quot;terraform-aws-modules/vpc/aws&quot;
  version = &quot;3.14.4&quot;

  name = local.cluster_name
  cidr = &quot;10.0.0.0/16&quot;

  azs             = [&quot;us-east-1a&quot;, &quot;us-east-1b&quot;, &quot;us-east-1c&quot;]
  private_subnets = [&quot;10.0.1.0/24&quot;, &quot;10.0.2.0/24&quot;, &quot;10.0.3.0/24&quot;]
  public_subnets  = [&quot;10.0.101.0/24&quot;, &quot;10.0.102.0/24&quot;, &quot;10.0.103.0/24&quot;]

  enable_nat_gateway     = true
  single_nat_gateway     = true
  one_nat_gateway_per_az = false

  public_subnet_tags = {
    &quot;kubernetes.io/cluster/${local.cluster_name}&quot; = &quot;shared&quot;
    &quot;kubernetes.io/role/elb&quot;                      = 1
  }

  private_subnet_tags = {
    &quot;kubernetes.io/cluster/${local.cluster_name}&quot; = &quot;shared&quot;
    &quot;kubernetes.io/role/internal-elb&quot;             = 1
  }
}

module &quot;eks&quot; {
  # https://registry.terraform.io/modules/terraform-aws-modules/eks/aws/latest
  source  = &quot;terraform-aws-modules/eks/aws&quot;
  version = &quot;18.29.0&quot;

  cluster_name    = local.cluster_name
  cluster_version = &quot;1.22&quot;

  vpc_id     = module.vpc.vpc_id
  subnet_ids = module.vpc.private_subnets

  # Required for Karpenter role below
  enable_irsa = true

  node_security_group_additional_rules = {
    ingress_nodes_karpenter_port = {
      description                   = &quot;Cluster API to Node group for Karpenter webhook&quot;
      protocol                      = &quot;tcp&quot;
      from_port                     = 8443
      to_port                       = 8443
      type                          = &quot;ingress&quot;
      source_cluster_security_group = true
    }
  }

  node_security_group_tags = {
    # NOTE - if creating multiple security groups with this module, only tag the
    # security group that Karpenter should utilize with the following tag
    # (i.e. - at most, only one security group should have this tag in your account)
    &quot;karpenter.sh/discovery/${local.cluster_name}&quot; = local.cluster_name
  }

  # Only need one node to get Karpenter up and running.
  # This ensures core services such as VPC CNI, CoreDNS, etc. are up and running
  # so that Karpenter can be deployed and start managing compute capacity as required
  eks_managed_node_groups = {
    initial = {
      instance_types = [&quot;m5.large&quot;]
      # Not required nor used - avoid tagging two security groups with same tag as well
      create_security_group = false

      min_size     = 1
      max_size     = 1
      desired_size = 1

      iam_role_additional_policies = [
        &quot;arn:${local.partition}:iam::aws:policy/AmazonSSMManagedInstanceCore&quot;, # Required by Karpenter
        &quot;arn:${local.partition}:iam::aws:policy/AmazonEKSWorkerNodePolicy&quot;,
        &quot;arn:${local.partition}:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly&quot;, #for access to ECR images
        &quot;arn:${local.partition}:iam::aws:policy/CloudWatchAgentServerPolicy&quot;
      ]

      tags = {
        # This will tag the launch template created for use by Karpenter
        &quot;karpenter.sh/discovery/${local.cluster_name}&quot; = local.cluster_name
      }
    }
  }
}

#The EKS module creates an IAM role for the EKS managed node group nodes. We’ll use that for Karpenter.
#We need to create an instance profile we can reference.
#Karpenter can use this instance profile to launch new EC2 instances and those instances will be able to connect to your cluster.
resource &quot;aws_iam_instance_profile&quot; &quot;karpenter&quot; {
  name = &quot;KarpenterNodeInstanceProfile-${local.cluster_name}&quot;
  role = module.eks.eks_managed_node_groups[&quot;initial&quot;].iam_role_name
}

#Create the KarpenterController IAM Role
#Karpenter requires permissions like launching instances, which means it needs an IAM role that grants it access. The config
#below will create an AWS IAM Role, attach a policy, and authorize the Service Account to assume the role using IRSA. We will
#create the ServiceAccount and connect it to this role during the Helm chart install.

module &quot;karpenter_irsa&quot; {
  source  = &quot;terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks&quot;
  version = &quot;5.3.3&quot;

  role_name                          = &quot;karpenter-controller-${local.cluster_name}&quot;
  attach_karpenter_controller_policy = true

  karpenter_tag_key               = &quot;karpenter.sh/discovery/${local.cluster_name}&quot;
  karpenter_controller_cluster_id = module.eks.cluster_id
  karpenter_controller_node_iam_role_arns = [
    module.eks.eks_managed_node_groups[&quot;initial&quot;].iam_role_arn
  ]

  oidc_providers = {
    ex = {
      provider_arn               = module.eks.oidc_provider_arn
      namespace_service_accounts = [&quot;karpenter:karpenter&quot;]
    }
  }
}

#Install Karpenter Helm Chart
#Use helm to deploy Karpenter to the cluster. We are going to use the helm_release Terraform resource to do the deploy and pass in the
#cluster details and IAM role Karpenter needs to assume.
provider &quot;helm&quot; {
  kubernetes {
    host                   = module.eks.cluster_endpoint
    cluster_ca_certificate = base64decode(module.eks.cluster_certificate_authority_data)

    exec {
      api_version = &quot;client.authentication.k8s.io/v1beta1&quot;
      command     = &quot;aws&quot;
      args        = [&quot;eks&quot;, &quot;get-token&quot;, &quot;--cluster-name&quot;, local.cluster_name]
    }
  }
}

resource &quot;helm_release&quot; &quot;karpenter&quot; {
  namespace        = &quot;karpenter&quot;
  create_namespace = true

  name       = &quot;karpenter&quot;
  repository = &quot;https://charts.karpenter.sh&quot;
  chart      = &quot;karpenter&quot;
  version    = &quot;v0.16.1&quot;

  set {
    name  = &quot;serviceAccount.annotations.eks\\.amazonaws\\.com/role-arn&quot;
    value = module.karpenter_irsa.iam_role_arn
  }

  set {
    name  = &quot;clusterName&quot;
    value = module.eks.cluster_id
  }

  set {
    name  = &quot;clusterEndpoint&quot;
    value = module.eks.cluster_endpoint
  }

  set {
    name  = &quot;aws.defaultInstanceProfile&quot;
    value = aws_iam_instance_profile.karpenter.name
  }
}

#Provisioner
#Create a default provisioner using the command below. This provisioner configures instances to connect to your cluster’s endpoint and
#discovers resources like subnets and security groups using the cluster’s name.
#This provisioner will create capacity as long as the sum of all created capacity is less than the specified limit.
provider &quot;kubectl&quot; {
  apply_retry_count      = 5
  host                   = module.eks.cluster_endpoint
  cluster_ca_certificate = base64decode(module.eks.cluster_certificate_authority_data)
  load_config_file       = false

  exec {
    api_version = &quot;client.authentication.k8s.io/v1beta1&quot;
    command     = &quot;aws&quot;
    args        = [&quot;eks&quot;, &quot;get-token&quot;, &quot;--cluster-name&quot;, module.eks.cluster_id]
  }
}

resource &quot;kubectl_manifest&quot; &quot;karpenter_provisioner&quot; {
  yaml_body = &lt;&lt;-YAML
  apiVersion: karpenter.sh/v1alpha5
  kind: Provisioner
  metadata:
    name: default
  spec:
    requirements:
      - key: karpenter.sh/capacity-type
        operator: In
        values: [&quot;spot&quot;]
    limits:
      resources:
        cpu: 1000
    provider:
      subnetSelector:
        Name: &quot;*private*&quot;
      securityGroupSelector:
        karpenter.sh/discovery/${module.eks.cluster_id}: ${module.eks.cluster_id}
      tags:
        karpenter.sh/discovery/${module.eks.cluster_id}: ${module.eks.cluster_id}
    ttlSecondsAfterEmpty: 30
  YAML

  depends_on = [
    helm_release.karpenter
  ]
}

cat &lt;&lt;EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: inflate
spec:
  replicas: 0
  selector:
    matchLabels:
      app: inflate
  template:
    metadata:
      labels:
        app: inflate
    spec:
      terminationGracePeriodSeconds: 0
      containers:
        - name: inflate
          image: public.ecr.aws/eks-distro/kubernetes/pause:3.2
          resources:
            requests:
              cpu: 1
EOF
</code></pre>
<p>kubectl scale deployment inflate --replicas 5</p>
<p>kubectl logs -f -n karpenter -l app.kubernetes.io/name=karpenter -c controller</p>
<p>DEBUG   controller.provisioning Relaxing soft constraints for pod since it previously failed to schedule, removing: spec.topologySpreadConstraints = {&quot;maxSkew&quot;:1,&quot;topologyKey&quot;:&quot;topology.kubernetes.io/zone&quot;,&quot;whenUnsatisfiable&quot;:&quot;ScheduleAnyway&quot;,&quot;labelSelector&quot;:{&quot;matchLabels&quot;:{&quot;app.kubernetes.io/instance&quot;:&quot;karpenter&quot;,&quot;app.kubernetes.io/name&quot;:&quot;karpenter&quot;}}} {&quot;commit&quot;: &quot;b157d45&quot;, &quot;pod&quot;: &quot;karpenter/karpenter-5755bb5b54-rh65t&quot;}
2022-09-10T00:13:13.122Z</p>
<p>ERROR   controller.provisioning Could not schedule pod, incompatible with provisioner &quot;default&quot;, incompatible requirements, key karpenter.sh/provisioner-name, karpenter.sh/provisioner-name DoesNotExist not in karpenter.sh/provisioner-name In [default] {&quot;commit&quot;: &quot;b157d45&quot;, &quot;pod&quot;: &quot;karpenter/karpenter-5755bb5b54-rh65t&quot;}</p>

Gary
<p>I belive this is due to the pod topology defined in the Karpenter deployment here:</p>
<p><a href="https://github.com/aws/karpenter/blob/main/charts/karpenter/values.yaml#L73-L77" rel="nofollow noreferrer">https://github.com/aws/karpenter/blob/main/charts/karpenter/values.yaml#L73-L77</a></p>
<p>, you can read further on what pod topologySpreadConstraints does here:</p>
<p><a href="https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/" rel="nofollow noreferrer">https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/</a></p>
<p>If you increase the desired_size to 2 which matches the default deployment replicas above, that should resove the error.</p>
