<p>Terraform <a href="https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/container_cluster" rel="nofollow noreferrer">google_container_cluster</a> example removes the default node pool.</p>
<pre><code>resource &quot;google_container_cluster&quot; &quot;primary&quot; {
  name     = &quot;my-gke-cluster&quot;
  location = &quot;us-central1&quot;

  # We can't create a cluster with no node pool defined, but we want to only use
  # separately managed node pools. So we create the smallest possible default
  # node pool and immediately delete it.
  remove_default_node_pool = true
  initial_node_count       = 1
}
</code></pre>
<p>However, as the default pool is removed and there can be no node to deploy the system pods.</p>
<p><a href="https://i.stack.imgur.com/vjDDY.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/vjDDY.png" alt="enter image description here" /></a></p>
<h1>Question</h1>
<p>What is the good way to have a node to deploy the system pods?</p>
<ol>
<li>Have a default pool with number of nodes is 1 and schedule system pods in the defaujlt node. Set the min autoscaling node count to 0 and do not schedule system pods in auto scaling pods</li>
<li>Remove the default pool. Set the min autoscaling node count to 1 so that system pods can be deployed in a autoscaling node.</li>
</ol>
<h1>Node pool defnition</h1>
<pre><code>resource &quot;google_container_node_pool&quot; &quot;primary&quot; {
  name     = &quot;${google_container_cluster.primary.name}-node-pool&quot;
  project  = var.PROJECT_ID
  location = var.location

  cluster  = google_container_cluster.primary.name

  #--------------------------------------------------------------------------------
  # Node instantiation based on auto-scaling setting.
  # node_count and autoscaling are mutually exclusive.
  #--------------------------------------------------------------------------------
  node_count = var.autoscaling == true ? null : var.num_nodes
  dynamic &quot;autoscaling&quot; {
    for_each = var.autoscaling ? [1] : []
    content {
      min_node_count = var.min_node_count   # Set to 0 currently
      max_node_count = var.max_node_count
    }
  }

  #--------------------------------------------------------------------------------
  # Node configurations
  #--------------------------------------------------------------------------------
  node_config {
    #--------------------------------------------------------------------------------
    # Service Account, the roles of which the node assumes
    #--------------------------------------------------------------------------------
    service_account = var.service_account

    #--------------------------------------------------------------------------------
    # Instance configurations
    #--------------------------------------------------------------------------------
    machine_type = var.machine_type
    preemptible  = var.node_preemptive
    disk_size_gb = var.disk_size_gb
    disk_type    = var.disk_type

    metadata = {
      disable-legacy-endpoints = &quot;true&quot;
    }

    #--------------------------------------------------------------------------------
    # The K8S labels (key/value pairs) to be applied to each node
    #--------------------------------------------------------------------------------
    labels = var.labels

    oauth_scopes = [
      &quot;https://www.googleapis.com/auth/logging.write&quot;,
      &quot;https://www.googleapis.com/auth/monitoring&quot;,
    ]
    tags = var.tags
  }
}
</code></pre>

<p>Those are system pods that get deployed on each node, maybe kube-dns, kube-proxy however metrics server and other if running as deployment those can be running as single replicas on one of node.</p>
<blockquote>
<p>What is the good way to have a node to deploy the system pods?</p>
</blockquote>
<p>Keep at least <strong>1-2 node</strong> for system pods like metrics server, ingress if you are installing inside kube-system etc.</p>
<blockquote>
<p>Have a default pool with number of nodes is 1 and schedule system pods
in the defaujlt node. Set the min autoscaling node count to 0 and do
not schedule system pods in auto scaling pods</p>
</blockquote>
<p>There is no guarantee if you have set the autoscaling node count to 0 which means it will scale down. There are limitation with GKE about scaling down node you might need to take a look first however yes it's possible you can scale down to zero by updating PDB and checking other limitations.</p>
<p><strong>Limitation</strong> : <a href="https://cloud.google.com/kubernetes-engine/docs/concepts/cluster-autoscaler#limitations" rel="nofollow noreferrer">https://cloud.google.com/kubernetes-engine/docs/concepts/cluster-autoscaler#limitations</a></p>
<blockquote>
<p>Remove the default pool. Set the min autoscaling node count to 1 so
that system pods can be deployed in a autoscaling node.</p>
</blockquote>
<p>This is good if min count is 1 system pods should be up and running on that node, and when you deploy your application your nodes start scaling up as per need while system pods were already running.</p>
<p>You can also scale down to zero in that case if autoscaling is set and you deploy the application it will auto-start the application and system on available nodes so not much to worry about in that case also.</p>
<p>If you don't want to remove the default Node pool and keep running <strong>1</strong> Node, you can use this field : <a href="https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/container_cluster#remove_default_node_pool" rel="nofollow noreferrer">https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/container_cluster#remove_default_node_pool</a></p>
