<p>Using Terraform I spin up the following resources for my primary using a unique service account for this cluster:</p>
<pre><code>resource &quot;google_container_cluster&quot; &quot;primary&quot; {
  name     = var.gke_cluster_name
  location = var.region
  
  # We can't create a cluster with no node pool defined, but we want to only use
  # separately managed node pools. So we create the smallest possible default
  # node pool and immediately delete it.
  remove_default_node_pool = true
  initial_node_count       = 1

  ip_allocation_policy {}
  networking_mode = &quot;VPC_NATIVE&quot;

  node_config {
    service_account = google_service_account.cluster_sa.email
    oauth_scopes = [
      &quot;https://www.googleapis.com/auth/cloud-platform&quot;
    ]
  }

  cluster_autoscaling {
    enabled = true
    
    resource_limits {
      resource_type = &quot;cpu&quot;
      maximum = 40
      minimum = 3
    }

    resource_limits {
      resource_type = &quot;memory&quot;
      maximum = 100
      minimum = 12
    }
  }

  network    = google_compute_network.vpc.name
  subnetwork = google_compute_subnetwork.subnet.name
}

resource &quot;google_container_node_pool&quot; &quot;primary_nodes&quot; {
  name       = &quot;${google_container_cluster.primary.name}-node-pool&quot;
  location   = var.region
  cluster    = google_container_cluster.primary.name
  node_count = var.gke_num_nodes

  node_config {

    service_account = google_service_account.cluster_sa.email
    oauth_scopes = [
      &quot;https://www.googleapis.com/auth/cloud-platform&quot;
    ]

    labels = {
      env = var.project_id
    }
    disk_size_gb = 150
    preemptible  = true
    machine_type = var.machine_type
    tags         = [&quot;gke-node&quot;, &quot;${var.project_id}-gke&quot;]
    metadata = {
      disable-legacy-endpoints = &quot;true&quot;
    }
  }
}
</code></pre>
<p>Even though I provide with the nodes with the appropriate permissions to pull from the google container registry (<code>roles/containerregistry.ServiceAgent</code>) sometimes I get an <code>ImagePullError</code> randomly from kubernetes:</p>
<pre><code>Unexpected status code [manifests latest]: 401 Unauthorized
</code></pre>
<p>After using the following command to inspect the service accounts assigned to the node pools:</p>
<p><code>gcloud container clusters describe master-env --zone=&quot;europe-west2&quot; | grep &quot;serviceAccount&quot;</code></p>
<p>I see the following output:</p>
<pre><code>serviceAccount: default
serviceAccount: master-env@&lt;project-id&gt;.iam.gserviceaccount.com
serviceAccount: master-env@&lt;project-id&gt;.iam.gserviceaccount.com
</code></pre>
<p>Indicating that although I've specified the correct service account to assign to the nodes, for some reason (I think for the <code>primary</code> pool) it instead assigns the <code>default</code> service account which uses the wrong <code>oauth</code> scopes:</p>
<pre><code>oauthScopes:
    - https://www.googleapis.com/auth/logging.write
    - https://www.googleapis.com/auth/monitoring
</code></pre>
<p>Instead of <code>https://www.googleapis.com/auth/cloud-platform</code>.</p>
<p><em><strong>How can I make sure the same service account is used for all nodes?</strong></em></p>
<h2>Edit 1:</h2>
<p>After implementing the fix from @GariSingh now all my node-pools use the same <code>Service Account</code> as expected however I still get the <code>unexpected status code [manifests latest]: 401 Unauthorized</code> error sometimes when installing my services onto the cluster.</p>
<p>This unusual, as other services installed onto cluster seem to pull their images from <code>gcr</code> without issue.</p>
<p>Describing the pod events shows the following:</p>
<pre><code>Events:
  Type     Reason       Age                  From               Message
  ----     ------       ----                 ----               -------
  Normal   Scheduled    11m                  default-scheduler  Successfully assigned default/&lt;my-deployment&gt; to gke-master-env-nap-e2-standard-2-&lt;id&gt;
  Warning  FailedMount  11m                  kubelet            MountVolume.SetUp failed for volume &quot;private-key&quot; : failed to sync secret cache: timed out waiting for the condition
  Warning  FailedMount  11m                  kubelet            MountVolume.SetUp failed for volume &quot;kube-api-access-5hh9r&quot; : failed to sync configmap cache: timed out waiting for the condition
  Warning  Failed       9m34s (x5 over 10m)  kubelet            Error: ImagePullBackOff
</code></pre>
<h2>Edit 2:</h2>
<p>The final piece of the puzzle was to add <code>oauth_scopes</code> to the <code>auto_provisioning_defaults</code> similar to the node configs so that the <code>ServiceAccount</code> could be used properly.</p>

<p>Not sure if you intended to use <a href="https://cloud.google.com/kubernetes-engine/docs/how-to/node-auto-provisioning" rel="nofollow noreferrer">Node auto-provisioning (NAP)</a> (which I highly recommend you use unless it does not meet your needs), but the <code>cluster_autoscaling</code> argument for <code>google_container_cluster</code> actually enables this.  It does not enable the cluster autoscaler for individual node pools.</p>
<p>If your goal is to enable cluster autoscaling for the node pool you created in your config and not use NAP, then you'll need to delete the <code>cluster_autoscaling</code> block and add an <a href="https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/container_node_pool#nested_autoscaling" rel="nofollow noreferrer">autoscaling block</a> under your <code>google_container_node_pool</code> resource and change <code>node_count</code> to <code>initial_node_count</code>:</p>
<pre><code>resource &quot;google_container_node_pool&quot; &quot;primary_nodes&quot; {
  name       = &quot;${google_container_cluster.primary.name}-node-pool&quot;
  location   = var.region
  cluster    = google_container_cluster.primary.name
  initial_node_count = var.gke_num_nodes

  node_config {
    autoscaling {
      min_node_count = var.min_nodes
      max_node_count = var.max_nodes
    }
    service_account = google_service_account.cluster_sa.email
    oauth_scopes = [
      &quot;https://www.googleapis.com/auth/cloud-platform&quot;
    ]

    labels = {
      env = var.project_id
    }
    disk_size_gb = 150
    preemptible  = true
    machine_type = var.machine_type
    tags         = [&quot;gke-node&quot;, &quot;${var.project_id}-gke&quot;]
    metadata = {
      disable-legacy-endpoints = &quot;true&quot;
    }
  }
}
</code></pre>
<p>(the above assume you set variables for min and max nodes)</p>
<p>If you do want to use NAP, then you'll need to add an <a href="https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/container_cluster#nested_auto_provisioning_defaults" rel="nofollow noreferrer">auto_provisioning_defaults block</a> and configure the <code>service_account</code> property:</p>
<pre><code>resource &quot;google_container_cluster&quot; &quot;primary&quot; {
  name     = var.gke_cluster_name
  location = var.region
  
  # We can't create a cluster with no node pool defined, but we want to only use
  # separately managed node pools. So we create the smallest possible default
  # node pool and immediately delete it.
  remove_default_node_pool = true
  initial_node_count       = 1

  ip_allocation_policy {}
  networking_mode = &quot;VPC_NATIVE&quot;

  node_config {
    service_account = google_service_account.cluster_sa.email
    oauth_scopes = [
      &quot;https://www.googleapis.com/auth/cloud-platform&quot;
    ]
  }

  cluster_autoscaling {
    enabled = true
    
    auto_provisioning_defaults {
      service_account = google_service_account.cluster_sa.email
    }      

    resource_limits {
      resource_type = &quot;cpu&quot;
      maximum = 40
      minimum = 3
    }

    resource_limits {
      resource_type = &quot;memory&quot;
      maximum = 100
      minimum = 12
    }
  }

  network    = google_compute_network.vpc.name
  subnetwork = google_compute_subnetwork.subnet.name
}
</code></pre>
