<p>I have terraformed (Terraform version 11.10) a private Kubernetes cluster on the Google Kubernetes Engine (GKE) using the following <code>.tf</code>:</p>

<pre><code>module "nat" {
  source     = "GoogleCloudPlatform/nat-gateway/google"
  region     = "europe-west1"
  network    = "default"
  subnetwork = "default"
}

resource "google_container_node_pool" "cluster_1_np" {
  name               = "cluster-1-np"
  region             = "europe-west1"
  cluster            = "${google_container_cluster.cluster_1.name}"
  initial_node_count = 1

  lifecycle {
    ignore_changes = ["node_count"]
  }

  autoscaling {
    min_node_count = 1
    max_node_count = 50
  }

  management {
    auto_repair  = true
    auto_upgrade = true
  }

  node_config {
    oauth_scopes = [
      "https://www.googleapis.com/auth/compute",
      "https://www.googleapis.com/auth/devstorage.read_only",
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring",
      "https://www.googleapis.com/auth/pubsub",
    ]

    tags = ["${module.nat.routing_tag_regional}"]
  }
}

resource "google_container_cluster" "cluster_1" {
  provider                 = "google-beta"
  name                     = "cluster-1"
  region                   = "europe-west1"
  remove_default_node_pool = true

  private_cluster_config {
    enable_private_endpoint = false
    enable_private_nodes    = true
    master_ipv4_cidr_block  = "172.16.0.0/28"
  }

  ip_allocation_policy {
    create_subnetwork = true
  }

  lifecycle {
    ignore_changes = ["initial_node_count", "network_policy", "node_config", "node_pool"]
  }

  node_pool {
    name = "default-pool"
  }

  addons_config {
    http_load_balancing {
      disabled = false
    }

    horizontal_pod_autoscaling {
      disabled = false
    }
  }

  master_authorized_networks_config {
    cidr_blocks = [
      {
        cidr_block   = "&lt;MY_OFFICE_CIDR&gt;"
        display_name = "Office"
      },
    ]
  }
}
</code></pre>

<p>Which works great, giving me a private cluster (and the NAT works, giving the nodes access to the internet), and machines in my office can run <code>kubectl</code> commands to interact with it no bother.</p>

<p>The problem I now face is integrating any web-based Continuous Integration (CI) or Continuous Deployment (CD). Private clusters are a new feature of the Google Cloud Platform (GCP), and the documentation is a bit lacking in this area.</p>

<p>My attempts thus far have completely failed, my networking knowledge is simply insufficient. I tried <a href="https://cloud.google.com/solutions/creating-kubernetes-engine-private-clusters-with-net-proxies" rel="noreferrer">this solution</a> but it seems the automation machine must be on the same network as the proxy.</p>

<p>I found <a href="https://stackoverflow.com/questions/51944817/google-cloud-build-deploy-to-gke-private-cluster">this similar SO question</a> (almost exactly the same but his is Cloud Build specific). In the comments to one of the answers of that question the OP mentions he found a workaround where he temporarily modifies the master authorized networks of the build machine but he has not stated exactly how he is carrying this out.</p>

<p>I attempted to replicate his workaround but the relevant <code>gcloud</code> commands seem to be able to <code>update</code> the list of networks, or completely remove all of them, not add/remove one at a time as his comment suggests.</p>

<p>Help from networking wizards would be much appreciated.</p>

Liam Humphreys
<p>This is a common problem while interfacing with CI systems like <a href="https://circleci.com/" rel="nofollow noreferrer">CircleCI</a> or <a href="https://travis-ci.org/" rel="nofollow noreferrer">Travis</a> that live in the public cloud. You can use this command to update your <code>master authorized networks</code></p>

<pre><code>gcloud container clusters update [CLUSTER_NAME] \
  --enable-master-authorized-networks \
  --master-authorized-networks=&lt;MY_OFFICE_CIDR&gt;,&lt;NEW-CIDR-FROM-CI&gt; \
  --zone=&lt;your-zone&gt;
</code></pre>

<p>To remove the CI system network you can do something like this (just remove the network from the cli):</p>

<pre><code>gcloud container clusters update [CLUSTER_NAME] \
  --enable-master-authorized-networks \
  --master-authorized-networks=&lt;MY_OFFICE_CIDR&gt; \
  --zone=&lt;your-zone&gt;
</code></pre>

<p>To completely remove all authorized networks (disable):</p>

<pre><code>gcloud container clusters update [CLUSTER_NAME] \
  --no-enable-master-authorized-networks 
</code></pre>

<p>You can also do it from the UI:</p>

<p><a href="https://i.stack.imgur.com/3pl9L.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/3pl9L.png" alt="authorized networks"></a></p>

<p>It's actually documented <a href="https://cloud.google.com/kubernetes-engine/docs/how-to/authorized-networks" rel="nofollow noreferrer">here</a>.</p>
