<p>I'm trying to create a multi-cluster ingress on Google Kubernetes Engine using <a href="https://github.com/GoogleCloudPlatform/k8s-multicluster-ingress" rel="nofollow noreferrer">kubemci</a>, however when running the following command the program waits indefinitely for the ingress service to get the <code>ingress.gcp.kubernetes.io/instance-groups</code> annotation (as illustrated in the output below).</p>

<p>What is preventing this annotation from being set?</p>

<p><strong>Input</strong></p>

<pre><code>./kubemci create app-mci \
    --ingress=ingress.yaml \
    --gcp-project=app-prod \
    --kubeconfig=mcikubeconfig
</code></pre>

<p><strong>Output</strong></p>

<pre><code>% ./kubemci create app-mci --ingress=ingress.yaml --gcp-project=app-prod --kubeconfig=clusters.yaml        
Created Ingress in cluster: gke_app-prod_europe-west4-a_app-europe-west4
Created Ingress in cluster: gke_app-prod_us-east4-a_app-us-east4
Ensuring health checks
Pod app-deployment-c99578769-xdmql matching service selectors app=app (targetport ): lacks a matching HTTP probe for use in health checks.
Pod app-deployment-c99578769-xgq2m matching service selectors app=app (targetport ): lacks a matching HTTP probe for use in health checks.
Pod app-deployment-c99578769-qms7r matching service selectors app=app (targetport ): lacks a matching HTTP probe for use in health checks.
Pod app-deployment-c99578769-tsrsw matching service selectors app=app (targetport ): lacks a matching HTTP probe for use in health checks.
Path for healthcheck is /
Ensuring health check for port: {SvcName:default/app-service SvcPort:{Type:0 IntVal:80 StrVal:} NodePort:30061 Protocol:HTTP SvcTargetPort: NEGEnabled:false}
Health check mci1-hc-30061--app-mci exists already. Checking if it matches our desired health check
Desired health check exists already
Determining instance groups for cluster gke_app-prod_europe-west4-a_app-europe-west4
Waiting for ingress ( default : app-ingress ) to get ingress.gcp.kubernetes.io/instance-groups annotation.....
Waiting for ingress ( default : app-ingress ) to get ingress.gcp.kubernetes.io/instance-groups annotation.....
Waiting for ingress ( default : app-ingress ) to get ingress.gcp.kubernetes.io/instance-groups annotation.....
⋮
</code></pre>

<p>As you can see my configuration is identical (aside from resource names) to that in the <a href="https://cloud.google.com/kubernetes-engine/docs/how-to/multi-cluster-ingress" rel="nofollow noreferrer">multi-cluster ingress guide</a>:</p>

<p><strong>deployment.yaml</strong></p>

<pre><code>apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-deployment
spec:
  selector:
    matchLabels:
      app: app
  replicas: 2
  template:
    metadata:
      labels:
        app: app
    spec:
      containers:
        - name: app
          image: gcr.io/app-prod/app:tag
          ports:
            - containerPort: 8080
</code></pre>

<p><strong>service.yaml</strong></p>

<pre><code>apiVersion: v1
kind: Service
metadata:
  labels:
    app: app
  name: app-service
spec:
  ports:
    - port: 80
      protocol: TCP
      targetPort: 8080
      name: http
      nodePort: 30061
  selector:
    app: app
  type: NodePort
</code></pre>

<p><strong>ingress.yaml</strong></p>

<pre><code>apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: app-ingress
  annotations:
    kubernetes.io/ingress.global-static-ip-name: app-ip
    kubernetes.io/ingress.class: gce-multi-cluster
spec:
  backend:
    serviceName: app-service
    servicePort: 80
</code></pre>

Luke Gareth Ribchester
<h1>Enable HTTP load balancing</h1>

<p>Enable the HTTP load balancing add-on to allow the <a href="https://github.com/kubernetes/ingress-gce" rel="nofollow noreferrer">load balancer controller</a> to set the <code>ingress.gcp.kubernetes.io/instance-groups</code> annotation.</p>

<h2>Console</h2>

<ol>
<li>Edit a cluster.</li>
<li>Expand <em>add-ons</em>.</li>
<li>Enable HTTP load balancing:</li>
</ol>

<p><a href="https://i.stack.imgur.com/4GSl3m.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/4GSl3m.png" alt="enter image description here"></a></p>

<h2>Command line</h2>

<pre><code>% gcloud container clusters update [CLUSTER_NAME] --update-addons HttpLoadBalancing=ENABLED

Updating ***...done.                                                                                                                                                              
Updated [https://container.googleapis.com/v1/projects/***/zones/us-east4-a/clusters/***].
To inspect the contents of your cluster, go to: https://console.cloud.google.com/kubernetes/workload_/gcloud/us-east4-a/***?project=***
</code></pre>

<p>View the cluster configuration:</p>

<pre><code>% gcloud container clusters describe [CLUSTER_NAME]

# ENABLED
addonsConfig:
  httpLoadBalancing: {}

# DISABLED
addonsConfig:
  httpLoadBalancing:
    disabled: true
</code></pre>

<p><br></p>

<h1>Configure Services</h1>

<p>Ensure that the backend Services used in the multi-cluster ingress are configured correctly.</p>

<p>Services must:</p>

<blockquote>
  <ul>
  <li>Have the same name in all of the clusters.</li>
  <li>Be in the same namespace in all of the clusters.</li>
  <li>Be of type NodePort.</li>
  <li><p>Use the same port number for all of the clusters.</p>
  
  <p>&mdash; <a href="https://cloud.google.com/kubernetes-engine/docs/how-to/multi-cluster-ingress#prepare_your_clusters" rel="nofollow noreferrer">Setting up a multi-cluster Ingress</a>, Google</p></li>
  </ul>
</blockquote>

<p><br></p>

<h3>Credit</h3>

<ul>
<li>Nikhil Jindal for his insight.</li>
<li>Ivan for raising <a href="https://github.com/GoogleCloudPlatform/k8s-multicluster-ingress/issues/204" rel="nofollow noreferrer">this issue</a>.</li>
</ul>
