<p>Basically I'm trying to deploy a cluster on GCE via <code>kubeadm</code> with StorageClass supported (without using Google Kubernetes Engine).</p>

<p>Say I deployed a cluster with a master node at Tokyo, and 3 work nodes in Hong Kong, Taiwan and Oregon.</p>

<pre><code>NAME              STATUS   ROLES    AGE     VERSION
k8s-node-hk       Ready    &lt;none&gt;   3h35m   v1.14.2
k8s-node-master   Ready    master   3h49m   v1.14.2
k8s-node-oregon   Ready    &lt;none&gt;   3h33m   v1.14.2
k8s-node-tw       Ready    &lt;none&gt;   3h34m   v1.14.2
</code></pre>

<p><code>Kube-controller-manager</code> and <code>kubelet</code> both started with <code>cloud-provider=gce</code>, and now I can apply <code>StorageClass</code> and <code>PersistentVolumeClaim</code> then get disks created automatically (say a disk in Taiwan) on GCP and get <code>PV</code> and <code>PVC</code> bound. </p>

<p><code>kubectl get pvc</code>:</p>

<pre><code>NAME      STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
eth-pvc   Bound    pvc-bf35e3c9-81e2-11e9-8926-42010a920002   10Gi       RWO            slow           137m
</code></pre>

<p><code>kubectl get pv</code></p>

<pre><code>NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM             STORAGECLASS   REASON   AGE
pvc-bf35e3c9-81e2-11e9-8926-42010a920002   10Gi       RWO            Delete           Bound    default/eth-pvc   slow                    137m
</code></pre>

<p>However, <code>kube-controller-manager</code> cannot find the Taiwan node and mount the disk to the node in same zone, and it logged (we can see the zone <code>asia-northeast1-a</code> is not correct):</p>

<pre><code>I0529 07:25:46.366500       1 reconciler.go:288] attacherDetacher.AttachVolume started for volume "pvc-bf35e3c9-81e2-11e9-8926-42010a920002" (UniqueName: "kubernetes.io/gce-pd/kubernetes-dynamic-pvc-bf35e3c9-81e2-11e9-8926-42010a920002") from node "k8s-node-tokyo" 
E0529 07:25:47.296824       1 attacher.go:102] Error attaching PD "kubernetes-dynamic-pvc-bf35e3c9-81e2-11e9-8926-42010a920002" to node "k8s-node-tokyo": GCE persistent disk not found: diskName="kubernetes-dynamic-pvc-bf35e3c9-81e2-11e9-8926-42010a920002" zone="asia-northeast1-a"
</code></pre>

<p>The kubelet on each node started with <code>--cloud-provider=gce</code>, but I didn't find how to configure the zone. And when I checked kubelet's log, I find this flag is <strong>already deprecated</strong> on kubernetes v1.14.2 (the latest in May 2019).</p>

<pre><code>May 29 04:36:03 k8s-node-tw kubelet[29971]: I0529 04:36:03.623704   29971 server.go:417] Version: v1.14.2
May 29 04:36:03 k8s-node-tw kubelet[29971]: W0529 04:36:03.624039   29971 plugins.go:118] WARNING: gce built-in cloud provider is now deprecated. The GCE provider is deprecated and will be removed in a future release
</code></pre>

<p>However, kubelet annotated <code>k8s-node-tw</code> node with the correct zone and region:</p>

<pre><code>May 29 04:36:05 k8s-node-tw kubelet[29971]: I0529 04:36:05.157665   29971 kubelet_node_status.go:331] Adding node label from cloud provider: beta.kubernetes.io/instance-type=n1-standard-1
May 29 04:36:05 k8s-node-tw kubelet[29971]: I0529 04:36:05.158093   29971 kubelet_node_status.go:342] Adding node label from cloud provider: failure-domain.beta.kubernetes.io/zone=asia-east1-a
May 29 04:36:05 k8s-node-tw kubelet[29971]: I0529 04:36:05.158201   29971 kubelet_node_status.go:346] Adding node label from cloud provider: failure-domain.beta.kubernetes.io/region=asia-east1
</code></pre>

<p>Thanks for reading here. My question is:</p>

<p>If it's possible, how can I configure <code>kubelet</code> or <code>kube-controller-manager</code> correctly to make it support GCP's storage class and the created disks attached and mounted successfully?</p>

<p>==================K8s config files======================</p>

<p>Deployment (related part):</p>

<pre><code>  volumes:
  - name: data
    persistentVolumeClaim:
      claimName: eth-pvc
  - name: config
    configMap:
      name: {{ template "ethereum.fullname" . }}-geth-config
  - name: account
    secret:
      secretName: {{ template "ethereum.fullname" . }}-geth-account
</code></pre>

<p>PVC:</p>

<pre><code>apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: eth-pvc
spec:
  storageClassName: slow
  resources:
    requests:
      storage: 10Gi
  accessModes:
    - ReadWriteOnce
</code></pre>

<p>SC:</p>

<pre><code>apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: slow
provisioner: kubernetes.io/gce-pd
parameters:
  type: pd-standard
  replication-type: none
  zone: asia-east1-a
</code></pre>

kigawas
<p>After several days' research, I found that the reason is:</p>

<ol>
<li>Master node is in <code>asia-northeast1-a</code> (Tokyo)</li>
<li>Worker nodes are in <code>asia-east1-a</code> (Taiwan) and other zones</li>
<li><a href="https://github.com/kubernetes/cloud-provider-gcp/blob/master/vendor/k8s.io/kubernetes/pkg/cloudprovider/providers/gce/gce.go#L352" rel="nofollow noreferrer"><code>cloud-provider-gcp</code></a> only search the zone in one region (normally the master node's zone, but you can specify it by setting <code>local-zone</code> in the cloud config file), which means it can only support one zone or multiple zones in one region by default</li>
</ol>

<p>Conclusion:</p>

<p>In order to support multiple zones among multiple regions, we need to modify the gce provider code of configuration, like add another field to configure which zones should be searched.</p>

<p>==========================UPDATE=========================</p>

<p>I modified the k8s code to add a <code>extra-zones</code> config field like <a href="https://github.com/kubernetes/kubernetes/compare/release-1.14...kigawas:release-1.14" rel="nofollow noreferrer">this diff on github</a> to make it work on my use case.</p>
