<p>I'm getting a strange problem on my Terraformed GKE cluster,</p>
<p>I have a deployment that request a GcePersistentVolume with a PVC, when it got created, I have a <code>Can’t scale up nodes</code> notification on my GCloud console.
If I inspect the log, it say that :</p>
<pre><code>reason: {
messageId: &quot;no.scale.up.mig.failing.predicate&quot;
parameters: [
0: &quot;&quot;
1: &quot;pod has unbound immediate PersistentVolumeClaims&quot;
</code></pre>
<p>Without creating this deployment, I have no Scale UP error at all.</p>
<p>The PVC in question :</p>
<pre><code>apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  finalizers:
  - kubernetes.io/pvc-protection
  name: nfs
  namespace: nfs
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: standard
  volumeMode: Filesystem
status:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 10Gi
  phase: Bound
</code></pre>
<p>My deployment is running fine and the PV is directly created and bound to my PVC.</p>
<p>So I find this <code>Can't Scale up Nodes</code> really strange ?</p>
<p>(It's a single zone cluster, with a single NodePool).</p>
<p>Any idea for me ?</p>
<p>Thanks a lot</p>

Valentin Ouvrard
<p>I'm having the same problem. It is weird because if you are creating a PVC in GKE, the PV is created dynamically (and indeed it is), so you go and check with <code>kubectl get pv,pvc --all-namespaces</code> and everything seems normal. But it seems that when a deployment (that uses a PVC) is created and while waiting for the creation of the PVC this error appears and the cluster acknowledges it and displays the alert (creating some false positive alerts). It seems like a timing issue.</p>
<p>One turnaround is to change the value of the <code>storageClassName</code> definition. If instead of <code>standard</code> you use <code>standard-rwo</code> (both appear as default in Storage Classes tab in Storage) the problem seems to disappear. The consequence of this is that the type of the underlying disk changes from <code>Standard persistent disk</code> to <code>Balanced persistent disk</code>. Anyhow, the latter one performs better.</p>
<p>EDIT:
It is about Storage Classes. The volumeBindingMode of the default <code>standard</code> class is <code>Immediate</code>. According to the <a href="https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode" rel="nofollow noreferrer">documentation</a>:</p>
<blockquote>
<p>The Immediate mode indicates that volume binding and dynamic
provisioning occurs once the PersistentVolumeClaim is created. For
storage backends that are topology-constrained and not globally
accessible from all Nodes in the cluster, PersistentVolumes will be
bound or provisioned without knowledge of the Pod's scheduling
requirements. This may result in unschedulable Pods.</p>
<p>A cluster administrator can address this issue by specifying the
WaitForFirstConsumer mode which will delay the binding and
provisioning of a PersistentVolume until a Pod using the
PersistentVolumeClaim is created. PersistentVolumes will be selected
or provisioned conforming to the topology that is specified by the
Pod's scheduling constraints. These include, but are not limited to,
resource requirements, node selectors, pod affinity and anti-affinity,
and taints and tolerations.</p>
</blockquote>
<p>So, if all the properties of the <code>standard</code> <code>Storage class</code> are required to be kept, another solution would be to create another <code>Storage class</code>:</p>
<ol>
<li>Download the YAML of the <code>standard</code> <code>Storage class</code></li>
<li>Change the <code>name</code> definition</li>
<li>Change the property from <code>volumeBindingMode: Immediate</code> to <code>volumeBindingMode: WaitForFirstConsumer</code>.</li>
<li>Apply it (<code>kubectl apply -f &lt;file path&gt;</code> )</li>
<li>And in the <code>storageClassName</code> definition of the PVC, change it to the name of the step #2</li>
</ol>
