title: mongo statefulset FailedMount volume after scale up replica
body: <p>I am testing mongodb statefulset to have 3 replicas after using it for a while. It has some pods that restart and show error FailedMount.</p>
  <p>this my file mongo-statefulset.yaml</p>
  <pre><code>apiVersion: apps/v1
  kind: StatefulSet
  metadata:
    name: mongodb-stg
    namespace: mongo
  spec:
    serviceName: mongo
    replicas: 3
    selector:
      matchLabels:
        app: mongo
    template:
      metadata:
        labels:
          app: mongo
          selector: mongo
      spec:
        terminationGracePeriodSeconds: 30
        serviceAccount: mongo-account
        containers:
        - name: mongodb
          image: docker.io/mongo:4.2
          env:
          command: [&quot;/bin/sh&quot;]
          args: [&quot;-c&quot;, &quot;mongod --replSet=rs0 --bind_ip_all&quot;]
          resources:
            requests:
              cpu: 50m
              memory: 128Mi
          ports:
          - name: mongo-port
            containerPort: 27017
          volumeMounts:
          - name: mongo-data
            mountPath: /data/db
    volumeClaimTemplates:
    - metadata:
        name: mongo-data
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 1Gi
  </code></pre>
  <p>I am checked PV and PVC is found that all 3 disks still exist.</p>
  <p>check &quot;kubectl get volumeattachment&quot; is not have mount node with PV of pods at fail.
  (node status is Ready all)</p>
  <p>This logs on the nodes failing to start that pod.</p>
  <pre><code>{MESSAGE: E0825 18:33:45.862680 1629 kubelet.go:1761] &quot;Unable to attach or mount volumes for pod; skipping pod&quot; err=&quot;unmounted volumes=[mongo-data], unattached volumes=[mongo-data kube-api-access-mjvpr]: timed out waiting for the condition&quot; pod=&quot;mongo/mongo-stg-0&quot;, PRIORITY: 6, SYSLOG_FACILITY: 3…
  
  {MESSAGE: E0825 18:33:45.862835 1629 pod_workers.go:951] &quot;Error syncing pod, skipping&quot; err=&quot;unmounted volumes=[mongo-data], unattached volumes=[mongo-data kube-api-access-mjvpr]: timed out waiting for the condition&quot; pod=&quot;mongo/mongo-stg-0&quot; podUID=de8858da-251c-48c3-be7e-c6bda3286350, PRIORITY: 6,…
  </code></pre>
  <p>Tried delete pvc and scale down/up replica mongo pods is comeback Running but using it for a while is fail again.</p>
  <p>Why not create volumeattachment on node at pods run.</p>