<p>I'm using persistent volume claim to store data in container:</p>

<pre><code>kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: test-pvc
  labels:
    type: amazonEBS
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
</code></pre>

<p>Declaration in spec:</p>

<pre><code>spec:
  volumes:
  - name: test-data-vol
    persistentVolumeClaim:
      claimName: test-pvc
  containers:
  - name: test
    image: my.docker.registry/test:1.0
    volumeMounts:
    - mountPath: /var/data
      name: test-data-vol
</code></pre>

<p>When I started it first time, this volume was mounted correctly. But when I Tried to update container image:</p>

<pre><code>-    image: my.docker.registry/test:1.0
+    image: my.docker.registry/test:1.1
</code></pre>

<p>This volume failed to mount to new pod:</p>

<pre><code># kubectl get pods
test-7655b79cb6-cgn5r       0/1     ContainerCreating   0          3m
test-bf6498559-42vvb        1/1     Running             0          11m

# kubectl describe test-7655b79cb6-cgn5r
Events:
  Type     Reason                 Age   From                                                  Message
  ----     ------                 ----  ----                                                  -------
  Normal   Scheduled              3m5s  default-scheduler                                     Successfully assigned test-7655b79cb6-cgn5r to ip-*-*-*-*.us-west-2.compute.internal
  Warning  FailedAttachVolume     3m5s  attachdetach-controller                               Multi-Attach error for volume "pvc-2312eb4c-c270-11e8-8d4e-065333a7774e" Volume is already exclusively attached to one node and can't be attached to another
  Normal   SuccessfulMountVolume  3m4s  kubelet, ip-*-*-*-*.us-west-2.compute.internal  MountVolume.SetUp succeeded for volume "default-token-x82km"
  Warning  FailedMount            62s   kubelet, ip-*-*-*-*.us-west-2.compute.internal  Unable to mount volumes for pod "test-7655b79cb6-cgn5r(fab0862c-d1cf-11e8-8d4e-065333a7774e)": timeout expired waiting for volumes to attach/mount for pod "test-7655b79cb6-cgn5r". list of unattached/unmounted volumes=[test-data-vol]
</code></pre>

<p>It seems that Kubernetes can't re-attach this volume from one container to another. How to handle it correctly? I need this data on volume to be used by new version of deployment when old version stopped.</p>

Kirill
<p>From the context you provided in your question, I can't tell if your intention was to run a <strong>single instance</strong> stateful application, or a <strong>clustered</strong> stateful application.</p>
<p>I ran into this problem recently and from <a href="https://kubernetes.io/docs/tasks/run-application/run-single-instance-stateful-application/#updating" rel="noreferrer">this section in the docs</a>, here's how to go about this...</p>
<p><strong>If you're running a single instance stateful app:</strong></p>
<ul>
<li>You should <strong>not</strong> scale the app, that is, leave the default value of <code>spec.replicas</code> as 1 if you're using a <code>Deployment</code></li>
<li>You should instruct Kubernetes to <strong>not</strong> use rolling updates, that is, you should set <code>spec.strategy.type</code> to <code>Recreate</code> in your <code>Deployment</code></li>
</ul>
<p>Sample <code>Deployment</code> (from the <a href="https://kubernetes.io/docs/tasks/run-application/run-single-instance-stateful-application/#deploy-mysql" rel="noreferrer">docs</a>):</p>
<pre>
<strong># application/mysql/mysql-deployment.yaml</strong>

apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
<strong>kind: Deployment</strong>
metadata:
  name: mysql
spec:
  selector:
    matchLabels:
      app: mysql
  <strong>strategy:</strong>
    <strong>type: Recreate</strong>
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - image: mysql:5.6
        name: mysql
        env:
          # Use secret in real usage
        - name: MYSQL_ROOT_PASSWORD
          value: password
        ports:
        - containerPort: 3306
          name: mysql
        volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql
      volumes:
      - name: mysql-persistent-storage
        persistentVolumeClaim:
          claimName: mysql-pv-claim
</pre>
<p>And the sample <code>PersistentVolume</code> &amp; <code>PersistentVolumeClaim</code> (from the <a href="https://kubernetes.io/docs/tasks/run-application/run-single-instance-stateful-application/#deploy-mysql" rel="noreferrer">docs</a>):</p>
<pre>
<strong># application/mysql/mysql-pv.yaml</strong>

apiVersion: v1
kind: PersistentVolume
metadata:
  name: mysql-pv-volume
  labels:
    type: local
spec:
  storageClassName: manual
  capacity:
    storage: 20Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/mnt/data"

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pv-claim
spec:
  storageClassName: manual
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi
</pre>
<p>The obvious underlying matter here is that a rolling update will not work, because there can be no more than one pod running at any time. Setting <code>spec.strategy.type</code> to <code>Recreate</code> tells Kubernetes to stop the running pod before deploying a new one, so presumably there will be some downtime, even if minimal.</p>
<p><strong>If you need a clustered stateful application</strong>, then using the already mentioned <code>StatefulSet</code> as a controller type or <code>ReadWriteMany</code> as a storage type would probably be the way to go.</p>
