<p>I recently evaluated Kubernetes with a simple test project and I was able to update image of StatefulSet with command like this:</p>

<pre><code>kubectl set image statefulset/cloud-stateful-set cloud-stateful-container=ncccloud:v716
</code></pre>

<p>I'm now trying to get our real system to work in Kubernetes and the pods don't do anything when I try to update image, even though I'm using basically the same command.</p>

<p>It says:</p>

<blockquote>
  <p>statefulset.apps "cloud-stateful-set" image updated</p>
</blockquote>

<p>And <code>kubectl describe statefulset.apps/cloud-stateful-set</code> says:</p>

<blockquote>
  <p>Image: ncccloud:v716"</p>
</blockquote>

<p>But <code>kubectl describe pod cloud-stateful-set-0</code> and <code>kubectl describe pod cloud-stateful-set-1</code> say:</p>

<blockquote>
  <p>"Image: ncccloud:latest"</p>
</blockquote>

<p>The ncccloud:latest is an image which doesn't work:</p>

<pre><code>$ kubectl get pods
NAME                               READY     STATUS             RESTARTS   AGE
cloud-stateful-set-0               0/1       CrashLoopBackOff   7          13m
cloud-stateful-set-1               0/1       CrashLoopBackOff   7          13m
mssql-deployment-6cd4ff766-pzz99   1/1       Running            1          55m
</code></pre>

<p>Another strange thing is that every time I try to apply the StatefulSet it says <strong>configured</strong> instead of <strong>unchanged</strong>.</p>

<pre><code>$ kubectl apply -f k8s/cloud-stateful-set.yaml
statefulset.apps "cloud-stateful-set" configured
</code></pre>

<p>Here is my cloud-stateful-set.yaml:</p>

<pre><code>apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: cloud-stateful-set
  labels:
    app: cloud
    group: service
spec:
  replicas: 2
#  podManagementPolicy: Parallel
  serviceName: cloud-stateful-set
  selector:
    matchLabels:
      app: cloud
  template:
    metadata:
      labels:
        app: cloud
        group: service
    spec:
      containers:
        - name: cloud-stateful-container
          image: ncccloud:latest
          imagePullPolicy: Never
          ports:
            - containerPort: 80
          volumeMounts:
            - name: cloud-stateful-storage
              mountPath: /cloud-stateful-data
  volumeClaimTemplates:
    - metadata:
        name: cloud-stateful-storage
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Mi
</code></pre>

<p>Here is full output of <code>kubectl describe pod/cloud-stateful-set-1</code>:</p>

<pre><code>Name:           cloud-stateful-set-1
Namespace:      default
Node:           docker-for-desktop/192.168.65.3
Start Time:     Tue, 02 Jul 2019 11:03:01 +0300
Labels:         app=cloud
                controller-revision-hash=cloud-stateful-set-5c9964c897
                group=service
                statefulset.kubernetes.io/pod-name=cloud-stateful-set-1
Annotations:    &lt;none&gt;
Status:         Running
IP:             10.1.0.20
Controlled By:  StatefulSet/cloud-stateful-set
Containers:
  cloud-stateful-container:
    Container ID:   docker://3ec26930c1a81caa39d5c5a16c4e25adf7584f90a71e0110c0b03ecb60dd9592
    Image:          ncccloud:latest
    Image ID:       docker://sha256:394427c40e964e34ca6c9db3ce1df1f8f6ce34c4ba8f3ab10e25da6e89678830
    Port:           80/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    139
      Started:      Tue, 02 Jul 2019 11:19:03 +0300
      Finished:     Tue, 02 Jul 2019 11:19:03 +0300
    Ready:          False
    Restart Count:  8
    Environment:    &lt;none&gt;
    Mounts:
      /cloud-stateful-data from cloud-stateful-storage (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-gzxpx (ro)
Conditions:
  Type           Status
  Initialized    True
  Ready          False
  PodScheduled   True
Volumes:
  cloud-stateful-storage:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  cloud-stateful-storage-cloud-stateful-set-1
    ReadOnly:   false
  default-token-gzxpx:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-gzxpx
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  &lt;none&gt;
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason                 Age                From                         Message
  ----     ------                 ----               ----                         -------
  Normal   Scheduled              19m                default-scheduler            Successfully assigned cloud-stateful-set-1 to docker-for-desktop
  Normal   SuccessfulMountVolume  19m                kubelet, docker-for-desktop  MountVolume.SetUp succeeded for volume "pvc-4c9e1796-9c9a-11e9-998f-00155d64fa03"
  Normal   SuccessfulMountVolume  19m                kubelet, docker-for-desktop  MountVolume.SetUp succeeded for volume "default-token-gzxpx"
  Normal   Pulled                 17m (x5 over 19m)  kubelet, docker-for-desktop  Container image "ncccloud:latest" already present on machine
  Normal   Created                17m (x5 over 19m)  kubelet, docker-for-desktop  Created container
  Normal   Started                17m (x5 over 19m)  kubelet, docker-for-desktop  Started container
  Warning  BackOff                4m (x70 over 19m)  kubelet, docker-for-desktop  Back-off restarting failed container
</code></pre>

<p>Here is full output of <code>kubectl describe statefulset.apps/cloud-stateful-set</code>:</p>

<pre><code>Name:               cloud-stateful-set
Namespace:          default
CreationTimestamp:  Tue, 02 Jul 2019 11:02:59 +0300
Selector:           app=cloud
Labels:             app=cloud
                    group=service
Annotations:        kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"apps/v1","kind":"StatefulSet","metadata":{"annotations":{},"labels":{"app":"cloud","group":"service"},"name":"cloud-stateful-set","names...
Replicas:           2 desired | 2 total
Pods Status:        2 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
  Labels:  app=cloud
           group=service
  Containers:
   cloud-stateful-container:
    Image:        ncccloud:v716
    Port:         80/TCP
    Host Port:    0/TCP
    Environment:  &lt;none&gt;
    Mounts:
      /cloud-stateful-data from cloud-stateful-storage (rw)
  Volumes:  &lt;none&gt;
Volume Claims:
  Name:          cloud-stateful-storage
  StorageClass:
  Labels:        &lt;none&gt;
  Annotations:   &lt;none&gt;
  Capacity:      10Mi
  Access Modes:  [ReadWriteOnce]
Events:
  Type    Reason            Age   From                    Message
  ----    ------            ----  ----                    -------
  Normal  SuccessfulCreate  25m   statefulset-controller  create Pod cloud-stateful-set-0 in StatefulSet cloud-stateful-set successful
  Normal  SuccessfulCreate  25m   statefulset-controller  create Pod cloud-stateful-set-1 in StatefulSet cloud-stateful-set successful
</code></pre>

<p>I'm using Docker Desktop on Windows, if it matters.</p>

<p>in my case imagePullPolicy was set to Always already:</p>
<pre><code>kubectl patch statefulset web --type='json' -p='[{&quot;op&quot;: &quot;replace&quot;, &quot;path&quot;: &quot;/spec/template/spec/containers/0/image&quot;, &quot;value&quot;:&quot;gcr.io/google_containers/nginx-slim:0.8&quot;}]'
</code></pre>
<p>helped in my case, see k8s docs: <a href="https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/#rolling-update" rel="nofollow noreferrer">https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/#rolling-update</a></p>
