<p>I try to build a continuous deployment pipeline for my GKE cluster. I use my own gitlab-runner as CI pipeline build and push images to gcr.io/PROJECT/APP:google tag there. </p>

<p>Is there any possibility to implement the rolling restart of the containers that use this image after its update? I have seen a lot of examples of how to do it using Jenkins and Google Source Repository directly in a Kubernetes cluster, but is there any possibility to trigger only on image changes?</p>

<p>I have found something that I need here <a href="https://cloud.google.com/container-registry/docs/configuring-notifications" rel="nofollow noreferrer">https://cloud.google.com/container-registry/docs/configuring-notifications</a>. But still, I have no idea how to connect these notifications with the cluster. </p>

<p>After some tests, finally I made it works using PubSub and Kubernetes Cronjob.</p>

<h2>How it works:</h2>

<p>When a new image is pushe to <strong>Container Registry</strong>, a message is sent to <strong>Pub/Sub</strong> that contains some important data, like this:</p>

<pre><code>{
  "action":"INSERT",
  "digest":"gcr.io/my-project/hello-world@sha256:6ec128e26cd5...",
  "tag":"gcr.io/my-project/hello-world:1.1"
}
</code></pre>

<p>The <code>action</code> with value <code>INSERT</code> means that a new image was pushed to Pub/Sub.
The key <code>tag</code> contains the name of the image that was pushed.</p>

<p>So, all we need to do is, read this data and update the deployment image.</p>

<p>First, We need some code to retrieve the message from Pub/Sub. Unfortunately I can't find anything "ready" for this task, so you need to create you own. <a href="https://cloud.google.com/pubsub/docs/pull" rel="nofollow noreferrer">Here</a> there are some examples of how you can retrive the messages from Pub/Sub.</p>

<blockquote>
  <p>As a <strong><em>proof-of-concept</em></strong>, my choice was to use a shell script (<a href="https://github.com/MrKoopaKiller/docker-gcloud-kubectl/blob/master/imageUpdater.sh" rel="nofollow noreferrer">imageUpdater.sh</a>) to retrieve the message from Pub/Sub and execute the <code>kubectl set image...</code> command to update the deployment image.</p>
</blockquote>

<p>Second, create a Cronjob using the first code to read the message and update the deployment.</p>

<blockquote>
  <p>In my example, I've create a docker image with gcloud and kubectl command to perform the tasks, you can find the code <a href="https://github.com/MrKoopaKiller/docker-gcloud-kubectl/blob/master/Dockerfile" rel="nofollow noreferrer">Here</a>.</p>
</blockquote>

<p>But, to make it all work, your must to grant permissions to job pod's to execute "kubectl set image", and for this wee need to configure the <a href="https://kubernetes.io/docs/reference/access-authn-authz/rbac/" rel="nofollow noreferrer">RBAC</a> permissions.</p>

<blockquote>
  <p>To create a "isolated" PoC, I will create every resources describe here in a new namespace named "myns". The RBAC will only work in this name space, because <code>Role</code> is namespace, if you what to use in all namespaces, change to <code>ClusterRole</code></p>
</blockquote>

<h2>1.  PubSub Configuration</h2>

<p>First of all, you need to configure Container Registry to sent messages to Pub/Sub. You can follow <a href="https://cloud.google.com/container-registry/docs/configuring-notifications" rel="nofollow noreferrer">this</a> guide. </p>

<blockquote>
  <p>In this example I will use a nginx image to demonstrate.</p>
</blockquote>

<pre><code>gcloud pubsub topics create projects/[PROJECT-ID]/topics/gcr
</code></pre>

<p>From the system where Docker images are pushed or tagged run the following command:</p>

<pre><code>gcloud pubsub subscriptions create nginx --topic=gcr
</code></pre>

<h2>2. GKE Cluster</h2>

<p>GKE needs permission to access Pub/Sub, and it can be done only when a new cluster is created using the <code>--scope "https://www.googleapis.com/auth/pubsub"</code>.  So I will create a new cluster to our example:</p>

<p><strong>Create a new cluster to our example:</strong></p>

<pre><code>gcloud container clusters create "my-cluster" --num-nodes "1" --scopes "https://www.googleapis.com/auth/devstorage.read_only","https://www.googleapis.com/auth/logging.write","https://www.googleapis.com/auth/monitoring","https://www.googleapis.com/auth/cloud-platform","https://www.googleapis.com/auth/pubsub","https://www.googleapis.com/auth/servicecontrol","https://www.googleapis.com/auth/service.management.readonly","https://www.googleapis.com/auth/trace.append"
</code></pre>

<p>More information about <code>scopes</code> <a href="https://adilsoncarvalho.com/changing-a-running-kubernetes-cluster-permissions-a-k-a-scopes-3e90a3b95636" rel="nofollow noreferrer">here</a>.</p>

<p><strong>Getting the cluster credentials:</strong></p>

<pre><code>gcloud container clusters get-credentials my-cluster
</code></pre>

<h2>3. Configuring RBAC permissions</h2>

<p>As mentioned before all resources will be created in the namespace <code>myns</code>. So let's create the namespace:</p>

<p><code>kubectl create ns myns</code></p>

<p>After that we can create a new Service account called <code>sa-image-update</code> and apply the RBAC permissions:</p>

<pre class="lang-yaml prettyprint-override"><code>apiVersion: v1
kind: ServiceAccount
metadata:
  name: sa-image-update
  namespace: myns
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: myns
  name: role-set-image
rules:
- apiGroups: ["apps", "extensions"]
  resources: ["deployments"]
  verbs: ["get", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: rolebinding-set-image
  namespace: myns
roleRef:
  kind: Role
  name: role-set-image
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
  name: sa-image-update
  namespace: myns
</code></pre>

<h2>4. Confimap</h2>

<p>To make it easiest as possible, I will create a configmap with the shell script file that will be mounted and executed by the pod:</p>

<pre><code># Download script
wget https://raw.githubusercontent.com/MrKoopaKiller/docker-gcloud-kubectl/master/imageUpdater.sh
# Create configmap
kubectl create configmap imageupdater -n myns --from-file imageUpdater.sh
</code></pre>

<h2>5. CronJob</h2>

<p>The shell script need 3 variables to work:</p>

<p><code>PROJECT-NAME</code>: The name of gcloud project
<code>DEPLOYMENT-NAME</code>: The name of the deployment that will be updated
<code>IMAGE-NAME</code>: The name of the image to update without the tag. </p>

<p>In this case, my example deployment will be called <code>nginx</code> and the image <code>nginx</code>.</p>

<p>The image is from the dockerfile I mentioned before, you can find <a href="https://github.com/MrKoopaKiller/docker-gcloud-kubectl/blob/master/Dockerfile" rel="nofollow noreferrer">here</a> and build it.</p>

<pre class="lang-yaml prettyprint-override"><code>apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: image-updater
  namespace: myns
spec:
  schedule: "*/2 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          serviceAccountName: sa-image-update
          volumes:
            - name: imageupdater
              configMap:
                name: imageupdater
          containers:
          - name: image-updater
            image: &lt;your_custom_image&gt;
            volumeMounts:
            - name: imageupdater
              mountPath: /bin/imageUpdater.sh
              subPath: imageUpdater.sh
            command: ['bash', '/bin/imageUpdater.sh', 'PROJECT-NAME', 'nginx', 'nginx']
          restartPolicy: Never
</code></pre>

<p>OK, everything is done. Now we need to create a deployment as example to demonstrated:</p>

<h2>Example deployment: nginx</h2>

<pre class="lang-yaml prettyprint-override"><code>apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  namespace: myns
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - name: http
          containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-svc
  namespace: myns
spec:
  type: LoadBalancer
  selector:
    app: nginx
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
</code></pre>

<p>Ok, now when your gitlab push a new image to COntainer Registry, a message will be sent to Pub/Sub, the cronjob will runs every 2 minutes, verify if the image name is <code>nginx</code> and if yes, will run the <code>kubectl set image</code>.</p>

<h3>References:</h3>

<p><a href="https://medium.com/better-programming/k8s-tips-using-a-serviceaccount-801c433d0023" rel="nofollow noreferrer">https://medium.com/better-programming/k8s-tips-using-a-serviceaccount-801c433d0023</a></p>

<p><a href="https://cloud.google.com/solutions/integrating-microservices-with-pubsub#creating_a_gke_cluster" rel="nofollow noreferrer">https://cloud.google.com/solutions/integrating-microservices-with-pubsub#creating_a_gke_cluster</a></p>
