<p>I created a schedule configuration inside my Gcloud project to create snapshots of a bunch of virtual disks.</p>
<p>Now I want to add my schedule configuration to my disks, but I dont know how to do it in a automated way, because I have more than 1200 disks.</p>
<p>I tryed to use a POD with a cron inside, but I cannot execute the kubectl command to list all my persistent volumes:</p>
<pre><code>kubectl describe pv | grep &quot;Name&quot; | awk 'NR % 2 == 1' | awk '{print $2}'
</code></pre>
<p>I want to use this list with the next command in a Loop to add automatically my programmed schedule to my disks:</p>
<pre><code>gcloud compute disks add-resource-policies [DISK_NAME] --resource-policies [SCHEDULE_NAME] --zone [ZONE]
</code></pre>
<p>Thanks in advance for your help.</p>
<p>Edit 1: After some comments I changed my code to add a Kubernetes CronJob, but the result is the same, the code doesn't work (the pod is created, but it gives me an error: ImagePullBackOff):</p>
<pre><code>resource &quot;kubernetes_cron_job&quot; &quot;schedulerdemo&quot; {
  metadata {
    name = &quot;schedulerdemo&quot;
  }
  spec {
    concurrency_policy            = &quot;Replace&quot;
    failed_jobs_history_limit     = 5
    schedule                      = &quot;*/5 * * * *&quot;
    starting_deadline_seconds     = 10
    successful_jobs_history_limit = 10
    job_template {
      metadata {}
      spec {
        backoff_limit = 2
        ttl_seconds_after_finished    = 10
        template {
          metadata {}
          spec {
            container {
              name    = &quot;scheduler&quot;
              image   = &quot;imgscheduler&quot;
              command = [&quot;/bin/sh&quot;, &quot;-c&quot;, &quot;date; kubectl describe pv | grep 'Name' | awk 'NR % 2 == 1' | awk '{print $2}'&quot;]
            }
          }
        }
      }
    }
  }
}
</code></pre>

Barragán Louisenbairn
<p>Answering the comment:</p>
<blockquote>
<p>Ok, shame on me, wrong image name. Now I have an error in the Container Log: /bin/sh: kubectl: not found</p>
</blockquote>
<p>It means that the image that you are using doesn't have <code>kubectl</code> installed (or it's not in the <code>PATH</code>). You can use image: <code>google/cloud-sdk:latest</code>. This image already have <code>cloud-sdk</code> installed which includes:</p>
<ul>
<li><code>gcloud</code></li>
<li><code>kubectl</code></li>
</ul>
<hr />
<p>To run a <code>CronJob</code> that will get the information about <code>PV</code>'s and change the configuration of <code>GCP</code> storage  you will need following accesses:</p>
<ul>
<li><code>Kubernetes/GKE</code> API(<code>kubectl</code>) - <code>ServiceAccount</code> with a <code>Role</code> and <code>RoleBinding</code>.</li>
<li><code>GCP</code> API (<code>gcloud</code>) - <code>Google Service account</code> with <code>IAM</code> permissions for storage operations.</li>
</ul>
<p>I found this links helpful when assigning permissions to list <code>PV</code>'s:</p>
<ul>
<li><em><a href="https://kubernetes.io/docs/reference/access-authn-authz/rbac/" rel="nofollow noreferrer">Kubernetes.io: RBAC</a></em></li>
<li><em><a href="https://success.mirantis.com/article/user-unable-to-list-persistent-volumes" rel="nofollow noreferrer">Success.mirantis.com: Article: User unable to list persistent volumes</a></em></li>
</ul>
<p>The recommended way to assign specific permissions for <code>GCP</code> access:</p>
<blockquote>
<p>Workload Identity is the recommended way to access Google Cloud services from applications running within GKE due to its improved security properties and manageability.</p>
<p>-- <em><a href="https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity" rel="nofollow noreferrer">Cloud.google.com: Kubernetes Engine: Workload Identity: How to</a></em></p>
</blockquote>
<p>I encourage you to read documentation I linked above and check other <a href="https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity#alternatives" rel="nofollow noreferrer">alternatives</a>.</p>
<hr />
<p>As for the script used inside of a <code>CronJob</code>. You should look for <code>pdName</code> instead of <code>Name</code> as the <code>pdName</code> is representation of the <code>gce-pd</code> disk in <code>GCP</code> (assuming that we are talking about in-tree plugin).</p>
<p>You will have multiple options to retrieve the disk name from the API to use it in the <code>gcloud</code> command.</p>
<p>One of the options:</p>
<pre class="lang-sh prettyprint-override"><code>kubectl get pv -o yaml | grep &quot;pdName&quot; | cut -d &quot; &quot; -f 8 | xargs -n 1 gcloud compute disks add-resource-policies --zone=ZONE --resource-policies=POLICY
</code></pre>
<blockquote>
<p>Disclaimer!</p>
<p>Please treat above command only as an <strong>example</strong>.</p>
</blockquote>
<p>Above command will get the <code>PDName</code> attribute from the <code>PV</code>'s and iterate with each of them in the command after <code>xargs</code>.</p>
<p>Some of the things to take into consideration when creating a script/program:</p>
<ul>
<li>Running this command more than once on a single disk will issue an error that you cannot assign multiple policies. You  could have a list of already configured disks that do not require assigning a policy.</li>
<li>Consider using <code>.spec.concurrencyPolicy: Forbid</code> instead of <code>Replace</code>. Replaced <code>CronJob</code> will start from the beginning iterating over all of those disks. Command could not complete in the desired time and <code>CronJob</code> will be replaced.</li>
<li>You will need to check for the correct <code>kubectl</code> version as the official support allows +1/-1 version difference between client and a server (<code>cloud-sdk:latest</code> uses <code>v1.19.3</code>).</li>
</ul>
<hr />
<p>I highly encourage you to look on other methods to backup your <code>PVC</code>'s (like for example <code>VolumeSnapshots</code>).</p>
<p>Take a look on below links for more reference/ideas:</p>
<ul>
<li><em><a href="https://stackoverflow.com/a/60931080/12257134">Stackoverflow.com: Answer: Periodic database backup in kubernetes</a></em></li>
<li><em><a href="https://stash.run/docs/v0.9.0-rc.2/guides/latest/volumesnapshot/pvc/" rel="nofollow noreferrer">Stash.run: Guides: Latest: Volumesnapshot: PVC</a></em></li>
<li><em><a href="https://velero.io/" rel="nofollow noreferrer">Velero.io</a></em></li>
</ul>
<p>It's worth to mention that:</p>
<blockquote>
<p>CSI drivers are the future of storage extension in Kubernetes. <strong>Kubernetes has announced that the in-tree volume plugins are expected to be removed from Kubernetes in version 1.21.</strong> For details, see <a href="https://kubernetes.io/blog/2019/12/09/kubernetes-1-17-feature-csi-migration-beta/" rel="nofollow noreferrer">Kubernetes In-Tree to CSI Volume Migration Moves to Beta</a>. After this change happens, existing volumes using in-tree volume plugins will communicate through CSI drivers instead.</p>
<p>-- <em><a href="https://cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/gce-pd-csi-driver#benefits_of_using" rel="nofollow noreferrer">Cloud.google.com: Kubernetes Engine: Persistent Volumes: GCE PD CSI Driver: Benefits of using</a></em></p>
</blockquote>
<p>Switching to <code>CSI</code> plugin for your <code>StorageClass</code> will allow you to use <code>Volume Snapshots</code> inside of <code>GKE</code>:</p>
<blockquote>
<p>Volume snapshots let you create a copy of your volume at a specific point in time. You can use this copy to bring a volume back to a prior state or to provision a new volume.</p>
<p>-- <em><a href="https://cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/volume-snapshots" rel="nofollow noreferrer">Cloud.google.com: Kubernetes Engine: Persistent Volumes: Volume snaphosts: How to</a></em></p>
</blockquote>
<hr />
<p>Additional resources:</p>
<ul>
<li><em><a href="https://cloud.google.com/kubernetes-engine/docs/concepts/persistent-volumes" rel="nofollow noreferrer">Cloud.google.com: Kubernetes Engine: Persistent Volumes</a></em></li>
<li><em><a href="https://cloud.google.com/kubernetes-engine/docs/how-to/cronjobs" rel="nofollow noreferrer">Cloud.google.com: Kubernetes Engine: Cronjobs: How to</a></em></li>
<li><em><a href="https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/cron_job" rel="nofollow noreferrer">Terraform.io: Kubernetes: CronJob</a></em></li>
<li><em><a href="https://cloud.google.com/compute/docs/disks/create-snapshots" rel="nofollow noreferrer">Cloud.google.com: Compute: Disks: Create snapshot</a></em></li>
</ul>
