<p>I have a GKE cluster running with several persistent disks for storage.
To set up a staging environment, I created a second cluster inside the same project.
Now I want to use the data from the persistent disks of the production cluster in the staging cluster.</p>
<p>I already created persistent disks for the staging cluster. What is the best approach to move over the production data to the disks of the staging cluster.</p>

jjmurre
<p>You can use the open source tool <a href="https://velero.io/" rel="nofollow noreferrer">Velero</a> which is designed to migrate Kubernetes cluster resources.</p>
<p>Follow these steps to migrate a persistent disk within GKE clusters:</p>
<ol>
<li>Create a GCS bucket:</li>
</ol>
<pre><code>BUCKET=&lt;your_bucket_name&gt;
gsutil mb gs://$BUCKET/
</code></pre>
<ol start="2">
<li>Create a <a href="https://cloud.google.com/iam/docs/service-accounts" rel="nofollow noreferrer">Google Service Account</a> and store the associated email in a variable for later use:</li>
</ol>
<pre><code>GSA_NAME=&lt;your_service_account_name&gt;
gcloud iam service-accounts create $GSA_NAME \
    --display-name &quot;Velero service account&quot; 

SERVICE_ACCOUNT_EMAIL=$(gcloud iam service-accounts list \
  --filter=&quot;displayName:Velero service account&quot; \
  --format 'value(email)')
</code></pre>
<ol start="3">
<li>Create a custom role for the Service Account:</li>
</ol>
<pre><code>PROJECT_ID=&lt;your_project_id&gt;
ROLE_PERMISSIONS=(
    compute.disks.get
    compute.disks.create
    compute.disks.createSnapshot
    compute.snapshots.get
    compute.snapshots.create
    compute.snapshots.useReadOnly
    compute.snapshots.delete
    compute.zones.get
    storage.objects.create
    storage.objects.delete
    storage.objects.get
    storage.objects.list
)

gcloud iam roles create velero.server \
    --project $PROJECT_ID \
    --title &quot;Velero Server&quot; \
    --permissions &quot;$(IFS=&quot;,&quot;; echo &quot;${ROLE_PERMISSIONS[*]}&quot;)&quot;

gcloud projects add-iam-policy-binding $PROJECT_ID \
    --member serviceAccount:$SERVICE_ACCOUNT_EMAIL \
    --role projects/$PROJECT_ID/roles/velero.server

gsutil iam ch serviceAccount:$SERVICE_ACCOUNT_EMAIL:objectAdmin gs://${BUCKET}
</code></pre>
<ol start="4">
<li>Grant access to Velero:</li>
</ol>
<pre><code>gcloud iam service-accounts keys create credentials-velero \
    --iam-account $SERVICE_ACCOUNT_EMAIL
</code></pre>
<ol start="5">
<li>Download and install Velero on the source cluster:</li>
</ol>
<pre><code>wget https://github.com/vmware-tanzu/velero/releases/download/v1.8.1/velero-v1.8.1-linux-amd64.tar.gz
tar -xvzf velero-v1.8.1-linux-amd64.tar.gz
sudo mv velero-v1.8.1-linux-amd64/velero /usr/local/bin/velero

velero install \
    --provider gcp \
    --plugins velero/velero-plugin-for-gcp:v1.4.0 \
    --bucket $BUCKET \
    --secret-file ./credentials-velero
</code></pre>
<p>Note: The download and installation was performed on a Linux system, which is the OS used by Cloud Shell. If you are managing your GCP resources via Cloud SDK, the release and installation process could vary.</p>
<ol start="6">
<li>Confirm that the velero pod is running:</li>
</ol>
<pre><code>$ kubectl get pods -n velero
NAME                      READY   STATUS    RESTARTS   AGE
velero-xxxxxxxxxxx-xxxx   1/1     Running   0          11s
</code></pre>
<ol start="7">
<li>Create a backup for the PV,PVCs:</li>
</ol>
<pre><code>velero backup create &lt;your_backup_name&gt; --include-resources pvc,pv --selector app.kubernetes.io/&lt;your_label_name&gt;=&lt;your_label_value&gt; 
</code></pre>
<ol start="8">
<li>Verify that your backup was successful with no errors or warnings:</li>
</ol>
<pre><code>$ velero backup describe &lt;your_backup_name&gt;  --details
Name:         your_backup_name
Namespace:    velero
Labels:       velero.io/storage-location=default
Annotations:  velero.io/source-cluster-k8s-gitversion=v1.21.6-gke.1503
              velero.io/source-cluster-k8s-major-version=1
              velero.io/source-cluster-k8s-minor-version=21

Phase:  Completed

Errors:    0
Warnings:  0
</code></pre>
<hr />
<p>Now that the Persistent Volumes are backed up, you can proceed with the migration to the destination cluster following these steps:</p>
<ol>
<li>Authenticate in the destination cluster</li>
</ol>
<pre><code>gcloud container clusters get-credentials &lt;your_destination_cluster&gt; --zone &lt;your_zone&gt; --project &lt;your_project&gt;
</code></pre>
<ol start="2">
<li>Install Velero using the same parameters as step 5 on the first part:</li>
</ol>
<pre><code>velero install \
    --provider gcp \
    --plugins velero/velero-plugin-for-gcp:v1.4.0 \
    --bucket $BUCKET \
    --secret-file ./credentials-velero
</code></pre>
<ol start="3">
<li>Confirm that the velero pod is running:</li>
</ol>
<pre><code>kubectl get pods -n velero
NAME                      READY   STATUS    RESTARTS   AGE
velero-xxxxxxxxxx-xxxxx   1/1     Running   0          19s
</code></pre>
<ol start="4">
<li>To avoid the backup data being overwritten, change the bucket to read-only mode:</li>
</ol>
<pre><code>kubectl patch backupstoragelocation default -n velero --type merge --patch '{&quot;spec&quot;:{&quot;accessMode&quot;:&quot;ReadOnly&quot;}}'
</code></pre>
<ol start="5">
<li>Confirm Velero is able to access the backup from bucket:</li>
</ol>
<pre><code>velero backup describe &lt;your_backup_name&gt; --details
</code></pre>
<ol start="6">
<li>Restore the backed up Volumes:</li>
</ol>
<pre><code>velero restore create --from-backup &lt;your_backup_name&gt;
</code></pre>
<ol start="7">
<li>Confirm that the persistent volumes have been restored on the destination cluster:</li>
</ol>
<pre><code>kubectl get pvc
NAME                                     STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
redis-data-my-release-redis-master-0     Bound    pvc-ae11172a-13fa-4ac4-95c5-d0a51349d914   8Gi        RWO            standard       79s
redis-data-my-release-redis-replicas-0   Bound    pvc-f2cc7e07-b234-415d-afb0-47dd7b9993e7   8Gi        RWO            standard       79s
redis-data-my-release-redis-replicas-1   Bound    pvc-ef9d116d-2b12-4168-be7f-e30b8d5ccc69   8Gi        RWO            standard       79s
redis-data-my-release-redis-replicas-2   Bound    pvc-65d7471a-7885-46b6-a377-0703e7b01484   8Gi        RWO            standard       79s
</code></pre>
<p>Check out this <a href="https://faun.pub/clone-migrate-data-between-kubernetes-clusters-with-velero-e298196ec3d8" rel="nofollow noreferrer">tutorial</a> as a reference.</p>
