<p>I have a GKE Cluster current scaled upto multiple nodes, the scale up happened during high load due to a DDOS attack on our services , but now the cluster is unable to scale down due to redis-master and redis-slave, this is ending up costing alot of overhead costs that has become an issue for us now.</p>
<p>The autoscale down shows the error: <code>no.scale.down.node.pod.has.local.storage</code>. I have seen in multiple answers setting the option <code>cluster-autoscaler.kubernetes.io/safe-to-evict</code> to true should solve the issue (GCP Suggests the same) but before i do that i want to know if doing so and scaling down redis-slave instances can cause any data loss? Any suggestion for this would be ideal as currently we are paying over 2x of what is needed.</p>
<p>I also checked the config as well , and i saw there are volumes the redis-master and redis-slaves under volumes and the config yaml also has that as :</p>
<pre><code>volumeClaimTemplates:
  - apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      creationTimestamp: null
      labels:
        app: redis
        component: master
        heritage: Tiller
        release: prod-redis
      name: redis-data
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 20Gi
      volumeMode: Filesystem
</code></pre>

<p>Redis master or slave mostly save the data in to memory for backup and restore purpose it takes snapshot every min or second based on config.</p>
<p>You can checkout the deployment config or configmap if any snapshot option running or not. In Redis terms known as the <strong>AOF</strong> and <strong>RDB</strong> backup.</p>
<p>Whenver any Redis POD crash in K8s it starts and restores the data from the above file if present.</p>
<p>So make sure you <strong>exec</strong> into the POD and check files are present or not.</p>
<pre><code>image: redislabs/redis
args: [&quot;--requirepass&quot;, &quot;admin&quot;, &quot;--appendonly&quot;, &quot;yes&quot;, &quot;--save&quot;, &quot;900&quot;, &quot;1&quot;, &quot;--save&quot;, &quot;30&quot;, &quot;2&quot;]
</code></pre>
<p>Check the config or YAML of deployment might be having some options like above.</p>
<p>Ref Document : <a href="https://redis.io/topics/persistence" rel="nofollow noreferrer">https://redis.io/topics/persistence</a></p>
<p>If files are not there you can manually take the backup so if POD crashes it start with old data.</p>
<p>Command to generate the backup in background: <code>BG SAVE</code></p>
<p>if you have large data in POD things might go wrong, <code>BG SAVE</code> will kick the process to save the data to the <code>filesystem</code> in PVC which can lead to Higher resource requirements and POD will get killed if resource set.</p>
<p>Once data is saved using the background command you can start removing the slaves with their respective PVCs.</p>
<p>So if <strong>AOF</strong> and <strong>RDB</strong> are disabled data loss will be there.</p>
<p>Just RDB is also not a good option as it takes periodic backup while <strong>AOF</strong> is an instant option.</p>
<p>If just <strong>RDB</strong> there, could be chances it has taken <strong>snapshot</strong> by night and you remove POD in the morning so lastest data after snapshot is in memory you might not get in snapshot.</p>
