<p>We deployed Gridgain cluster in Google Kubernetes cluster and it working properly with persistence enable. We need to auto scale enable. At scale up no any errors, but at the scale down given &quot;Partition loss&quot;. We need to recover this loss partitions using control.sh script. But it is not possible at every time.</p>
<p>What is the solution for this? Is scale down not working for Gridgain nodes?</p>

Nuwan Sameera
<p>In addition to the @alamar's answer, you need to ensure that your nodes are being stopped gracefully. <a href="https://ignite.apache.org/docs/latest/starting-nodes#shutting-down-nodes" rel="nofollow noreferrer">Graceful shutdown</a> performs additional verification on your data and ensures that you won't have lost partitions in your cluster when a node leaves.</p>
<p>You might verify graceful shutdown by searching for the following messages:</p>
<pre><code>Invoking shutdown hook...
...
Ensuring that caches have sufficient backups and local rebalance completion...
</code></pre>
<p>You can set it with the following options:</p>
<ul>
<li>Configuring <a href="https://www.gridgain.com/docs/latest/developers-guide/starting-nodes" rel="nofollow noreferrer">IGNITE_WAIT_FOR_BACKUPS_ON_SHUTDOWN</a> system property</li>
<li>Setting <a href="https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html#setShutdownPolicy-org.apache.ignite.ShutdownPolicy-" rel="nofollow noreferrer">IgniteConfiguration#setShutdownPolicy</a> to be <em>GRACEFUL</em> explicitely with XML/code config. The default one is <a href="https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/ShutdownPolicy.html#IMMEDIATE" rel="nofollow noreferrer">IMMEDIATE</a>.</li>
</ul>
<p>It feels that sometimes I can see that #1 approach with the system property might not work well in k8s world. Please check the latter one with explicit Ignite config adjustment if the first one seems to be not working properly.</p>
<p>In addition to the above, you might want to disable baslientAdutoAdjustment (if enabled) to prevent data rebalancing for short scaling up and down.</p>
