<p>I'm running some jobs using Spark on K8S and sometimes my executors will die mid-job. Whenever that happens the driver immediately deletes the failed pod and spawns a new one.</p>

<p>Is there a way to stop Spark from deleting terminated executor pods? It would make debugging the failure a lot easier.</p>

<p>Right now I'm already collecting the logs of all pods to another storage so I can see the logs. But it's quite a hassle to query through logs for every pod and I won't be able to see K8S metadata for them.</p>

Jochen Ullrich
<p>This setting was added in <a href="https://issues.apache.org/jira/browse/SPARK-25515" rel="nofollow noreferrer">SPARK-25515</a>. It sadly isn't available for the currently released version but it should be made available in Spark 3.0.0</p>
