{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/23136","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/23136/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/23136/comments","events_url":"https://api.github.com/repos/apache/spark/issues/23136/events","html_url":"https://github.com/apache/spark/pull/23136","id":384117539,"node_id":"MDExOlB1bGxSZXF1ZXN0MjMzMzk5Nzk5","number":23136,"title":"[SPARK-25515][K8s] Adds a config option to keep executor pods for debugging","user":{"login":"skonto","id":7945591,"node_id":"MDQ6VXNlcjc5NDU1OTE=","avatar_url":"https://avatars1.githubusercontent.com/u/7945591?v=4","gravatar_id":"","url":"https://api.github.com/users/skonto","html_url":"https://github.com/skonto","followers_url":"https://api.github.com/users/skonto/followers","following_url":"https://api.github.com/users/skonto/following{/other_user}","gists_url":"https://api.github.com/users/skonto/gists{/gist_id}","starred_url":"https://api.github.com/users/skonto/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/skonto/subscriptions","organizations_url":"https://api.github.com/users/skonto/orgs","repos_url":"https://api.github.com/users/skonto/repos","events_url":"https://api.github.com/users/skonto/events{/privacy}","received_events_url":"https://api.github.com/users/skonto/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-11-25T20:50:40Z","updated_at":"2018-11-25T21:04:20Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/23136","html_url":"https://github.com/apache/spark/pull/23136","diff_url":"https://github.com/apache/spark/pull/23136.diff","patch_url":"https://github.com/apache/spark/pull/23136.patch"},"body":"## What changes were proposed in this pull request?\r\nKeeps K8s executor resources present if case of failure or normal termination.\r\nIntroduces a new boolean config option: `spark.kubernetes.deleteExecutors`, with default value set to true.\r\nThe idea is to update Spark K8s backend structures but leave the resources around.\r\nThe assumption is that since entries are not removed from the `removedExecutorsCache` we are immune to updates that refer to the the executor resources previously removed.\r\nThe only delete operation not touched is the one in the `doKillExecutors` method.\r\nReason is right now we dont support [blacklisting](https://issues.apache.org/jira/browse/SPARK-23485) and dynamic allocation with Spark on K8s. In both cases in the future we might want to handle these scenarios although its more complicated.\r\nMore tests can be added if approach is approved. \r\n## How was this patch tested?\r\nManually by running a Spark job and verifying pods are not deleted.\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/441473062","html_url":"https://github.com/apache/spark/pull/23136#issuecomment-441473062","issue_url":"https://api.github.com/repos/apache/spark/issues/23136","id":441473062,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0MTQ3MzA2Mg==","user":{"login":"SparkQA","id":8108735,"node_id":"MDQ6VXNlcjgxMDg3MzU=","avatar_url":"https://avatars1.githubusercontent.com/u/8108735?v=4","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2018-11-25T21:04:20Z","updated_at":"2018-11-25T21:04:20Z","author_association":"NONE","body":"**[Test build #99245 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99245/testReport)** for PR 23136 at commit [`5cdca31`](https://github.com/apache/spark/commit/5cdca3167e2f8acb84a23f9c64e5c3bc524f04ac).\n * This patch passes all tests.\n * This patch merges cleanly.\n * This patch adds no public classes."}}
{"actor":{"display_login":"SparkQA"}}