{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/14557","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/14557/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/14557/comments","events_url":"https://api.github.com/repos/apache/spark/issues/14557/events","html_url":"https://github.com/apache/spark/pull/14557","id":170095253,"number":14557,"title":"[SPARK-16709][CORE] Kill the running task if stage failed","user":{"login":"shenh062326","id":9527867,"avatar_url":"https://avatars.githubusercontent.com/u/9527867?v=3","gravatar_id":"","url":"https://api.github.com/users/shenh062326","html_url":"https://github.com/shenh062326","followers_url":"https://api.github.com/users/shenh062326/followers","following_url":"https://api.github.com/users/shenh062326/following{/other_user}","gists_url":"https://api.github.com/users/shenh062326/gists{/gist_id}","starred_url":"https://api.github.com/users/shenh062326/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shenh062326/subscriptions","organizations_url":"https://api.github.com/users/shenh062326/orgs","repos_url":"https://api.github.com/users/shenh062326/repos","events_url":"https://api.github.com/users/shenh062326/events{/privacy}","received_events_url":"https://api.github.com/users/shenh062326/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2016-08-09T06:58:26Z","updated_at":"2016-08-15T04:12:50Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/14557","html_url":"https://github.com/apache/spark/pull/14557","diff_url":"https://github.com/apache/spark/pull/14557.diff","patch_url":"https://github.com/apache/spark/pull/14557.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nAt SPARK-16709,  when a stage failed, but the running task is still running, the retry stage will rerun the running task, it could cause TaskCommitDeniedException and task retry forever.\r\n\r\nHere is the log:\r\n\r\n```\r\n16/07/28 05:22:15 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 1.0 (TID 175, 10.215.146.81, partition 1,PROCESS_LOCAL, 1930 bytes)\r\n16/07/28 05:28:35 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 1.1 (TID 207, 10.196.147.232, partition 1,PROCESS_LOCAL, 1930 bytes)\r\n16/07/28 05:28:48 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 1.0 (TID 175) in 393261 ms on 10.215.146.81 (3/50)\r\n16/07/28 05:34:11 WARN scheduler.TaskSetManager: Lost task 1.0 in stage 1.1 (TID 207, 10.196.147.232): TaskCommitDenied (Driver denied task commit) for job: 1, partition: 1, attemptNumber: 207\r\n16/07/28 05:34:40 INFO scheduler.TaskSetManager: Starting task 1.1 in stage 1.1 (TID 227, 10.196.129.22, partition 1,PROCESS_LOCAL, 1930 bytes)\r\n16/07/28 05:40:08 WARN scheduler.TaskSetManager: Lost task 1.1 in stage 1.1 (TID 227, 10.196.129.22): TaskCommitDenied (Driver denied task commit) for job: 1, partition: 1, attemptNumber: 227\r\n16/07/28 05:42:28 INFO scheduler.TaskSetManager: Starting task 1.2 in stage 1.1 (TID 250, 10.215.146.82, partition 1,PROCESS_LOCAL, 1930 bytes)\r\n...\r\n```\r\n1 task 1.0 in stage1.0 start\r\n2 stage1.0 failed, start stage1.1.\r\n3 task 1.0 in stage1.1 start\r\n4 task 1.0 in stage1.0 finished.\r\n5 task 1.0 in stage1.1 failed with TaskCommitDenied Exception, then retry forever.\r\n\r\n\r\n\r\n## How was this patch tested?\r\n\r\n(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)\r\n\r\n\r\n(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/239726360","html_url":"https://github.com/apache/spark/pull/14557#issuecomment-239726360","issue_url":"https://api.github.com/repos/apache/spark/issues/14557","id":239726360,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2016-08-15T04:12:50Z","updated_at":"2016-08-15T04:12:50Z","body":"**[Test build #63768 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63768/consoleFull)** for PR 14557 at commit [`fbe31eb`](https://github.com/apache/spark/commit/fbe31eb63fa34a19c1b7c44e4ec40fb31835a3fc).\n * This patch passes all tests.\n * This patch merges cleanly.\n * This patch adds no public classes."}}
{"actor":{"display_login":"SparkQA"}}