{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/14079","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/14079/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/14079/comments","events_url":"https://api.github.com/repos/apache/spark/issues/14079/events","html_url":"https://github.com/apache/spark/pull/14079","id":164189047,"number":14079,"title":"[SPARK-8425][CORE] Application Level Blacklisting","user":{"login":"squito","id":71240,"avatar_url":"https://avatars.githubusercontent.com/u/71240?v=3","gravatar_id":"","url":"https://api.github.com/users/squito","html_url":"https://github.com/squito","followers_url":"https://api.github.com/users/squito/followers","following_url":"https://api.github.com/users/squito/following{/other_user}","gists_url":"https://api.github.com/users/squito/gists{/gist_id}","starred_url":"https://api.github.com/users/squito/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/squito/subscriptions","organizations_url":"https://api.github.com/users/squito/orgs","repos_url":"https://api.github.com/users/squito/repos","events_url":"https://api.github.com/users/squito/events{/privacy}","received_events_url":"https://api.github.com/users/squito/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":57,"created_at":"2016-07-06T22:26:00Z","updated_at":"2016-10-27T04:32:30Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/14079","html_url":"https://github.com/apache/spark/pull/14079","diff_url":"https://github.com/apache/spark/pull/14079.diff","patch_url":"https://github.com/apache/spark/pull/14079.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nThis builds upon the blacklisting introduced in SPARK-17675 to add blacklisting of executors and nodes for an entire Spark application.  Resources are blacklisted based on tasks that fail, in tasksets that eventually complete successfully; they are automatically returned to the pool of active resources based on a timeout.  Full details are available in a design doc attached to the jira. \r\n\r\n## How was this patch tested?\r\n\r\nAdded unit tests, ran them via Jenkins, also ran a handful of them in a loop to check for flakiness.\r\n\r\nThe added tests include:\r\n* verifying BlacklistTracker works correctly\r\n* verifying TaskSchedulerImpl interacts with BlacklistTracker correctly (via a mock BlacklistTracker)\r\n* an integration test for the entire scheduler with blacklisting in a few different scenarios"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/256544155","html_url":"https://github.com/apache/spark/pull/14079#issuecomment-256544155","issue_url":"https://api.github.com/repos/apache/spark/issues/14079","id":256544155,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2016-10-27T04:32:30Z","updated_at":"2016-10-27T04:32:30Z","body":"**[Test build #67620 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67620/consoleFull)** for PR 14079 at commit [`cdd9f33`](https://github.com/apache/spark/commit/cdd9f336c11572dd104b3e762defb9a257e9655f)."}}
{"actor":{"display_login":"SparkQA"}}