{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/11720","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/11720/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/11720/comments","events_url":"https://api.github.com/repos/apache/spark/issues/11720/events","html_url":"https://github.com/apache/spark/pull/11720","id":140910808,"number":11720,"title":"[SPARK-13902][SCHEDULER] Make DAGScheduler.getAncestorShuffleDependencies() return in topological order to ensure building ancestor stages first.","user":{"login":"ueshin","id":506656,"avatar_url":"https://avatars.githubusercontent.com/u/506656?v=3","gravatar_id":"","url":"https://api.github.com/users/ueshin","html_url":"https://github.com/ueshin","followers_url":"https://api.github.com/users/ueshin/followers","following_url":"https://api.github.com/users/ueshin/following{/other_user}","gists_url":"https://api.github.com/users/ueshin/gists{/gist_id}","starred_url":"https://api.github.com/users/ueshin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ueshin/subscriptions","organizations_url":"https://api.github.com/users/ueshin/orgs","repos_url":"https://api.github.com/users/ueshin/repos","events_url":"https://api.github.com/users/ueshin/events{/privacy}","received_events_url":"https://api.github.com/users/ueshin/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2016-03-15T08:52:09Z","updated_at":"2016-03-28T08:26:41Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/11720","html_url":"https://github.com/apache/spark/pull/11720","diff_url":"https://github.com/apache/spark/pull/11720.diff","patch_url":"https://github.com/apache/spark/pull/11720.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\n`DAGScheduler`sometimes generate incorrect stage graph.\r\nSome stages are generated for the same shuffleId twice or more and they are referenced by the child stages because the building order of the graph is not correct.\r\n\r\nThis patch is fixing it.\r\n\r\n## How was this patch tested?\r\n\r\nI added the sample RDD graph to show the illegal stage graph to `DAGSchedulerSuite`.\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/202298736","html_url":"https://github.com/apache/spark/pull/11720#issuecomment-202298736","issue_url":"https://api.github.com/repos/apache/spark/issues/11720","id":202298736,"user":{"login":"ueshin","id":506656,"avatar_url":"https://avatars.githubusercontent.com/u/506656?v=3","gravatar_id":"","url":"https://api.github.com/users/ueshin","html_url":"https://github.com/ueshin","followers_url":"https://api.github.com/users/ueshin/followers","following_url":"https://api.github.com/users/ueshin/following{/other_user}","gists_url":"https://api.github.com/users/ueshin/gists{/gist_id}","starred_url":"https://api.github.com/users/ueshin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ueshin/subscriptions","organizations_url":"https://api.github.com/users/ueshin/orgs","repos_url":"https://api.github.com/users/ueshin/repos","events_url":"https://api.github.com/users/ueshin/events{/privacy}","received_events_url":"https://api.github.com/users/ueshin/received_events","type":"User","site_admin":false},"created_at":"2016-03-28T08:26:40Z","updated_at":"2016-03-28T08:26:40Z","body":"@maropu Thank you for your review.\r\nI modified as you mentioned.\r\n\r\n> do we get wrong answers from this kind of incorrect stage graphs?\r\n\r\nI don't think this kind of incorrect stage graphs causes the wrong answers for now."}}