{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/22011","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/22011/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/22011/comments","events_url":"https://api.github.com/repos/apache/spark/issues/22011/events","html_url":"https://github.com/apache/spark/pull/22011","id":348027900,"node_id":"MDExOlB1bGxSZXF1ZXN0MjA2NDc1NzUx","number":22011,"title":"[SPARK-24822][PySpark] Python support for barrier execution mode","user":{"login":"jiangxb1987","id":4784782,"node_id":"MDQ6VXNlcjQ3ODQ3ODI=","avatar_url":"https://avatars1.githubusercontent.com/u/4784782?v=4","gravatar_id":"","url":"https://api.github.com/users/jiangxb1987","html_url":"https://github.com/jiangxb1987","followers_url":"https://api.github.com/users/jiangxb1987/followers","following_url":"https://api.github.com/users/jiangxb1987/following{/other_user}","gists_url":"https://api.github.com/users/jiangxb1987/gists{/gist_id}","starred_url":"https://api.github.com/users/jiangxb1987/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jiangxb1987/subscriptions","organizations_url":"https://api.github.com/users/jiangxb1987/orgs","repos_url":"https://api.github.com/users/jiangxb1987/repos","events_url":"https://api.github.com/users/jiangxb1987/events{/privacy}","received_events_url":"https://api.github.com/users/jiangxb1987/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":13,"created_at":"2018-08-06T18:09:51Z","updated_at":"2018-08-10T21:31:31Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/22011","html_url":"https://github.com/apache/spark/pull/22011","diff_url":"https://github.com/apache/spark/pull/22011.diff","patch_url":"https://github.com/apache/spark/pull/22011.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nThis PR add python support for barrier execution mode, thus enable launch a job containing barrier stage(s) from PySpark.\r\n\r\nWe just forked the existing `RDDBarrier` and `RDD.barrier()` in Python api.\r\n\r\n## How was this patch tested?\r\n\r\nManually tested:\r\n```\r\n>>> rdd = sc.parallelize([1, 2, 3, 4])\r\n>>> def f(iterator): yield sum(iterator)\r\n... \r\n>>> rdd.barrier().mapPartitions(f).isBarrier() == True\r\nTrue\r\n```\r\n\r\nUnit tests will be added in a follow-up PR that implements BarrierTaskContext on python side."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/412212099","html_url":"https://github.com/apache/spark/pull/22011#issuecomment-412212099","issue_url":"https://api.github.com/repos/apache/spark/issues/22011","id":412212099,"node_id":"MDEyOklzc3VlQ29tbWVudDQxMjIxMjA5OQ==","user":{"login":"AmplabJenkins","id":4031291,"node_id":"MDQ6VXNlcjQwMzEyOTE=","avatar_url":"https://avatars3.githubusercontent.com/u/4031291?v=4","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2018-08-10T21:31:31Z","updated_at":"2018-08-10T21:31:31Z","author_association":"NONE","body":"Test FAILed.\nRefer to this link for build results (access rights to CI server needed): \nhttps://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94575/\nTest FAILed."}}
{"actor":{"display_login":"AmplabJenkins"}}