{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/15722","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/15722/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/15722/comments","events_url":"https://api.github.com/repos/apache/spark/issues/15722/events","html_url":"https://github.com/apache/spark/pull/15722","id":186670374,"number":15722,"title":"[SPARK-18208] [Shuffle] Executor OOM due to a growing LongArray in BytesToBytesMap","user":{"login":"jiexiong","id":6044805,"avatar_url":"https://avatars.githubusercontent.com/u/6044805?v=3","gravatar_id":"","url":"https://api.github.com/users/jiexiong","html_url":"https://github.com/jiexiong","followers_url":"https://api.github.com/users/jiexiong/followers","following_url":"https://api.github.com/users/jiexiong/following{/other_user}","gists_url":"https://api.github.com/users/jiexiong/gists{/gist_id}","starred_url":"https://api.github.com/users/jiexiong/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jiexiong/subscriptions","organizations_url":"https://api.github.com/users/jiexiong/orgs","repos_url":"https://api.github.com/users/jiexiong/repos","events_url":"https://api.github.com/users/jiexiong/events{/privacy}","received_events_url":"https://api.github.com/users/jiexiong/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":7,"created_at":"2016-11-01T22:57:31Z","updated_at":"2016-11-02T01:44:45Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/15722","html_url":"https://github.com/apache/spark/pull/15722","diff_url":"https://github.com/apache/spark/pull/15722.diff","patch_url":"https://github.com/apache/spark/pull/15722.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nFixed the OOM problem in BytesToBytesMap.\r\n\r\n## How was this patch tested?\r\n\r\n build the package and submit a query to spark-shell with the new package.\r\n\r\n./bin/fb-spark-shell.sh -q ad_metrics -t -- --conf spark.dynamicAllocation.maxEx\r\necutors=500 --conf spark.sql.shuffle.partitions=4000\r\n\r\nHistory server UI location is https://fburl.com/484871973\r\nScuba dataset link for task metrics is https://fburl.com/484871984\r\nScuba dataset link for resource metrics is https://fburl.com/484871989\r\nScuba data link for profiling is https://fburl.com/484871993\r\nResource Manager Session link is https://our.intern.facebook.com/intern/bumblebee_proxy/?proxy_url=http://hadoop2016.atn1.facebook.com:8810/resourceManager/session.html?sessionId=126507000000179\r\nSpark context Web UI available at http://[2401:db00:11:d0af:face:0:19:0]:8087\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/257749827","html_url":"https://github.com/apache/spark/pull/15722#issuecomment-257749827","issue_url":"https://api.github.com/repos/apache/spark/issues/15722","id":257749827,"user":{"login":"rxin","id":323388,"avatar_url":"https://avatars.githubusercontent.com/u/323388?v=3","gravatar_id":"","url":"https://api.github.com/users/rxin","html_url":"https://github.com/rxin","followers_url":"https://api.github.com/users/rxin/followers","following_url":"https://api.github.com/users/rxin/following{/other_user}","gists_url":"https://api.github.com/users/rxin/gists{/gist_id}","starred_url":"https://api.github.com/users/rxin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rxin/subscriptions","organizations_url":"https://api.github.com/users/rxin/orgs","repos_url":"https://api.github.com/users/rxin/repos","events_url":"https://api.github.com/users/rxin/events{/privacy}","received_events_url":"https://api.github.com/users/rxin/received_events","type":"User","site_admin":false},"created_at":"2016-11-02T01:44:44Z","updated_at":"2016-11-02T01:44:44Z","body":"@jiexiong can you update the pr description to include more information about the OOM and remove the facebook internal links? Thanks.\r\n"}}
{"actor":{"display_login":"rxin"}}