{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/12740","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/12740/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/12740/comments","events_url":"https://api.github.com/repos/apache/spark/issues/12740/events","html_url":"https://github.com/apache/spark/pull/12740","id":151466527,"number":12740,"title":"[SPARK-14961] Build HashedRelation larger than 1G","user":{"login":"davies","id":40902,"avatar_url":"https://avatars.githubusercontent.com/u/40902?v=3","gravatar_id":"","url":"https://api.github.com/users/davies","html_url":"https://github.com/davies","followers_url":"https://api.github.com/users/davies/followers","following_url":"https://api.github.com/users/davies/following{/other_user}","gists_url":"https://api.github.com/users/davies/gists{/gist_id}","starred_url":"https://api.github.com/users/davies/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/davies/subscriptions","organizations_url":"https://api.github.com/users/davies/orgs","repos_url":"https://api.github.com/users/davies/repos","events_url":"https://api.github.com/users/davies/events{/privacy}","received_events_url":"https://api.github.com/users/davies/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2016-04-27T19:07:37Z","updated_at":"2016-04-28T02:15:13Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/12740","html_url":"https://github.com/apache/spark/pull/12740","diff_url":"https://github.com/apache/spark/pull/12740.diff","patch_url":"https://github.com/apache/spark/pull/12740.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nCurrently, LongToUnsafeRowMap use byte array as the underlying page, which can't be larger 1G.\r\n\r\nThis PR improves LongToUnsafeRowMap  to scale up to 8G bytes by using array of Long instead of array of byte. \r\n\r\n## How was this patch tested?\r\n\r\nManually ran a test to confirm that both UnsafeHashedRelation and LongHashedRelation could build a map that larger than 2G.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/215288385","html_url":"https://github.com/apache/spark/pull/12740#issuecomment-215288385","issue_url":"https://api.github.com/repos/apache/spark/issues/12740","id":215288385,"user":{"login":"sameeragarwal","id":166454,"avatar_url":"https://avatars.githubusercontent.com/u/166454?v=3","gravatar_id":"","url":"https://api.github.com/users/sameeragarwal","html_url":"https://github.com/sameeragarwal","followers_url":"https://api.github.com/users/sameeragarwal/followers","following_url":"https://api.github.com/users/sameeragarwal/following{/other_user}","gists_url":"https://api.github.com/users/sameeragarwal/gists{/gist_id}","starred_url":"https://api.github.com/users/sameeragarwal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sameeragarwal/subscriptions","organizations_url":"https://api.github.com/users/sameeragarwal/orgs","repos_url":"https://api.github.com/users/sameeragarwal/repos","events_url":"https://api.github.com/users/sameeragarwal/events{/privacy}","received_events_url":"https://api.github.com/users/sameeragarwal/received_events","type":"User","site_admin":false},"created_at":"2016-04-28T02:15:13Z","updated_at":"2016-04-28T02:15:13Z","body":"Looks great!"}}