{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/11308","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/11308/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/11308/comments","events_url":"https://api.github.com/repos/apache/spark/issues/11308/events","html_url":"https://github.com/apache/spark/pull/11308","id":135528729,"number":11308,"title":"[SPARK-12546][SQL] Change default number of open parquet files","user":{"login":"marmbrus","id":527,"avatar_url":"https://avatars.githubusercontent.com/u/527?v=3","gravatar_id":"","url":"https://api.github.com/users/marmbrus","html_url":"https://github.com/marmbrus","followers_url":"https://api.github.com/users/marmbrus/followers","following_url":"https://api.github.com/users/marmbrus/following{/other_user}","gists_url":"https://api.github.com/users/marmbrus/gists{/gist_id}","starred_url":"https://api.github.com/users/marmbrus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/marmbrus/subscriptions","organizations_url":"https://api.github.com/users/marmbrus/orgs","repos_url":"https://api.github.com/users/marmbrus/repos","events_url":"https://api.github.com/users/marmbrus/events{/privacy}","received_events_url":"https://api.github.com/users/marmbrus/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2016-02-22T20:04:46Z","updated_at":"2016-02-22T23:27:16Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/11308","html_url":"https://github.com/apache/spark/pull/11308","diff_url":"https://github.com/apache/spark/pull/11308.diff","patch_url":"https://github.com/apache/spark/pull/11308.patch"},"body":"A common problem that users encounter with Spark 1.6.0 is that writing to a partitioned parquet table OOMs.  The root cause is that parquet allocates a significant amount of memory that is not accounted for by our own mechanisms.  As a workaround, we can ensure that only a single file is open per task unless the user explicitly asks for more."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/187435553","html_url":"https://github.com/apache/spark/pull/11308#issuecomment-187435553","issue_url":"https://api.github.com/repos/apache/spark/issues/11308","id":187435553,"user":{"login":"marmbrus","id":527,"avatar_url":"https://avatars.githubusercontent.com/u/527?v=3","gravatar_id":"","url":"https://api.github.com/users/marmbrus","html_url":"https://github.com/marmbrus","followers_url":"https://api.github.com/users/marmbrus/followers","following_url":"https://api.github.com/users/marmbrus/following{/other_user}","gists_url":"https://api.github.com/users/marmbrus/gists{/gist_id}","starred_url":"https://api.github.com/users/marmbrus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/marmbrus/subscriptions","organizations_url":"https://api.github.com/users/marmbrus/orgs","repos_url":"https://api.github.com/users/marmbrus/repos","events_url":"https://api.github.com/users/marmbrus/events{/privacy}","received_events_url":"https://api.github.com/users/marmbrus/received_events","type":"User","site_admin":false},"created_at":"2016-02-22T23:27:16Z","updated_at":"2016-02-22T23:27:16Z","body":"Merging to master and 1.6"}}