{"action":"created","issue":{"url":"https://api.github.com/repos/modin-project/modin/issues/205","repository_url":"https://api.github.com/repos/modin-project/modin","labels_url":"https://api.github.com/repos/modin-project/modin/issues/205/labels{/name}","comments_url":"https://api.github.com/repos/modin-project/modin/issues/205/comments","events_url":"https://api.github.com/repos/modin-project/modin/issues/205/events","html_url":"https://github.com/modin-project/modin/issues/205","id":373973998,"node_id":"MDU6SXNzdWUzNzM5NzM5OTg=","number":205,"title":"Improve speed of read_parquet","user":{"login":"devin-petersohn","id":10732128,"node_id":"MDQ6VXNlcjEwNzMyMTI4","avatar_url":"https://avatars0.githubusercontent.com/u/10732128?v=4","gravatar_id":"","url":"https://api.github.com/users/devin-petersohn","html_url":"https://github.com/devin-petersohn","followers_url":"https://api.github.com/users/devin-petersohn/followers","following_url":"https://api.github.com/users/devin-petersohn/following{/other_user}","gists_url":"https://api.github.com/users/devin-petersohn/gists{/gist_id}","starred_url":"https://api.github.com/users/devin-petersohn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/devin-petersohn/subscriptions","organizations_url":"https://api.github.com/users/devin-petersohn/orgs","repos_url":"https://api.github.com/users/devin-petersohn/repos","events_url":"https://api.github.com/users/devin-petersohn/events{/privacy}","received_events_url":"https://api.github.com/users/devin-petersohn/received_events","type":"User","site_admin":false},"labels":[{"id":985295309,"node_id":"MDU6TGFiZWw5ODUyOTUzMDk=","url":"https://api.github.com/repos/modin-project/modin/labels/Performance%20%F0%9F%9A%80","name":"Performance 🚀","color":"d93f0b","default":false},{"id":971728240,"node_id":"MDU6TGFiZWw5NzE3MjgyNDA=","url":"https://api.github.com/repos/modin-project/modin/labels/good%20first%20issue%20:beginner:","name":"good first issue :beginner:","color":"7057ff","default":false}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-10-25T14:23:44Z","updated_at":"2018-11-07T22:32:05Z","closed_at":"2018-11-07T22:32:05Z","author_association":"CONTRIBUTOR","body":"\r\n### Describe the problem\r\n<!-- Describe the problem clearly here. -->\r\n\r\nCurrently `modin.pandas.io.read_parquet` uses Ray to read each column individually in-parallel. It would be more efficient to choose the number of partitions based on the number of cores and read multiple columns at once. It may not immediately improve read times, but it would improve the speed of operations after.\r\n\r\nCurrently io.py needs a minor refactor (#203).\r\n\r\nThis issue would be good for a first time contributor familiar with Parquet. Feel free to reply to this issue with any questions.\r\n\r\n### Source code / logs\r\n\r\n[Link to source](https://github.com/modin-project/modin/blob/master/modin/pandas/io.py#L30)\r\n[Link to Ray Remote Task](https://github.com/modin-project/modin/blob/master/modin/pandas/io.py#L613)\r\n"},"comment":{"url":"https://api.github.com/repos/modin-project/modin/issues/comments/436802227","html_url":"https://github.com/modin-project/modin/issues/205#issuecomment-436802227","issue_url":"https://api.github.com/repos/modin-project/modin/issues/205","id":436802227,"node_id":"MDEyOklzc3VlQ29tbWVudDQzNjgwMjIyNw==","user":{"login":"devin-petersohn","id":10732128,"node_id":"MDQ6VXNlcjEwNzMyMTI4","avatar_url":"https://avatars0.githubusercontent.com/u/10732128?v=4","gravatar_id":"","url":"https://api.github.com/users/devin-petersohn","html_url":"https://github.com/devin-petersohn","followers_url":"https://api.github.com/users/devin-petersohn/followers","following_url":"https://api.github.com/users/devin-petersohn/following{/other_user}","gists_url":"https://api.github.com/users/devin-petersohn/gists{/gist_id}","starred_url":"https://api.github.com/users/devin-petersohn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/devin-petersohn/subscriptions","organizations_url":"https://api.github.com/users/devin-petersohn/orgs","repos_url":"https://api.github.com/users/devin-petersohn/repos","events_url":"https://api.github.com/users/devin-petersohn/events{/privacy}","received_events_url":"https://api.github.com/users/devin-petersohn/received_events","type":"User","site_admin":false},"created_at":"2018-11-07T22:32:05Z","updated_at":"2018-11-07T22:32:05Z","author_association":"CONTRIBUTOR","body":"Your fix looks great! I think we can close this issue. Thanks again!"}}
{"actor":{"display_login":"devin-petersohn"}}