{"action":"created","issue":{"url":"https://api.github.com/repos/pandas-dev/pandas/issues/24387","repository_url":"https://api.github.com/repos/pandas-dev/pandas","labels_url":"https://api.github.com/repos/pandas-dev/pandas/issues/24387/labels{/name}","comments_url":"https://api.github.com/repos/pandas-dev/pandas/issues/24387/comments","events_url":"https://api.github.com/repos/pandas-dev/pandas/issues/24387/events","html_url":"https://github.com/pandas-dev/pandas/pull/24387","id":393564360,"node_id":"MDExOlB1bGxSZXF1ZXN0MjQwNTU2OTMx","number":24387,"title":"PERF: DataFrame dict constructor with columns","user":{"login":"TomAugspurger","id":1312546,"node_id":"MDQ6VXNlcjEzMTI1NDY=","avatar_url":"https://avatars3.githubusercontent.com/u/1312546?v=4","gravatar_id":"","url":"https://api.github.com/users/TomAugspurger","html_url":"https://github.com/TomAugspurger","followers_url":"https://api.github.com/users/TomAugspurger/followers","following_url":"https://api.github.com/users/TomAugspurger/following{/other_user}","gists_url":"https://api.github.com/users/TomAugspurger/gists{/gist_id}","starred_url":"https://api.github.com/users/TomAugspurger/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TomAugspurger/subscriptions","organizations_url":"https://api.github.com/users/TomAugspurger/orgs","repos_url":"https://api.github.com/users/TomAugspurger/repos","events_url":"https://api.github.com/users/TomAugspurger/events{/privacy}","received_events_url":"https://api.github.com/users/TomAugspurger/received_events","type":"User","site_admin":false},"labels":[{"id":31404521,"node_id":"MDU6TGFiZWwzMTQwNDUyMQ==","url":"https://api.github.com/repos/pandas-dev/pandas/labels/Dtypes","name":"Dtypes","color":"e102d8","default":false},{"id":8935311,"node_id":"MDU6TGFiZWw4OTM1MzEx","url":"https://api.github.com/repos/pandas-dev/pandas/labels/Performance","name":"Performance","color":"a10c02","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":7,"created_at":"2018-12-21T19:19:15Z","updated_at":"2018-12-21T20:39:21Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/pandas-dev/pandas/pulls/24387","html_url":"https://github.com/pandas-dev/pandas/pull/24387","diff_url":"https://github.com/pandas-dev/pandas/pull/24387.diff","patch_url":"https://github.com/pandas-dev/pandas/pull/24387.patch"},"body":"When passing a dict and `column=` to DataFrame, we previously\r\npassed the dict of {column: array} to the Series constructor. This\r\neventually hit `construct_1d_object_array_from_listlike`[1]. For\r\nextension arrays, this ends up calling `ExtensionArray.__iter__`,\r\niterating over the elements of the ExtensionArray, which is\r\nprohibiatively slow.\r\n\r\n---\r\n\r\n```python\r\nimport pandas as pd\r\nimport numpy as np\r\n\r\na = pd.Series(np.arange(1000))\r\nd = {i: a for i in range(30)}\r\n\r\n%timeit df = pd.DataFrame(d, columns=list(range(len(d))))\r\n```\r\n\r\nbefore\r\n\r\n```\r\n4.06 ms ± 53.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\r\n```\r\n\r\nafter\r\n\r\n```\r\n2.54 ms ± 113 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\r\n```\r\n\r\nWith Series with sparse values instead, the problem is exacerbated (note\r\nthe smaller and fewer series).\r\n\r\n```python\r\na = pd.Series(np.arange(1000), dtype=\"Sparse[int]\")\r\nd = {i: a for i in range(50)}\r\n\r\n%timeit df = pd.DataFrame(d, columns=list(range(len(d))))\r\n```\r\n\r\nBefore\r\n\r\n```\r\n213 ms ± 7.64 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\r\n```\r\n\r\nafter\r\n\r\n```\r\n4.41 ms ± 134 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\r\n```\r\n\r\n---\r\n\r\nWe try to properly handle all the edge cases that we were papering over\r\nearlier by just passing the `data` to Series.\r\n\r\nWe fix a bug or two along the way, but don't change any *tested*\r\nbehavior, even if it looks fishy (e.g. #24385).\r\n\r\n[1]: https://github.com/pandas-dev/pandas/issues/24368#issuecomment-449037989\r\n\r\nCloses https://github.com/pandas-dev/pandas/issues/24368\r\nCloses https://github.com/pandas-dev/pandas/issues/24386\r\n"},"comment":{"url":"https://api.github.com/repos/pandas-dev/pandas/issues/comments/449492874","html_url":"https://github.com/pandas-dev/pandas/pull/24387#issuecomment-449492874","issue_url":"https://api.github.com/repos/pandas-dev/pandas/issues/24387","id":449492874,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0OTQ5Mjg3NA==","user":{"login":"TomAugspurger","id":1312546,"node_id":"MDQ6VXNlcjEzMTI1NDY=","avatar_url":"https://avatars3.githubusercontent.com/u/1312546?v=4","gravatar_id":"","url":"https://api.github.com/users/TomAugspurger","html_url":"https://github.com/TomAugspurger","followers_url":"https://api.github.com/users/TomAugspurger/followers","following_url":"https://api.github.com/users/TomAugspurger/following{/other_user}","gists_url":"https://api.github.com/users/TomAugspurger/gists{/gist_id}","starred_url":"https://api.github.com/users/TomAugspurger/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TomAugspurger/subscriptions","organizations_url":"https://api.github.com/users/TomAugspurger/orgs","repos_url":"https://api.github.com/users/TomAugspurger/repos","events_url":"https://api.github.com/users/TomAugspurger/events{/privacy}","received_events_url":"https://api.github.com/users/TomAugspurger/received_events","type":"User","site_admin":false},"created_at":"2018-12-21T20:39:21Z","updated_at":"2018-12-21T20:39:21Z","author_association":"CONTRIBUTOR","body":"With the latest all of pandas/tests/frame is passing..."}}
{"actor":{"display_login":"TomAugspurger"}}