{"action":"created","issue":{"url":"https://api.github.com/repos/dask/dask/issues/3865","repository_url":"https://api.github.com/repos/dask/dask","labels_url":"https://api.github.com/repos/dask/dask/issues/3865/labels{/name}","comments_url":"https://api.github.com/repos/dask/dask/issues/3865/comments","events_url":"https://api.github.com/repos/dask/dask/issues/3865/events","html_url":"https://github.com/dask/dask/issues/3865","id":349096242,"node_id":"MDU6SXNzdWUzNDkwOTYyNDI=","number":3865,"title":"NotImplementedError; struct<...> while reading from parquet using pyarrow engine (works with pandas)","user":{"login":"RafaelIL","id":42240400,"node_id":"MDQ6VXNlcjQyMjQwNDAw","avatar_url":"https://avatars0.githubusercontent.com/u/42240400?v=4","gravatar_id":"","url":"https://api.github.com/users/RafaelIL","html_url":"https://github.com/RafaelIL","followers_url":"https://api.github.com/users/RafaelIL/followers","following_url":"https://api.github.com/users/RafaelIL/following{/other_user}","gists_url":"https://api.github.com/users/RafaelIL/gists{/gist_id}","starred_url":"https://api.github.com/users/RafaelIL/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/RafaelIL/subscriptions","organizations_url":"https://api.github.com/users/RafaelIL/orgs","repos_url":"https://api.github.com/users/RafaelIL/repos","events_url":"https://api.github.com/users/RafaelIL/events{/privacy}","received_events_url":"https://api.github.com/users/RafaelIL/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":7,"created_at":"2018-08-09T11:57:24Z","updated_at":"2018-08-09T13:53:52Z","closed_at":"2018-08-09T13:53:52Z","author_association":"NONE","body":"  File \"/home/rafael.moraes/.virtualenvs/normalization_backend/lib/python3.6/site-packages/dask/dataframe/io/parquet.py\", line 995, in read_parquet\r\n    categories=categories, index=index, infer_divisions=infer_divisions)\r\n  File \"/home/rafael.moraes/.virtualenvs/normalization_backend/lib/python3.6/site-packages/dask/dataframe/io/parquet.py\", line 579, in _read_pyarrow\r\n    dtypes = _get_pyarrow_dtypes(schema, categories)\r\n  File \"/home/rafael.moraes/.virtualenvs/normalization_backend/lib/python3.6/site-packages/dask/dataframe/io/utils.py\", line 26, in _get_pyarrow_dtypes\r\n    numpy_dtype = field.type.to_pandas_dtype()\r\n  File \"pyarrow/types.pxi\", line 176, in pyarrow.lib.DataType.to_pandas_dtype\r\nNotImplementedError: struct<country: string, admin_area: string, sub_admin_area: string...>\r\n\r\nI got this error while reading a parquet file using dd.read_parquet  with engine pyarrow, even if i remove the specific column with this struct type using (columns=) it still throw this error, if i switch to pandas it works fine, seems a dask related issue only.\r\n"},"comment":{"url":"https://api.github.com/repos/dask/dask/issues/comments/411765519","html_url":"https://github.com/dask/dask/issues/3865#issuecomment-411765519","issue_url":"https://api.github.com/repos/dask/dask/issues/3865","id":411765519,"node_id":"MDEyOklzc3VlQ29tbWVudDQxMTc2NTUxOQ==","user":{"login":"mrocklin","id":306380,"node_id":"MDQ6VXNlcjMwNjM4MA==","avatar_url":"https://avatars3.githubusercontent.com/u/306380?v=4","gravatar_id":"","url":"https://api.github.com/users/mrocklin","html_url":"https://github.com/mrocklin","followers_url":"https://api.github.com/users/mrocklin/followers","following_url":"https://api.github.com/users/mrocklin/following{/other_user}","gists_url":"https://api.github.com/users/mrocklin/gists{/gist_id}","starred_url":"https://api.github.com/users/mrocklin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mrocklin/subscriptions","organizations_url":"https://api.github.com/users/mrocklin/orgs","repos_url":"https://api.github.com/users/mrocklin/repos","events_url":"https://api.github.com/users/mrocklin/events{/privacy}","received_events_url":"https://api.github.com/users/mrocklin/received_events","type":"User","site_admin":false},"created_at":"2018-08-09T13:53:52Z","updated_at":"2018-08-09T13:53:52Z","author_association":"MEMBER","body":"This is an issue for upstream at PyArrow.  Closing"}}
{"actor":{"display_login":"mrocklin"}}