{"action":"created","issue":{"url":"https://api.github.com/repos/dask/distributed/issues/240","repository_url":"https://api.github.com/repos/dask/distributed","labels_url":"https://api.github.com/repos/dask/distributed/issues/240/labels{/name}","comments_url":"https://api.github.com/repos/dask/distributed/issues/240/comments","events_url":"https://api.github.com/repos/dask/distributed/issues/240/events","html_url":"https://github.com/dask/distributed/issues/240","id":149402047,"number":240,"title":"KeyError while computing on some columns","user":{"login":"frol","id":304265,"avatar_url":"https://avatars.githubusercontent.com/u/304265?v=3","gravatar_id":"","url":"https://api.github.com/users/frol","html_url":"https://github.com/frol","followers_url":"https://api.github.com/users/frol/followers","following_url":"https://api.github.com/users/frol/following{/other_user}","gists_url":"https://api.github.com/users/frol/gists{/gist_id}","starred_url":"https://api.github.com/users/frol/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/frol/subscriptions","organizations_url":"https://api.github.com/users/frol/orgs","repos_url":"https://api.github.com/users/frol/repos","events_url":"https://api.github.com/users/frol/events{/privacy}","received_events_url":"https://api.github.com/users/frol/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2016-04-19T09:26:54Z","updated_at":"2016-04-19T15:40:33Z","closed_at":"2016-04-19T15:40:33Z","body":"I have a dataset with 15004 columns and 50k rows in a CSV format on HDFS. The first 14k columns are integers and floats, the last 1k columns are text (though Pandas throws a warning `Columns ... have mixed types. Specify dtype option on import or set low_memory=False.`, neverthelss, Dask loads those columns as Python objects dtype [`dtype('o')`]).\r\n\r\nI'm facing the following traceback when I compute anything on a 8331+th columns:\r\n\r\n```Python\r\n>>> dataset.Var8331.count().compute()\r\n\r\n/opt/conda/lib/python3.5/site-packages/dask/base.py in compute(self, **kwargs)\r\n     35 \r\n     36     def compute(self, **kwargs):\r\n---> 37         return compute(self, **kwargs)[0]\r\n     38 \r\n     39     @classmethod\r\n\r\n/opt/conda/lib/python3.5/site-packages/dask/base.py in compute(*args, **kwargs)\r\n    108                 for opt, val in groups.items()])\r\n    109     keys = [var._keys() for var in variables]\r\n--> 110     results = get(dsk, keys, **kwargs)\r\n    111 \r\n    112     results_iter = iter(results)\r\n\r\n/opt/conda/lib/python3.5/site-packages/distributed/executor.py in get(self, dsk, keys, **kwargs)\r\n    986 \r\n    987         if status == 'error':\r\n--> 988             raise result\r\n    989         else:\r\n    990             return result\r\n\r\n/opt/conda/lib/python3.5/site-packages/distributed/executor.py in _get(self, dsk, keys, restrictions, raise_on_error)\r\n    954         else:\r\n    955             try:\r\n--> 956                 result = yield self._gather(packed)\r\n    957                 result = 'OK', result\r\n    958             except Exception as e:\r\n\r\n/opt/conda/lib/python3.5/site-packages/tornado/gen.py in run(self)\r\n   1006 \r\n   1007                     try:\r\n-> 1008                         value = future.result()\r\n   1009                     except Exception:\r\n   1010                         self.had_exception = True\r\n\r\n/opt/conda/lib/python3.5/site-packages/tornado/concurrent.py in result(self, timeout)\r\n    230             return self._result\r\n    231         if self._exc_info is not None:\r\n--> 232             raise_exc_info(self._exc_info)\r\n    233         self._check_done()\r\n    234         return self._result\r\n\r\n/opt/conda/lib/python3.5/site-packages/tornado/util.py in raise_exc_info(exc_info)\r\n\r\n/opt/conda/lib/python3.5/site-packages/tornado/gen.py in run(self)\r\n   1015                         exc_info = None\r\n   1016                     else:\r\n-> 1017                         yielded = self.gen.send(value)\r\n   1018 \r\n   1019                     if stack_context._state.contexts is not orig_stack_contexts:\r\n\r\n/opt/conda/lib/python3.5/site-packages/distributed/executor.py in _gather(self, futures, errors)\r\n    659                         six.reraise(type(d['exception']),\r\n    660                                     d['exception'],\r\n--> 661                                     d['traceback'])\r\n    662                     if errors == 'skip':\r\n    663                         bad_keys.add(key)\r\n\r\n/opt/conda/lib/python3.5/site-packages/six.py in reraise(tp, value, tb)\r\n    683             value = tp()\r\n    684         if value.__traceback__ is not tb:\r\n--> 685             raise value.with_traceback(tb)\r\n    686         raise value\r\n    687 \r\n\r\n/opt/conda/lib/python3.5/site-packages/pandas/core/frame.py in __getitem__()\r\n   1990             return self._getitem_multilevel(key)\r\n   1991         else:\r\n-> 1992             return self._getitem_column(key)\r\n   1993 \r\n   1994     def _getitem_column(self, key):\r\n\r\n/opt/conda/lib/python3.5/site-packages/pandas/core/frame.py in _getitem_column()\r\n   1997         # get column\r\n   1998         if self.columns.is_unique:\r\n-> 1999             return self._get_item_cache(key)\r\n   2000 \r\n   2001         # duplicate columns & possible reduce dimensionality\r\n\r\n/opt/conda/lib/python3.5/site-packages/pandas/core/generic.py in _get_item_cache()\r\n   1343         res = cache.get(item)\r\n   1344         if res is None:\r\n-> 1345             values = self._data.get(item)\r\n   1346             res = self._box_item_values(item, values)\r\n   1347             cache[item] = res\r\n\r\n/opt/conda/lib/python3.5/site-packages/pandas/core/internals.py in get()\r\n   3223 \r\n   3224             if not isnull(item):\r\n-> 3225                 loc = self.items.get_loc(item)\r\n   3226             else:\r\n   3227                 indexer = np.arange(len(self.items))[isnull(self.items)]\r\n\r\n/opt/conda/lib/python3.5/site-packages/pandas/indexes/base.py in get_loc()\r\n   1876                 return self._engine.get_loc(key)\r\n   1877             except KeyError:\r\n-> 1878                 return self._engine.get_loc(self._maybe_cast_indexer(key))\r\n   1879 \r\n   1880         indexer = self.get_indexer([key], method=method, tolerance=tolerance)\r\n\r\npandas/index.pyx in pandas.index.IndexEngine.get_loc (pandas/index.c:4027)()\r\n\r\npandas/index.pyx in pandas.index.IndexEngine.get_loc (pandas/index.c:3891)()\r\n\r\npandas/hashtable.pyx in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:12408)()\r\n\r\npandas/hashtable.pyx in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:12359)()\r\n\r\nKeyError: 'Var8331'\r\n```\r\n\r\nI have tried to reproduce this inside Dask without Distributed and HDFS, but everything seems to work fine. I have also tried to minimize the failing example, recreating CSV file over and over, so no Dask syntax is involved, but it doesn't get very small (70MB is the best I could get). Weird thing is that the issue I can reproduce the issue with exactly 1897 first rows, however, once I move the range from the first row to the second or deselect at least one column, the issue goes away,  the same story about trying to catch a wrong column (it works fine with the first 14340 columns, and fails if I change a number of records).\r\n\r\nIt looks like a memory overflow to me, but I have no idea where to start.\r\n\r\nDo you have any clue?\r\n\r\nMy environment:\r\n\r\n* Dask 0.8.2\r\n* Distributed 1.9.2 (BTW, how long does it usually take the latest version, which is 1.9.3 at the moment, reach conda channels?)\r\n* Alpine Linux 3.3.3 in a Docker container with glibc 2.23"},"comment":{"url":"https://api.github.com/repos/dask/distributed/issues/comments/211984881","html_url":"https://github.com/dask/distributed/issues/240#issuecomment-211984881","issue_url":"https://api.github.com/repos/dask/distributed/issues/240","id":211984881,"user":{"login":"mrocklin","id":306380,"avatar_url":"https://avatars.githubusercontent.com/u/306380?v=3","gravatar_id":"","url":"https://api.github.com/users/mrocklin","html_url":"https://github.com/mrocklin","followers_url":"https://api.github.com/users/mrocklin/followers","following_url":"https://api.github.com/users/mrocklin/following{/other_user}","gists_url":"https://api.github.com/users/mrocklin/gists{/gist_id}","starred_url":"https://api.github.com/users/mrocklin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mrocklin/subscriptions","organizations_url":"https://api.github.com/users/mrocklin/orgs","repos_url":"https://api.github.com/users/mrocklin/repos","events_url":"https://api.github.com/users/mrocklin/events{/privacy}","received_events_url":"https://api.github.com/users/mrocklin/received_events","type":"User","site_admin":false},"created_at":"2016-04-19T15:40:33Z","updated_at":"2016-04-19T15:40:33Z","body":"Closing for now.  Please reopen if the problem persists."}}