{"action":"created","issue":{"url":"https://api.github.com/repos/pydata/pandas/issues/11717","labels_url":"https://api.github.com/repos/pydata/pandas/issues/11717/labels{/name}","comments_url":"https://api.github.com/repos/pydata/pandas/issues/11717/comments","events_url":"https://api.github.com/repos/pydata/pandas/issues/11717/events","html_url":"https://github.com/pydata/pandas/pull/11717","id":119309647,"number":11717,"title":"BUG in .drop and .groupby for dataframes with multi-indexed columns","user":{"login":"nbonnotte","id":9929982,"avatar_url":"https://avatars.githubusercontent.com/u/9929982?v=3","gravatar_id":"","url":"https://api.github.com/users/nbonnotte","html_url":"https://github.com/nbonnotte","followers_url":"https://api.github.com/users/nbonnotte/followers","following_url":"https://api.github.com/users/nbonnotte/following{/other_user}","gists_url":"https://api.github.com/users/nbonnotte/gists{/gist_id}","starred_url":"https://api.github.com/users/nbonnotte/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nbonnotte/subscriptions","organizations_url":"https://api.github.com/users/nbonnotte/orgs","repos_url":"https://api.github.com/users/nbonnotte/repos","events_url":"https://api.github.com/users/nbonnotte/events{/privacy}","received_events_url":"https://api.github.com/users/nbonnotte/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/pydata/pandas/labels/Error%20Reporting","name":"Error Reporting","color":"ffa0ff"},{"url":"https://api.github.com/repos/pydata/pandas/labels/Groupby","name":"Groupby","color":"729FCF"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2015-11-28T17:46:30Z","updated_at":"2016-01-17T19:09:11Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/pydata/pandas/pulls/11717","html_url":"https://github.com/pydata/pandas/pull/11717","diff_url":"https://github.com/pydata/pandas/pull/11717.diff","patch_url":"https://github.com/pydata/pandas/pull/11717.patch"},"body":"Fix issue #11640 [edit: and #11741, xref PR #12063]\r\n\r\n## Bug 1\r\n\r\nThere was a bug deeply hidden in `pandas/core/index.py`, where it was assumed that the `.get_loc` method of an index would only return an integer or a slice, while it can also return a mask.\r\n\r\nSimply correcting that bug has two consequences:\r\n\r\n- now for a dataframe `df` with multi-indexed columns, `df.drop('a', axis=1)` works if `df['a']` returns something, e.g. if the column `('a', '')` exists\r\n- likewise, `df.groupby('a')` also works in that case\r\n\r\nExample:\r\n\r\n```\r\nIn [2]: df = pd.DataFrame(columns=['a','b','c','d'], data=[[1,'b1','c1',3], [1,'b2','c2',4]])\r\n\r\nIn [3]: dg = df.pivot_table(index='a', columns=['b','c'], values='d').reset_index()\r\n\r\nIn [4]: dg\r\nOut[4]: \r\nb  a b1 b2\r\nc    c1 c2\r\n0  1  3  4\r\n\r\nIn [5]: dg['a']\r\nOut[5]: \r\n0    1\r\nName: a, dtype: int64\r\n\r\nIn [8]: dg.drop('a', axis=1)\r\nOut[8]: \r\nb b1 b2\r\nc c1 c2\r\n0  3  4\r\n\r\nIn [9]: dg.groupby('a').mean()\r\nOut[9]: \r\nb b1 b2\r\nc c1 c2\r\na      \r\n1  3  4\r\n```\r\n\r\n## Bug 2 [edit: #11741, solved by PR #12063]\r\n\r\nI also correct a little bug I mentioned in the talk on issue #11640: `.groupby` failed to raise a `KeyError` for a non-existing column when there was only one row:\r\n```\r\nIn [6]: dg.groupby('z').mean()\r\nOut[6]: \r\nb  a b1 b2\r\nc    c1 c2\r\nz  1  3  4\r\n```\r\nNow we do have:\r\n```\r\nIn [4]: dg.groupby('z').mean()\r\n---------------------------------------------------------------------------\r\nKeyError                                  Traceback (most recent call last)\r\n...\r\nKeyError: 'z'\r\n```"},"comment":{"url":"https://api.github.com/repos/pydata/pandas/issues/comments/172367196","html_url":"https://github.com/pydata/pandas/pull/11717#issuecomment-172367196","issue_url":"https://api.github.com/repos/pydata/pandas/issues/11717","id":172367196,"user":{"login":"jreback","id":953992,"avatar_url":"https://avatars.githubusercontent.com/u/953992?v=3","gravatar_id":"","url":"https://api.github.com/users/jreback","html_url":"https://github.com/jreback","followers_url":"https://api.github.com/users/jreback/followers","following_url":"https://api.github.com/users/jreback/following{/other_user}","gists_url":"https://api.github.com/users/jreback/gists{/gist_id}","starred_url":"https://api.github.com/users/jreback/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jreback/subscriptions","organizations_url":"https://api.github.com/users/jreback/orgs","repos_url":"https://api.github.com/users/jreback/repos","events_url":"https://api.github.com/users/jreback/events{/privacy}","received_events_url":"https://api.github.com/users/jreback/received_events","type":"User","site_admin":false},"created_at":"2016-01-17T19:09:11Z","updated_at":"2016-01-17T19:09:11Z","body":"```\r\nIn [36]: df = pd.DataFrame(columns=['a','b','c','d'], data=[[1,'b1','c1',3], [1,'b2','c2',4]])\r\n\r\nIn [37]: \r\n\r\nIn [37]: In [3]: df = df.pivot_table(index='a', columns=['b','c'], values='d').reset_index()\r\n\r\nIn [39]: df.groupby('a').mean()\r\n---------------------------------------------------------------------------\r\nAttributeError                            Traceback (most recent call last)\r\n<ipython-input-39-a830c6135818> in <module>()\r\n----> 1 df.groupby('a').mean()\r\n\r\n/Users/jreback/pandas/pandas/core/groupby.pyc in mean(self)\r\n    908             self._set_selection_from_grouper()\r\n    909             f = lambda x: x.mean(axis=self.axis)\r\n--> 910             return self._python_agg_general(f)\r\n    911 \r\n    912     @Substitution(name='groupby')\r\n\r\n/Users/jreback/pandas/pandas/core/groupby.pyc in _python_agg_general(self, func, *args, **kwargs)\r\n    751                 output[name] = self._try_cast(values[mask], result)\r\n    752 \r\n--> 753         return self._wrap_aggregated_output(output)\r\n    754 \r\n    755     def _wrap_applied_output(self, *args, **kwargs):\r\n\r\n/Users/jreback/pandas/pandas/core/groupby.pyc in _wrap_aggregated_output(self, output, names)\r\n   3511     def _wrap_aggregated_output(self, output, names=None):\r\n   3512         agg_axis = 0 if self.axis == 1 else 1\r\n-> 3513         agg_labels = self._obj_with_exclusions._get_axis(agg_axis)\r\n   3514 \r\n   3515         output_keys = self._decide_output_index(output, agg_labels)\r\n\r\n/Users/jreback/pandas/pandas/core/groupby.pyc in __getattr__(self, attr)\r\n    457 \r\n    458         raise AttributeError(\"%r object has no attribute %r\" %\r\n--> 459                              (type(self).__name__, attr))\r\n    460 \r\n    461     plot = property(GroupByPlot)\r\n\r\nAttributeError: 'DataFrameGroupBy' object has no attribute '_obj_with_exclusions'\r\n\r\nIn [40]: df.groupby('a',level=0).mean()\r\nOut[40]: \r\nb  a b1 b2\r\nc    c1 c2\r\n0  1  3  4\r\n```\r\n\r\nfrom your original issue ``[39]`` should simply give a more informative errors message (and raise a ``KeyError``), not actually try to fix the issue when the level is not specified, which would be a major API change."}}