{"action":"created","issue":{"url":"https://api.github.com/repos/pydata/pandas/issues/12564","repository_url":"https://api.github.com/repos/pydata/pandas","labels_url":"https://api.github.com/repos/pydata/pandas/issues/12564/labels{/name}","comments_url":"https://api.github.com/repos/pydata/pandas/issues/12564/comments","events_url":"https://api.github.com/repos/pydata/pandas/issues/12564/events","html_url":"https://github.com/pydata/pandas/issues/12564","id":139413351,"number":12564,"title":"Categorical equality check raises ValueError in DataFrame","user":{"login":"pganssle","id":1377457,"avatar_url":"https://avatars.githubusercontent.com/u/1377457?v=3","gravatar_id":"","url":"https://api.github.com/users/pganssle","html_url":"https://github.com/pganssle","followers_url":"https://api.github.com/users/pganssle/followers","following_url":"https://api.github.com/users/pganssle/following{/other_user}","gists_url":"https://api.github.com/users/pganssle/gists{/gist_id}","starred_url":"https://api.github.com/users/pganssle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pganssle/subscriptions","organizations_url":"https://api.github.com/users/pganssle/orgs","repos_url":"https://api.github.com/users/pganssle/repos","events_url":"https://api.github.com/users/pganssle/events{/privacy}","received_events_url":"https://api.github.com/users/pganssle/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/pydata/pandas/labels/Bug","name":"Bug","color":"e10c02"},{"url":"https://api.github.com/repos/pydata/pandas/labels/Categorical","name":"Categorical","color":"e11d21"},{"url":"https://api.github.com/repos/pydata/pandas/labels/Difficulty%20Novice","name":"Difficulty Novice","color":"fbca04"},{"url":"https://api.github.com/repos/pydata/pandas/labels/Effort%20Low","name":"Effort Low","color":"006b75"},{"url":"https://api.github.com/repos/pydata/pandas/labels/Indexing","name":"Indexing","color":"0b02e1"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/pydata/pandas/milestones/38","html_url":"https://github.com/pydata/pandas/milestones/0.18.1","labels_url":"https://api.github.com/repos/pydata/pandas/milestones/38/labels","id":1570594,"number":38,"title":"0.18.1","description":"after 0.18.0 of course!","creator":{"login":"jreback","id":953992,"avatar_url":"https://avatars.githubusercontent.com/u/953992?v=3","gravatar_id":"","url":"https://api.github.com/users/jreback","html_url":"https://github.com/jreback","followers_url":"https://api.github.com/users/jreback/followers","following_url":"https://api.github.com/users/jreback/following{/other_user}","gists_url":"https://api.github.com/users/jreback/gists{/gist_id}","starred_url":"https://api.github.com/users/jreback/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jreback/subscriptions","organizations_url":"https://api.github.com/users/jreback/orgs","repos_url":"https://api.github.com/users/jreback/repos","events_url":"https://api.github.com/users/jreback/events{/privacy}","received_events_url":"https://api.github.com/users/jreback/received_events","type":"User","site_admin":false},"open_issues":110,"closed_issues":5,"state":"open","created_at":"2016-02-08T15:29:59Z","updated_at":"2016-03-08T23:32:25Z","due_on":"2016-04-12T04:00:00Z","closed_at":null},"comments":2,"created_at":"2016-03-08T22:29:11Z","updated_at":"2016-03-09T00:32:51Z","closed_at":null,"body":"Apparently there's an issue when comparing the equality of a scalar value against a categorical column as part of a DataFrame. In the example below, I'm checking against `-np.inf`, but comparing to a string or integer gives the same results.\r\n\r\nThis raises `ValueError: Wrong number of dimensions`.\r\n\r\n#### Code Sample\r\n```python\r\nfrom sys import version\r\nimport pandas as pd     # Version 0.17.1 on Linux and Windows\r\nimport numpy as np\r\nprint(version)\r\nprint(pd.__version__)\r\n\r\n# Arbitrary data set\r\ncolumns = ['Name', 'Type', 'Age', 'Weight (kg)', 'Cuteness']\r\ndataset = [['Snuggles', 'Cat', 5.2, 4.2, 9.7],\r\n           ['Rex', 'Dog', 2.1, 12, 2.1],\r\n           ['Mrs. Quiggleworth', 'Cat', 7.4, 3, 7],\r\n           ['Squirmy', 'Snake', 1.1, 0.2, 0.1],\r\n           ['Tarantula', 'Legs', 0.2, 0.01, -np.inf],\r\n           ['Groucho', 'Dog', 6.9, 8, 5.1]]\r\n\r\ndf = pd.DataFrame(dataset, columns=columns).set_index(['Name'])\r\n\r\n# Works fine\r\nprint(\"String type - are any of the columns negative infinity?\")\r\nneg_inf = (df == -np.inf)\r\nprint(neg_inf.any(axis=1))\r\n\r\n# Convert 'type' to a categorical\r\ndf['Type'] = df['Type'].astype('category')\r\n\r\nprint(\"Categorical type - is the Type column negative infinity?\")\r\nprint(df['Type'] == -np.inf)    # Works fine\r\n\r\nprint(\"Categorical type in dataframe - are any of them negative infinity?\")\r\nprint(df[['Type']] == -np.inf)  # Danger, Will Robinson!\r\n```\r\n\r\n#### Expected Output\r\n```\r\n3.5.1 |Anaconda 2.5.0 (64-bit)| (default, Dec  7 2015, 11:16:01) \r\n[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]\r\n0.17.1\r\nString type - are any of the columns negative infinity?\r\nName\r\nSnuggles             False\r\nRex                  False\r\nMrs. Quiggleworth    False\r\nSquirmy              False\r\nTarantula             True\r\nGroucho              False\r\ndtype: bool\r\nCategorical type - is the Type column negative infinity?\r\nName\r\nSnuggles             False\r\nRex                  False\r\nMrs. Quiggleworth    False\r\nSquirmy              False\r\nTarantula            False\r\nGroucho              False\r\nName: Type, dtype: bool\r\nCategorical type in dataframe - are any of them negative infinity?\r\nTraceback (most recent call last):\r\n  File \"pandas_demo.py\", line 30, in <module>\r\n    print(df[['Type']] == -np.inf)  # Danger, Will Robinson!\r\n  File \"~/anaconda3/lib/python3.5/site-packages/pandas/core/ops.py\", line 1115, in f\r\n    res = self._combine_const(other, func, raise_on_error=False)\r\n  File \"~/anaconda3/lib/python3.5/site-packages/pandas/core/frame.py\", line 3482, in _combine_const\r\n    new_data = self._data.eval(func=func, other=other, raise_on_error=raise_on_error)\r\n  File \"~/anaconda3/lib/python3.5/site-packages/pandas/core/internals.py\", line 2840, in eval\r\n    return self.apply('eval', **kwargs)\r\n  File \"~/anaconda3/lib/python3.5/site-packages/pandas/core/internals.py\", line 2823, in apply\r\n    applied = getattr(b, f)(**kwargs)\r\n  File \"~/anaconda3/lib/python3.5/site-packages/pandas/core/internals.py\", line 1155, in eval\r\n    fastpath=True,)]\r\n  File \"~/anaconda3/lib/python3.5/site-packages/pandas/core/internals.py\", line 169, in make_block\r\n    return make_block(values, placement=placement, ndim=ndim, **kwargs)\r\n  File \"~/anaconda3/lib/python3.5/site-packages/pandas/core/internals.py\", line 2454, in make_block\r\n    placement=placement)\r\n  File \"~/anaconda3/lib/python3.5/site-packages/pandas/core/internals.py\", line 78, in __init__\r\n    raise ValueError('Wrong number of dimensions')\r\nValueError: Wrong number of dimensions\r\n```\r\n#### output of ``pd.show_versions()``\r\n```\r\nINSTALLED VERSIONS\r\n------------------\r\ncommit: None\r\npython: 3.5.1.final.0\r\npython-bits: 64\r\nOS: Linux\r\nOS-release: 3.19.0-51-generic\r\nmachine: x86_64\r\nprocessor: x86_64\r\nbyteorder: little\r\nLC_ALL: None\r\nLANG: en_US.UTF-8\r\n\r\npandas: 0.17.1\r\nnose: 1.3.7\r\npip: 8.0.3\r\nsetuptools: 20.2.2\r\nCython: 0.23.4\r\nnumpy: 1.10.4\r\nscipy: 0.17.0\r\nstatsmodels: 0.6.1\r\nIPython: 4.0.3\r\nsphinx: 1.3.5\r\npatsy: 0.4.0\r\ndateutil: 2.5.0\r\npytz: 2015.7\r\nblosc: None\r\nbottleneck: 1.0.0\r\ntables: 3.2.2\r\nnumexpr: 2.4.6\r\nmatplotlib: 1.5.1\r\nopenpyxl: 2.3.2\r\nxlrd: 0.9.4\r\nxlwt: 1.0.0\r\nxlsxwriter: 0.8.4\r\nlxml: 3.5.0\r\nbs4: 4.4.1\r\nhtml5lib: None\r\nhttplib2: None\r\napiclient: None\r\nsqlalchemy: 1.0.11\r\npymysql: None\r\npsycopg2: None\r\nJinja2: None\r\n```"},"comment":{"url":"https://api.github.com/repos/pydata/pandas/issues/comments/194037238","html_url":"https://github.com/pydata/pandas/issues/12564#issuecomment-194037238","issue_url":"https://api.github.com/repos/pydata/pandas/issues/12564","id":194037238,"user":{"login":"pganssle","id":1377457,"avatar_url":"https://avatars.githubusercontent.com/u/1377457?v=3","gravatar_id":"","url":"https://api.github.com/users/pganssle","html_url":"https://github.com/pganssle","followers_url":"https://api.github.com/users/pganssle/followers","following_url":"https://api.github.com/users/pganssle/following{/other_user}","gists_url":"https://api.github.com/users/pganssle/gists{/gist_id}","starred_url":"https://api.github.com/users/pganssle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pganssle/subscriptions","organizations_url":"https://api.github.com/users/pganssle/orgs","repos_url":"https://api.github.com/users/pganssle/repos","events_url":"https://api.github.com/users/pganssle/events{/privacy}","received_events_url":"https://api.github.com/users/pganssle/received_events","type":"User","site_admin":false},"created_at":"2016-03-09T00:32:51Z","updated_at":"2016-03-09T00:32:51Z","body":"My use case, if you want to know, was that I just wanted to discard rows with `np.inf` in any column (though you could imagine the same thing with 0 or something). I find it easier and more readable to just broadcast the comparison across the entire DataFrame, even though logically speaking I know that the comparison really only needs to be applied to the float columns."}}