{"action":"opened","issue":{"url":"https://api.github.com/repos/pandas-dev/pandas/issues/21767","repository_url":"https://api.github.com/repos/pandas-dev/pandas","labels_url":"https://api.github.com/repos/pandas-dev/pandas/issues/21767/labels{/name}","comments_url":"https://api.github.com/repos/pandas-dev/pandas/issues/21767/comments","events_url":"https://api.github.com/repos/pandas-dev/pandas/issues/21767/events","html_url":"https://github.com/pandas-dev/pandas/issues/21767","id":338989175,"node_id":"MDU6SXNzdWUzMzg5ODkxNzU=","number":21767,"title":"`pandas.Categorical.from_codes` incorrectly converts NaN codes to 0.","user":{"login":"miker985","id":26555712,"node_id":"MDQ6VXNlcjI2NTU1NzEy","avatar_url":"https://avatars0.githubusercontent.com/u/26555712?v=4","gravatar_id":"","url":"https://api.github.com/users/miker985","html_url":"https://github.com/miker985","followers_url":"https://api.github.com/users/miker985/followers","following_url":"https://api.github.com/users/miker985/following{/other_user}","gists_url":"https://api.github.com/users/miker985/gists{/gist_id}","starred_url":"https://api.github.com/users/miker985/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/miker985/subscriptions","organizations_url":"https://api.github.com/users/miker985/orgs","repos_url":"https://api.github.com/users/miker985/repos","events_url":"https://api.github.com/users/miker985/events{/privacy}","received_events_url":"https://api.github.com/users/miker985/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-07-06T15:53:56Z","updated_at":"2018-07-06T15:53:56Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"#### Code Sample, a copy-pastable example if possible\r\n\r\n```python\r\nimport pandas\r\n\r\n\r\ndef test_categorical_from_codes():\r\n    \"Demonstrate issue with pandas.Categorical.from_codes.\"\r\n    codes = pandas.Series([1, 2, float('NaN')])\r\n    categories = ['foo', 'bar', 'baz']\r\n    c = pandas.Categorical.from_codes(codes, categories, ordered=True)\r\n\r\n    expected = pandas.Categorical.from_codes([1, 2, -1], categories,\r\n                                             ordered=True)\r\n\r\n    pandas.util.testing.assert_categorical_equal(c, expected)\r\n```\r\n#### Problem description\r\n\r\n`pandas.Categorical.from_codes` is incorrectly coercing `NaN` code values into `0` values. I think it should be raising a `ValueError`. I believe this is because `pandas._libs.algos.ensure_int8/16/32/64` is behaving in an unanticipated manner on `numpy` arrays.\r\n\r\nWhen given a single value all the ensure functions behave correctly. **But when given a `numpy.array` they do not**.\r\n\r\n```python\r\nimport pandas\r\nimport pytest\r\n\r\ndef test_ensure_on_value():\r\n    \"\"\"\r\n    Test that ensure_int functions raise a ValueError when converting a NaN.\r\n\r\n    These tests ALL PASS, and should pass.\r\n    \"\"\"\r\n    with pytest.raises(ValueError):\r\n        pandas._libs.algos.ensure_int8(float('NaN'))\r\n\r\n    with pytest.raises(ValueError):\r\n        pandas._libs.algos.ensure_int16(float('NaN'))\r\n\r\n    with pytest.raises(ValueError):\r\n        pandas._libs.algos.ensure_int32(float('NaN'))\r\n\r\n    with pytest.raises(ValueError):\r\n        pandas._libs.algos.ensure_int64(float('NaN'))\r\n\r\n\r\ndef test_ensure_on_array():\r\n    \"\"\"\r\n    These tests ALL PASS.\r\n\r\n    I assume all of these are supposed to raise ValueErrors.\r\n\r\n    Interestingly I did not get 0 values with ensure_int32 and ensure_int64.\r\n    \"\"\"\r\n    a8 = numpy.array([float('NaN')])\r\n    c8 = pandas._libs.algos.ensure_int8(a8)\r\n    numpy.testing.assert_array_equal(c8, numpy.array([0]))\r\n\r\n    a16 = numpy.array([float('NaN')])\r\n    c16 = pandas._libs.algos.ensure_int16(a16)\r\n    numpy.testing.assert_array_equal(c16, numpy.array([0]))\r\n\r\n    a32 = numpy.array([float('NaN')])\r\n    c32 = pandas._libs.algos.ensure_int32(a32)\r\n    numpy.testing.assert_array_equal(c32, numpy.array([-2147483648]))\r\n\r\n    a64 = numpy.array([float('NaN')])\r\n    c64 = pandas._libs.algos.ensure_int64(a64)\r\n    numpy.testing.assert_array_equal(c64, numpy.array([-9223372036854775808]))\r\n```\r\n\r\n#### Expected Output\r\n\r\n`test_categorical_from_codes` should pass. Instead it contains the codes `[1, 2, 0]`.\r\n\r\n#### Output of ``pd.show_versions()``\r\n\r\n<details>\r\n\r\n[paste the output of ``pd.show_versions()`` here below this line]\r\n>>> pandas.show_versions()\r\n/Users/miker985/miniconda3/envs/general3/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use \"pip install psycopg2-binary\" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.\r\n  \"\"\")\r\n\r\nINSTALLED VERSIONS\r\n------------------\r\ncommit: None\r\npython: 3.6.0.final.0\r\npython-bits: 64\r\nOS: Darwin\r\nOS-release: 17.3.0\r\nmachine: x86_64\r\nprocessor: i386\r\nbyteorder: little\r\nLC_ALL: None\r\nLANG: en_US.UTF-8\r\nLOCALE: en_US.UTF-8\r\n\r\npandas: 0.23.2\r\npytest: 3.6.3\r\npip: 10.0.1\r\nsetuptools: 27.2.0\r\nCython: None\r\nnumpy: 1.12.1\r\nscipy: None\r\npyarrow: None\r\nxarray: None\r\nIPython: None\r\nsphinx: None\r\npatsy: None\r\ndateutil: 2.6.0\r\npytz: 2017.2\r\nblosc: None\r\nbottleneck: None\r\ntables: None\r\nnumexpr: None\r\nfeather: None\r\nmatplotlib: 2.0.0\r\nopenpyxl: None\r\nxlrd: None\r\nxlwt: None\r\nxlsxwriter: None\r\nlxml: None\r\nbs4: 4.6.0\r\nhtml5lib: None\r\nsqlalchemy: 1.2.6\r\npymysql: None\r\npsycopg2: 2.7.4 (dt dec pq3 ext lo64)\r\njinja2: 2.10\r\ns3fs: None\r\nfastparquet: None\r\npandas_gbq: None\r\npandas_datareader: None\r\n\r\n</details>\r\n"}}
{"actor":{"display_login":"miker985"}}