{"action":"created","comment":{"url":"https://api.github.com/repos/pandas-dev/pandas/pulls/comments/242013022","pull_request_review_id":185423895,"id":242013022,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDI0MjAxMzAyMg==","diff_hunk":"@@ -1229,11 +1229,26 @@ def map(self, mapper):\n         Index(['first', 'second', nan], dtype='object')\n         \"\"\"\n         new_categories = self.categories.map(mapper)\n+\n         try:\n-            return self.from_codes(self._codes.copy(),\n-                                   categories=new_categories,\n-                                   ordered=self.ordered)\n+            if is_dict_like(mapper):\n+                new_value = mapper[np.nan]\n+            else:\n+                new_value = mapper(np.nan)\n+        except (AttributeError, KeyError, TypeError, ValueError):","path":"pandas/core/arrays/categorical.py","position":13,"original_position":13,"commit_id":"628bfac22b466c4f15a26373e099eb2b05598ad8","original_commit_id":"628bfac22b466c4f15a26373e099eb2b05598ad8","user":{"login":"TomAugspurger","id":1312546,"node_id":"MDQ6VXNlcjEzMTI1NDY=","avatar_url":"https://avatars3.githubusercontent.com/u/1312546?v=4","gravatar_id":"","url":"https://api.github.com/users/TomAugspurger","html_url":"https://github.com/TomAugspurger","followers_url":"https://api.github.com/users/TomAugspurger/followers","following_url":"https://api.github.com/users/TomAugspurger/following{/other_user}","gists_url":"https://api.github.com/users/TomAugspurger/gists{/gist_id}","starred_url":"https://api.github.com/users/TomAugspurger/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TomAugspurger/subscriptions","organizations_url":"https://api.github.com/users/TomAugspurger/orgs","repos_url":"https://api.github.com/users/TomAugspurger/repos","events_url":"https://api.github.com/users/TomAugspurger/events{/privacy}","received_events_url":"https://api.github.com/users/TomAugspurger/received_events","type":"User","site_admin":false},"body":"I'm not really comfortable with this. `mapper` is a user-defined function. Consider\r\n\r\n```python\r\ndef f(x):\r\n    if isnan(x):\r\n        raise TypeError\r\n    ...\r\n```\r\n\r\nthat TypeError would be swallowed by pandas.","created_at":"2018-12-17T02:40:03Z","updated_at":"2018-12-17T02:42:48Z","html_url":"https://github.com/pandas-dev/pandas/pull/24275#discussion_r242013022","pull_request_url":"https://api.github.com/repos/pandas-dev/pandas/pulls/24275","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/pandas-dev/pandas/pulls/comments/242013022"},"html":{"href":"https://github.com/pandas-dev/pandas/pull/24275#discussion_r242013022"},"pull_request":{"href":"https://api.github.com/repos/pandas-dev/pandas/pulls/24275"}}},"pull_request":{"url":"https://api.github.com/repos/pandas-dev/pandas/pulls/24275","id":238537835,"node_id":"MDExOlB1bGxSZXF1ZXN0MjM4NTM3ODM1","html_url":"https://github.com/pandas-dev/pandas/pull/24275","diff_url":"https://github.com/pandas-dev/pandas/pull/24275.diff","patch_url":"https://github.com/pandas-dev/pandas/pull/24275.patch","issue_url":"https://api.github.com/repos/pandas-dev/pandas/issues/24275","number":24275,"state":"open","locked":false,"title":"GH24241 make Categorical.map transform nans","user":{"login":"JustinZhengBC","id":29615021,"node_id":"MDQ6VXNlcjI5NjE1MDIx","avatar_url":"https://avatars1.githubusercontent.com/u/29615021?v=4","gravatar_id":"","url":"https://api.github.com/users/JustinZhengBC","html_url":"https://github.com/JustinZhengBC","followers_url":"https://api.github.com/users/JustinZhengBC/followers","following_url":"https://api.github.com/users/JustinZhengBC/following{/other_user}","gists_url":"https://api.github.com/users/JustinZhengBC/gists{/gist_id}","starred_url":"https://api.github.com/users/JustinZhengBC/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JustinZhengBC/subscriptions","organizations_url":"https://api.github.com/users/JustinZhengBC/orgs","repos_url":"https://api.github.com/users/JustinZhengBC/repos","events_url":"https://api.github.com/users/JustinZhengBC/events{/privacy}","received_events_url":"https://api.github.com/users/JustinZhengBC/received_events","type":"User","site_admin":false},"body":"- [ ] closes #24241\r\n- [X] tests added / passed\r\n- [X] passes `git diff upstream/master -u -- \"*.py\" | flake8 --diff`\r\n- [X] whatsnew entry\r\n\r\nAlters `Categorical.map` so that the mapper function is also applied to NaN values. The mentioned bug report brings up the example of calling `apply(pd.isna)` on a categorical series, arguing that the values and dtype of the returned list should be consistent with non-categorical series. This PR makes the values consistent. The discrepancies in dtypes are present in categorical series without NaN's and consistent with documentation.","created_at":"2018-12-13T21:27:36Z","updated_at":"2018-12-17T02:42:48Z","closed_at":null,"merged_at":null,"merge_commit_sha":"c20990a2f4cfd5bc2e1f7fbee321bf404ee05960","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[{"id":78527356,"node_id":"MDU6TGFiZWw3ODUyNzM1Ng==","url":"https://api.github.com/repos/pandas-dev/pandas/labels/Categorical","name":"Categorical","color":"e11d21","default":false},{"id":2822342,"node_id":"MDU6TGFiZWwyODIyMzQy","url":"https://api.github.com/repos/pandas-dev/pandas/labels/Missing-data","name":"Missing-data","color":"d7e102","default":false}],"milestone":null,"commits_url":"https://api.github.com/repos/pandas-dev/pandas/pulls/24275/commits","review_comments_url":"https://api.github.com/repos/pandas-dev/pandas/pulls/24275/comments","review_comment_url":"https://api.github.com/repos/pandas-dev/pandas/pulls/comments{/number}","comments_url":"https://api.github.com/repos/pandas-dev/pandas/issues/24275/comments","statuses_url":"https://api.github.com/repos/pandas-dev/pandas/statuses/628bfac22b466c4f15a26373e099eb2b05598ad8","head":{"label":"JustinZhengBC:BUG-24241","ref":"BUG-24241","sha":"628bfac22b466c4f15a26373e099eb2b05598ad8","user":{"login":"JustinZhengBC","id":29615021,"node_id":"MDQ6VXNlcjI5NjE1MDIx","avatar_url":"https://avatars1.githubusercontent.com/u/29615021?v=4","gravatar_id":"","url":"https://api.github.com/users/JustinZhengBC","html_url":"https://github.com/JustinZhengBC","followers_url":"https://api.github.com/users/JustinZhengBC/followers","following_url":"https://api.github.com/users/JustinZhengBC/following{/other_user}","gists_url":"https://api.github.com/users/JustinZhengBC/gists{/gist_id}","starred_url":"https://api.github.com/users/JustinZhengBC/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JustinZhengBC/subscriptions","organizations_url":"https://api.github.com/users/JustinZhengBC/orgs","repos_url":"https://api.github.com/users/JustinZhengBC/repos","events_url":"https://api.github.com/users/JustinZhengBC/events{/privacy}","received_events_url":"https://api.github.com/users/JustinZhengBC/received_events","type":"User","site_admin":false},"repo":{"id":151362202,"node_id":"MDEwOlJlcG9zaXRvcnkxNTEzNjIyMDI=","name":"pandas","full_name":"JustinZhengBC/pandas","private":false,"owner":{"login":"JustinZhengBC","id":29615021,"node_id":"MDQ6VXNlcjI5NjE1MDIx","avatar_url":"https://avatars1.githubusercontent.com/u/29615021?v=4","gravatar_id":"","url":"https://api.github.com/users/JustinZhengBC","html_url":"https://github.com/JustinZhengBC","followers_url":"https://api.github.com/users/JustinZhengBC/followers","following_url":"https://api.github.com/users/JustinZhengBC/following{/other_user}","gists_url":"https://api.github.com/users/JustinZhengBC/gists{/gist_id}","starred_url":"https://api.github.com/users/JustinZhengBC/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JustinZhengBC/subscriptions","organizations_url":"https://api.github.com/users/JustinZhengBC/orgs","repos_url":"https://api.github.com/users/JustinZhengBC/repos","events_url":"https://api.github.com/users/JustinZhengBC/events{/privacy}","received_events_url":"https://api.github.com/users/JustinZhengBC/received_events","type":"User","site_admin":false},"html_url":"https://github.com/JustinZhengBC/pandas","description":"Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more","fork":true,"url":"https://api.github.com/repos/JustinZhengBC/pandas","forks_url":"https://api.github.com/repos/JustinZhengBC/pandas/forks","keys_url":"https://api.github.com/repos/JustinZhengBC/pandas/keys{/key_id}","collaborators_url":"https://api.github.com/repos/JustinZhengBC/pandas/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/JustinZhengBC/pandas/teams","hooks_url":"https://api.github.com/repos/JustinZhengBC/pandas/hooks","issue_events_url":"https://api.github.com/repos/JustinZhengBC/pandas/issues/events{/number}","events_url":"https://api.github.com/repos/JustinZhengBC/pandas/events","assignees_url":"https://api.github.com/repos/JustinZhengBC/pandas/assignees{/user}","branches_url":"https://api.github.com/repos/JustinZhengBC/pandas/branches{/branch}","tags_url":"https://api.github.com/repos/JustinZhengBC/pandas/tags","blobs_url":"https://api.github.com/repos/JustinZhengBC/pandas/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/JustinZhengBC/pandas/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/JustinZhengBC/pandas/git/refs{/sha}","trees_url":"https://api.github.com/repos/JustinZhengBC/pandas/git/trees{/sha}","statuses_url":"https://api.github.com/repos/JustinZhengBC/pandas/statuses/{sha}","languages_url":"https://api.github.com/repos/JustinZhengBC/pandas/languages","stargazers_url":"https://api.github.com/repos/JustinZhengBC/pandas/stargazers","contributors_url":"https://api.github.com/repos/JustinZhengBC/pandas/contributors","subscribers_url":"https://api.github.com/repos/JustinZhengBC/pandas/subscribers","subscription_url":"https://api.github.com/repos/JustinZhengBC/pandas/subscription","commits_url":"https://api.github.com/repos/JustinZhengBC/pandas/commits{/sha}","git_commits_url":"https://api.github.com/repos/JustinZhengBC/pandas/git/commits{/sha}","comments_url":"https://api.github.com/repos/JustinZhengBC/pandas/comments{/number}","issue_comment_url":"https://api.github.com/repos/JustinZhengBC/pandas/issues/comments{/number}","contents_url":"https://api.github.com/repos/JustinZhengBC/pandas/contents/{+path}","compare_url":"https://api.github.com/repos/JustinZhengBC/pandas/compare/{base}...{head}","merges_url":"https://api.github.com/repos/JustinZhengBC/pandas/merges","archive_url":"https://api.github.com/repos/JustinZhengBC/pandas/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/JustinZhengBC/pandas/downloads","issues_url":"https://api.github.com/repos/JustinZhengBC/pandas/issues{/number}","pulls_url":"https://api.github.com/repos/JustinZhengBC/pandas/pulls{/number}","milestones_url":"https://api.github.com/repos/JustinZhengBC/pandas/milestones{/number}","notifications_url":"https://api.github.com/repos/JustinZhengBC/pandas/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/JustinZhengBC/pandas/labels{/name}","releases_url":"https://api.github.com/repos/JustinZhengBC/pandas/releases{/id}","deployments_url":"https://api.github.com/repos/JustinZhengBC/pandas/deployments","created_at":"2018-10-03T04:49:46Z","updated_at":"2018-11-21T05:48:53Z","pushed_at":"2018-12-15T02:43:44Z","git_url":"git://github.com/JustinZhengBC/pandas.git","ssh_url":"git@github.com:JustinZhengBC/pandas.git","clone_url":"https://github.com/JustinZhengBC/pandas.git","svn_url":"https://github.com/JustinZhengBC/pandas","homepage":"https://pandas.pydata.org","size":140274,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"bsd-3-clause","name":"BSD 3-Clause \"New\" or \"Revised\" License","spdx_id":"BSD-3-Clause","url":"https://api.github.com/licenses/bsd-3-clause","node_id":"MDc6TGljZW5zZTU="},"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"pandas-dev:master","ref":"master","sha":"31a351258ad36392a8251c195f7e0869b90c7467","user":{"login":"pandas-dev","id":21206976,"node_id":"MDEyOk9yZ2FuaXphdGlvbjIxMjA2OTc2","avatar_url":"https://avatars2.githubusercontent.com/u/21206976?v=4","gravatar_id":"","url":"https://api.github.com/users/pandas-dev","html_url":"https://github.com/pandas-dev","followers_url":"https://api.github.com/users/pandas-dev/followers","following_url":"https://api.github.com/users/pandas-dev/following{/other_user}","gists_url":"https://api.github.com/users/pandas-dev/gists{/gist_id}","starred_url":"https://api.github.com/users/pandas-dev/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pandas-dev/subscriptions","organizations_url":"https://api.github.com/users/pandas-dev/orgs","repos_url":"https://api.github.com/users/pandas-dev/repos","events_url":"https://api.github.com/users/pandas-dev/events{/privacy}","received_events_url":"https://api.github.com/users/pandas-dev/received_events","type":"Organization","site_admin":false},"repo":{"id":858127,"node_id":"MDEwOlJlcG9zaXRvcnk4NTgxMjc=","name":"pandas","full_name":"pandas-dev/pandas","private":false,"owner":{"login":"pandas-dev","id":21206976,"node_id":"MDEyOk9yZ2FuaXphdGlvbjIxMjA2OTc2","avatar_url":"https://avatars2.githubusercontent.com/u/21206976?v=4","gravatar_id":"","url":"https://api.github.com/users/pandas-dev","html_url":"https://github.com/pandas-dev","followers_url":"https://api.github.com/users/pandas-dev/followers","following_url":"https://api.github.com/users/pandas-dev/following{/other_user}","gists_url":"https://api.github.com/users/pandas-dev/gists{/gist_id}","starred_url":"https://api.github.com/users/pandas-dev/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pandas-dev/subscriptions","organizations_url":"https://api.github.com/users/pandas-dev/orgs","repos_url":"https://api.github.com/users/pandas-dev/repos","events_url":"https://api.github.com/users/pandas-dev/events{/privacy}","received_events_url":"https://api.github.com/users/pandas-dev/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/pandas-dev/pandas","description":"Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more","fork":false,"url":"https://api.github.com/repos/pandas-dev/pandas","forks_url":"https://api.github.com/repos/pandas-dev/pandas/forks","keys_url":"https://api.github.com/repos/pandas-dev/pandas/keys{/key_id}","collaborators_url":"https://api.github.com/repos/pandas-dev/pandas/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/pandas-dev/pandas/teams","hooks_url":"https://api.github.com/repos/pandas-dev/pandas/hooks","issue_events_url":"https://api.github.com/repos/pandas-dev/pandas/issues/events{/number}","events_url":"https://api.github.com/repos/pandas-dev/pandas/events","assignees_url":"https://api.github.com/repos/pandas-dev/pandas/assignees{/user}","branches_url":"https://api.github.com/repos/pandas-dev/pandas/branches{/branch}","tags_url":"https://api.github.com/repos/pandas-dev/pandas/tags","blobs_url":"https://api.github.com/repos/pandas-dev/pandas/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/pandas-dev/pandas/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/pandas-dev/pandas/git/refs{/sha}","trees_url":"https://api.github.com/repos/pandas-dev/pandas/git/trees{/sha}","statuses_url":"https://api.github.com/repos/pandas-dev/pandas/statuses/{sha}","languages_url":"https://api.github.com/repos/pandas-dev/pandas/languages","stargazers_url":"https://api.github.com/repos/pandas-dev/pandas/stargazers","contributors_url":"https://api.github.com/repos/pandas-dev/pandas/contributors","subscribers_url":"https://api.github.com/repos/pandas-dev/pandas/subscribers","subscription_url":"https://api.github.com/repos/pandas-dev/pandas/subscription","commits_url":"https://api.github.com/repos/pandas-dev/pandas/commits{/sha}","git_commits_url":"https://api.github.com/repos/pandas-dev/pandas/git/commits{/sha}","comments_url":"https://api.github.com/repos/pandas-dev/pandas/comments{/number}","issue_comment_url":"https://api.github.com/repos/pandas-dev/pandas/issues/comments{/number}","contents_url":"https://api.github.com/repos/pandas-dev/pandas/contents/{+path}","compare_url":"https://api.github.com/repos/pandas-dev/pandas/compare/{base}...{head}","merges_url":"https://api.github.com/repos/pandas-dev/pandas/merges","archive_url":"https://api.github.com/repos/pandas-dev/pandas/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/pandas-dev/pandas/downloads","issues_url":"https://api.github.com/repos/pandas-dev/pandas/issues{/number}","pulls_url":"https://api.github.com/repos/pandas-dev/pandas/pulls{/number}","milestones_url":"https://api.github.com/repos/pandas-dev/pandas/milestones{/number}","notifications_url":"https://api.github.com/repos/pandas-dev/pandas/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/pandas-dev/pandas/labels{/name}","releases_url":"https://api.github.com/repos/pandas-dev/pandas/releases{/id}","deployments_url":"https://api.github.com/repos/pandas-dev/pandas/deployments","created_at":"2010-08-24T01:37:33Z","updated_at":"2018-12-17T01:59:28Z","pushed_at":"2018-12-17T01:38:38Z","git_url":"git://github.com/pandas-dev/pandas.git","ssh_url":"git@github.com:pandas-dev/pandas.git","clone_url":"https://github.com/pandas-dev/pandas.git","svn_url":"https://github.com/pandas-dev/pandas","homepage":"https://pandas.pydata.org","size":140429,"stargazers_count":17392,"watchers_count":17392,"language":"Python","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":7003,"mirror_url":null,"archived":false,"open_issues_count":2819,"license":{"key":"bsd-3-clause","name":"BSD 3-Clause \"New\" or \"Revised\" License","spdx_id":"BSD-3-Clause","url":"https://api.github.com/licenses/bsd-3-clause","node_id":"MDc6TGljZW5zZTU="},"forks":7003,"open_issues":2819,"watchers":17392,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/pandas-dev/pandas/pulls/24275"},"html":{"href":"https://github.com/pandas-dev/pandas/pull/24275"},"issue":{"href":"https://api.github.com/repos/pandas-dev/pandas/issues/24275"},"comments":{"href":"https://api.github.com/repos/pandas-dev/pandas/issues/24275/comments"},"review_comments":{"href":"https://api.github.com/repos/pandas-dev/pandas/pulls/24275/comments"},"review_comment":{"href":"https://api.github.com/repos/pandas-dev/pandas/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/pandas-dev/pandas/pulls/24275/commits"},"statuses":{"href":"https://api.github.com/repos/pandas-dev/pandas/statuses/628bfac22b466c4f15a26373e099eb2b05598ad8"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"TomAugspurger"}}