{"action":"created","comment":{"url":"https://api.github.com/repos/pandas-dev/pandas/pulls/comments/176861923","pull_request_review_id":106647108,"id":176861923,"diff_hunk":"@@ -300,16 +300,19 @@ def get_dispatch(dtypes):\n \n     unique_template = unique_template.format(name=name, dtype=dtype, null_condition=null_condition, float_group=float_group)\n \n-    yield (name, dtype, null_condition, float_group, unique_template)\n+    yield (name, dtype, null_condition, float_group, default_na_value, unique_template)\n }}\n \n \n-{{for name, dtype, null_condition, float_group, unique_template in get_dispatch(dtypes)}}\n+{{for name, dtype, null_condition, float_group, default_na_value, unique_template in get_dispatch(dtypes)}}\n \n cdef class {{name}}HashTable(HashTable):\n \n-    def __cinit__(self, size_hint=1):\n+    def __cinit__(self, size_hint=1, {{dtype}}_t na_value={{default_na_value}},\n+                  bint use_na_value=False):\n         self.table = kh_init_{{dtype}}()\n+        self.na_value = na_value\n+        self.use_na_value = use_na_value","path":"pandas/_libs/hashtable_class_helper.pxi.in","position":null,"original_position":38,"commit_id":"3c184285e8ef3fbffd193985f324cfe42e0f3bab","original_commit_id":"872c24a0658a0a3deab91b8b76be46cf8fe34aed","user":{"login":"WillAyd","id":609873,"avatar_url":"https://avatars0.githubusercontent.com/u/609873?v=4","gravatar_id":"","url":"https://api.github.com/users/WillAyd","html_url":"https://github.com/WillAyd","followers_url":"https://api.github.com/users/WillAyd/followers","following_url":"https://api.github.com/users/WillAyd/following{/other_user}","gists_url":"https://api.github.com/users/WillAyd/gists{/gist_id}","starred_url":"https://api.github.com/users/WillAyd/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/WillAyd/subscriptions","organizations_url":"https://api.github.com/users/WillAyd/orgs","repos_url":"https://api.github.com/users/WillAyd/repos","events_url":"https://api.github.com/users/WillAyd/events{/privacy}","received_events_url":"https://api.github.com/users/WillAyd/received_events","type":"User","site_admin":false},"body":"Just to be a little more explicit something along the lines of\r\n\r\n```python\r\ncdef f(object x=None):\r\n    cdef:\r\n        {{template_type}} x_val\r\n\r\n    if x is not None:\r\n        x_val = <{{template_type}}> x\r\n    ...\r\n    with nogil:\r\n         ...\r\n```\r\n\r\nShould still work because you aren't trying to access any Python objects within the nogil - the compiler should know that x_val is of type `{{template_type}}` and work with that accordingly, even though that value doesn't get defined until runtime","created_at":"2018-03-23T20:59:50Z","updated_at":"2018-03-23T20:59:51Z","html_url":"https://github.com/pandas-dev/pandas/pull/20473#discussion_r176861923","pull_request_url":"https://api.github.com/repos/pandas-dev/pandas/pulls/20473","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/pandas-dev/pandas/pulls/comments/176861923"},"html":{"href":"https://github.com/pandas-dev/pandas/pull/20473#discussion_r176861923"},"pull_request":{"href":"https://api.github.com/repos/pandas-dev/pandas/pulls/20473"}},"in_reply_to_id":176849814},"pull_request":{"url":"https://api.github.com/repos/pandas-dev/pandas/pulls/20473","id":177161020,"html_url":"https://github.com/pandas-dev/pandas/pull/20473","diff_url":"https://github.com/pandas-dev/pandas/pull/20473.diff","patch_url":"https://github.com/pandas-dev/pandas/pull/20473.patch","issue_url":"https://api.github.com/repos/pandas-dev/pandas/issues/20473","number":20473,"state":"open","locked":false,"title":"[WIP]: Parametrized NA sentinel for factorize","user":{"login":"TomAugspurger","id":1312546,"avatar_url":"https://avatars3.githubusercontent.com/u/1312546?v=4","gravatar_id":"","url":"https://api.github.com/users/TomAugspurger","html_url":"https://github.com/TomAugspurger","followers_url":"https://api.github.com/users/TomAugspurger/followers","following_url":"https://api.github.com/users/TomAugspurger/following{/other_user}","gists_url":"https://api.github.com/users/TomAugspurger/gists{/gist_id}","starred_url":"https://api.github.com/users/TomAugspurger/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TomAugspurger/subscriptions","organizations_url":"https://api.github.com/users/TomAugspurger/orgs","repos_url":"https://api.github.com/users/TomAugspurger/repos","events_url":"https://api.github.com/users/TomAugspurger/events{/privacy}","received_events_url":"https://api.github.com/users/TomAugspurger/received_events","type":"User","site_admin":false},"body":"Adds a new keyword `na_value` to control the NA sentinel inside the factorize\r\nroutine.\r\n\r\n```python\r\nIn [3]: arr = np.array([0, 1, 0, 2], dtype='u8')\r\n\r\nIn [4]: pd.factorize(arr)\r\nOut[4]: (array([0, 1, 0, 2]), array([0, 1, 2], dtype=uint64))\r\n\r\nIn [5]: pd.factorize(arr, na_value=0)\r\nOut[5]: (array([-1,  0, -1,  1]), array([1, 2], dtype=uint64))\r\n```\r\n\r\nThe basic idea is that our hashtables now have two \"modes\" for detecting NA values\r\n\r\n1. The previous way, based on the templated na_condition\r\n2. Equality to a user-specified NA value.\r\n\r\nNote: specifying NA value does not \"turn off\" the old way of checking NAs.\r\n\r\n```python\r\nIn [2]: pd.factorize(np.array([0.0, np.nan, 1.0]), na_value=0.0)\r\nOut[2]: (array([-1, -1,  0]), array([1.]))\r\n```\r\n\r\nI'm sure the implementation can be cleaned up, but I wanted to put this up for feedback. Hopefully someone can tell me how to avoid the `use_default_na` nonsense :)\r\n\r\ncc @WillAyd\r\n\r\ncloses https://github.com/pandas-dev/pandas/issues/20328","created_at":"2018-03-23T19:12:53Z","updated_at":"2018-03-23T20:59:51Z","closed_at":null,"merged_at":null,"merge_commit_sha":"e749412e9f5b5969df9bc8feda353cbf8fc323a7","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[{"id":57296398,"url":"https://api.github.com/repos/pandas-dev/pandas/labels/Algos","name":"Algos","color":"eb6420","default":false},{"id":2822342,"url":"https://api.github.com/repos/pandas-dev/pandas/labels/Missing-data","name":"Missing-data","color":"d7e102","default":false}],"milestone":null,"commits_url":"https://api.github.com/repos/pandas-dev/pandas/pulls/20473/commits","review_comments_url":"https://api.github.com/repos/pandas-dev/pandas/pulls/20473/comments","review_comment_url":"https://api.github.com/repos/pandas-dev/pandas/pulls/comments{/number}","comments_url":"https://api.github.com/repos/pandas-dev/pandas/issues/20473/comments","statuses_url":"https://api.github.com/repos/pandas-dev/pandas/statuses/3c184285e8ef3fbffd193985f324cfe42e0f3bab","head":{"label":"TomAugspurger:parametrized-factorize-na-value","ref":"parametrized-factorize-na-value","sha":"3c184285e8ef3fbffd193985f324cfe42e0f3bab","user":{"login":"TomAugspurger","id":1312546,"avatar_url":"https://avatars3.githubusercontent.com/u/1312546?v=4","gravatar_id":"","url":"https://api.github.com/users/TomAugspurger","html_url":"https://github.com/TomAugspurger","followers_url":"https://api.github.com/users/TomAugspurger/followers","following_url":"https://api.github.com/users/TomAugspurger/following{/other_user}","gists_url":"https://api.github.com/users/TomAugspurger/gists{/gist_id}","starred_url":"https://api.github.com/users/TomAugspurger/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TomAugspurger/subscriptions","organizations_url":"https://api.github.com/users/TomAugspurger/orgs","repos_url":"https://api.github.com/users/TomAugspurger/repos","events_url":"https://api.github.com/users/TomAugspurger/events{/privacy}","received_events_url":"https://api.github.com/users/TomAugspurger/received_events","type":"User","site_admin":false},"repo":{"id":9450481,"name":"pandas","full_name":"TomAugspurger/pandas","owner":{"login":"TomAugspurger","id":1312546,"avatar_url":"https://avatars3.githubusercontent.com/u/1312546?v=4","gravatar_id":"","url":"https://api.github.com/users/TomAugspurger","html_url":"https://github.com/TomAugspurger","followers_url":"https://api.github.com/users/TomAugspurger/followers","following_url":"https://api.github.com/users/TomAugspurger/following{/other_user}","gists_url":"https://api.github.com/users/TomAugspurger/gists{/gist_id}","starred_url":"https://api.github.com/users/TomAugspurger/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TomAugspurger/subscriptions","organizations_url":"https://api.github.com/users/TomAugspurger/orgs","repos_url":"https://api.github.com/users/TomAugspurger/repos","events_url":"https://api.github.com/users/TomAugspurger/events{/privacy}","received_events_url":"https://api.github.com/users/TomAugspurger/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/TomAugspurger/pandas","description":"Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more","fork":true,"url":"https://api.github.com/repos/TomAugspurger/pandas","forks_url":"https://api.github.com/repos/TomAugspurger/pandas/forks","keys_url":"https://api.github.com/repos/TomAugspurger/pandas/keys{/key_id}","collaborators_url":"https://api.github.com/repos/TomAugspurger/pandas/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/TomAugspurger/pandas/teams","hooks_url":"https://api.github.com/repos/TomAugspurger/pandas/hooks","issue_events_url":"https://api.github.com/repos/TomAugspurger/pandas/issues/events{/number}","events_url":"https://api.github.com/repos/TomAugspurger/pandas/events","assignees_url":"https://api.github.com/repos/TomAugspurger/pandas/assignees{/user}","branches_url":"https://api.github.com/repos/TomAugspurger/pandas/branches{/branch}","tags_url":"https://api.github.com/repos/TomAugspurger/pandas/tags","blobs_url":"https://api.github.com/repos/TomAugspurger/pandas/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/TomAugspurger/pandas/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/TomAugspurger/pandas/git/refs{/sha}","trees_url":"https://api.github.com/repos/TomAugspurger/pandas/git/trees{/sha}","statuses_url":"https://api.github.com/repos/TomAugspurger/pandas/statuses/{sha}","languages_url":"https://api.github.com/repos/TomAugspurger/pandas/languages","stargazers_url":"https://api.github.com/repos/TomAugspurger/pandas/stargazers","contributors_url":"https://api.github.com/repos/TomAugspurger/pandas/contributors","subscribers_url":"https://api.github.com/repos/TomAugspurger/pandas/subscribers","subscription_url":"https://api.github.com/repos/TomAugspurger/pandas/subscription","commits_url":"https://api.github.com/repos/TomAugspurger/pandas/commits{/sha}","git_commits_url":"https://api.github.com/repos/TomAugspurger/pandas/git/commits{/sha}","comments_url":"https://api.github.com/repos/TomAugspurger/pandas/comments{/number}","issue_comment_url":"https://api.github.com/repos/TomAugspurger/pandas/issues/comments{/number}","contents_url":"https://api.github.com/repos/TomAugspurger/pandas/contents/{+path}","compare_url":"https://api.github.com/repos/TomAugspurger/pandas/compare/{base}...{head}","merges_url":"https://api.github.com/repos/TomAugspurger/pandas/merges","archive_url":"https://api.github.com/repos/TomAugspurger/pandas/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/TomAugspurger/pandas/downloads","issues_url":"https://api.github.com/repos/TomAugspurger/pandas/issues{/number}","pulls_url":"https://api.github.com/repos/TomAugspurger/pandas/pulls{/number}","milestones_url":"https://api.github.com/repos/TomAugspurger/pandas/milestones{/number}","notifications_url":"https://api.github.com/repos/TomAugspurger/pandas/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/TomAugspurger/pandas/labels{/name}","releases_url":"https://api.github.com/repos/TomAugspurger/pandas/releases{/id}","deployments_url":"https://api.github.com/repos/TomAugspurger/pandas/deployments","created_at":"2013-04-15T14:23:40Z","updated_at":"2017-04-20T12:53:33Z","pushed_at":"2018-03-23T20:36:55Z","git_url":"git://github.com/TomAugspurger/pandas.git","ssh_url":"git@github.com:TomAugspurger/pandas.git","clone_url":"https://github.com/TomAugspurger/pandas.git","svn_url":"https://github.com/TomAugspurger/pandas","homepage":"http://pandas.pydata.org","size":122682,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"bsd-3-clause","name":"BSD 3-Clause \"New\" or \"Revised\" License","spdx_id":"BSD-3-Clause","url":"https://api.github.com/licenses/bsd-3-clause"},"forks":1,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"pandas-dev:master","ref":"master","sha":"02f7da8eee1196ba91c691fbf7c4db2f7905eace","user":{"login":"pandas-dev","id":21206976,"avatar_url":"https://avatars2.githubusercontent.com/u/21206976?v=4","gravatar_id":"","url":"https://api.github.com/users/pandas-dev","html_url":"https://github.com/pandas-dev","followers_url":"https://api.github.com/users/pandas-dev/followers","following_url":"https://api.github.com/users/pandas-dev/following{/other_user}","gists_url":"https://api.github.com/users/pandas-dev/gists{/gist_id}","starred_url":"https://api.github.com/users/pandas-dev/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pandas-dev/subscriptions","organizations_url":"https://api.github.com/users/pandas-dev/orgs","repos_url":"https://api.github.com/users/pandas-dev/repos","events_url":"https://api.github.com/users/pandas-dev/events{/privacy}","received_events_url":"https://api.github.com/users/pandas-dev/received_events","type":"Organization","site_admin":false},"repo":{"id":858127,"name":"pandas","full_name":"pandas-dev/pandas","owner":{"login":"pandas-dev","id":21206976,"avatar_url":"https://avatars2.githubusercontent.com/u/21206976?v=4","gravatar_id":"","url":"https://api.github.com/users/pandas-dev","html_url":"https://github.com/pandas-dev","followers_url":"https://api.github.com/users/pandas-dev/followers","following_url":"https://api.github.com/users/pandas-dev/following{/other_user}","gists_url":"https://api.github.com/users/pandas-dev/gists{/gist_id}","starred_url":"https://api.github.com/users/pandas-dev/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pandas-dev/subscriptions","organizations_url":"https://api.github.com/users/pandas-dev/orgs","repos_url":"https://api.github.com/users/pandas-dev/repos","events_url":"https://api.github.com/users/pandas-dev/events{/privacy}","received_events_url":"https://api.github.com/users/pandas-dev/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/pandas-dev/pandas","description":"Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more","fork":false,"url":"https://api.github.com/repos/pandas-dev/pandas","forks_url":"https://api.github.com/repos/pandas-dev/pandas/forks","keys_url":"https://api.github.com/repos/pandas-dev/pandas/keys{/key_id}","collaborators_url":"https://api.github.com/repos/pandas-dev/pandas/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/pandas-dev/pandas/teams","hooks_url":"https://api.github.com/repos/pandas-dev/pandas/hooks","issue_events_url":"https://api.github.com/repos/pandas-dev/pandas/issues/events{/number}","events_url":"https://api.github.com/repos/pandas-dev/pandas/events","assignees_url":"https://api.github.com/repos/pandas-dev/pandas/assignees{/user}","branches_url":"https://api.github.com/repos/pandas-dev/pandas/branches{/branch}","tags_url":"https://api.github.com/repos/pandas-dev/pandas/tags","blobs_url":"https://api.github.com/repos/pandas-dev/pandas/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/pandas-dev/pandas/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/pandas-dev/pandas/git/refs{/sha}","trees_url":"https://api.github.com/repos/pandas-dev/pandas/git/trees{/sha}","statuses_url":"https://api.github.com/repos/pandas-dev/pandas/statuses/{sha}","languages_url":"https://api.github.com/repos/pandas-dev/pandas/languages","stargazers_url":"https://api.github.com/repos/pandas-dev/pandas/stargazers","contributors_url":"https://api.github.com/repos/pandas-dev/pandas/contributors","subscribers_url":"https://api.github.com/repos/pandas-dev/pandas/subscribers","subscription_url":"https://api.github.com/repos/pandas-dev/pandas/subscription","commits_url":"https://api.github.com/repos/pandas-dev/pandas/commits{/sha}","git_commits_url":"https://api.github.com/repos/pandas-dev/pandas/git/commits{/sha}","comments_url":"https://api.github.com/repos/pandas-dev/pandas/comments{/number}","issue_comment_url":"https://api.github.com/repos/pandas-dev/pandas/issues/comments{/number}","contents_url":"https://api.github.com/repos/pandas-dev/pandas/contents/{+path}","compare_url":"https://api.github.com/repos/pandas-dev/pandas/compare/{base}...{head}","merges_url":"https://api.github.com/repos/pandas-dev/pandas/merges","archive_url":"https://api.github.com/repos/pandas-dev/pandas/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/pandas-dev/pandas/downloads","issues_url":"https://api.github.com/repos/pandas-dev/pandas/issues{/number}","pulls_url":"https://api.github.com/repos/pandas-dev/pandas/pulls{/number}","milestones_url":"https://api.github.com/repos/pandas-dev/pandas/milestones{/number}","notifications_url":"https://api.github.com/repos/pandas-dev/pandas/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/pandas-dev/pandas/labels{/name}","releases_url":"https://api.github.com/repos/pandas-dev/pandas/releases{/id}","deployments_url":"https://api.github.com/repos/pandas-dev/pandas/deployments","created_at":"2010-08-24T01:37:33Z","updated_at":"2018-03-23T19:25:21Z","pushed_at":"2018-03-23T20:37:00Z","git_url":"git://github.com/pandas-dev/pandas.git","ssh_url":"git@github.com:pandas-dev/pandas.git","clone_url":"https://github.com/pandas-dev/pandas.git","svn_url":"https://github.com/pandas-dev/pandas","homepage":"http://pandas.pydata.org","size":120188,"stargazers_count":13563,"watchers_count":13563,"language":"Python","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":5512,"mirror_url":null,"archived":false,"open_issues_count":2489,"license":{"key":"bsd-3-clause","name":"BSD 3-Clause \"New\" or \"Revised\" License","spdx_id":"BSD-3-Clause","url":"https://api.github.com/licenses/bsd-3-clause"},"forks":5512,"open_issues":2489,"watchers":13563,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/pandas-dev/pandas/pulls/20473"},"html":{"href":"https://github.com/pandas-dev/pandas/pull/20473"},"issue":{"href":"https://api.github.com/repos/pandas-dev/pandas/issues/20473"},"comments":{"href":"https://api.github.com/repos/pandas-dev/pandas/issues/20473/comments"},"review_comments":{"href":"https://api.github.com/repos/pandas-dev/pandas/pulls/20473/comments"},"review_comment":{"href":"https://api.github.com/repos/pandas-dev/pandas/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/pandas-dev/pandas/pulls/20473/commits"},"statuses":{"href":"https://api.github.com/repos/pandas-dev/pandas/statuses/3c184285e8ef3fbffd193985f324cfe42e0f3bab"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"WillAyd"}}