{"action":"opened","issue":{"url":"https://api.github.com/repos/jamesturk/jellyfish/issues/63","repository_url":"https://api.github.com/repos/jamesturk/jellyfish","labels_url":"https://api.github.com/repos/jamesturk/jellyfish/issues/63/labels{/name}","comments_url":"https://api.github.com/repos/jamesturk/jellyfish/issues/63/comments","events_url":"https://api.github.com/repos/jamesturk/jellyfish/issues/63/events","html_url":"https://github.com/jamesturk/jellyfish/issues/63","id":189274550,"number":63,"title":"CJellyfish.soundex() chokes on bad Unicode string which Jellyfish.soundex() can handle","user":{"login":"nchammas","id":1039369,"avatar_url":"https://avatars.githubusercontent.com/u/1039369?v=3","gravatar_id":"","url":"https://api.github.com/users/nchammas","html_url":"https://github.com/nchammas","followers_url":"https://api.github.com/users/nchammas/followers","following_url":"https://api.github.com/users/nchammas/following{/other_user}","gists_url":"https://api.github.com/users/nchammas/gists{/gist_id}","starred_url":"https://api.github.com/users/nchammas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nchammas/subscriptions","organizations_url":"https://api.github.com/users/nchammas/orgs","repos_url":"https://api.github.com/users/nchammas/repos","events_url":"https://api.github.com/users/nchammas/events{/privacy}","received_events_url":"https://api.github.com/users/nchammas/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2016-11-15T01:26:59Z","updated_at":"2016-11-15T01:26:59Z","closed_at":null,"body":"Data is messy and oftentimes we get strings that have been encoded incorrectly. Detecting the correct encoding and fixing the mal-encoded strings seems difficult to automate.\r\n\r\nIn this particular case, the C implementation of `soundex()` seems to choke on a bad Unicode string that the Python implementation can handle fine. It would be ideal if the C implementation behaved like in the Python one.\r\n\r\nRepro in Python 3.5:\r\n\r\n```python\r\n>>> s = b'\\xef\\xbf\\xbd1 Championship Dr'.decode('utf-8')\r\n>>> import jellyfish\r\n>>> jellyfish.soundex(s)\r\nTraceback (most recent call last):\r\n  File \"<stdin>\", line 1, in <module>\r\nUnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte\r\n>>> import jellyfish._jellyfish as jellyfish\r\n>>> jellyfish.soundex(s)\r\n'�251'\r\n```\r\n\r\nRelates to #55 and #56 in that the C implementations of certain functions don't handle Unicode (whether good or bad) the same way that the Python implementations do."}}
{"actor":{"display_login":"nchammas"}}