{"action":"created","issue":{"url":"https://api.github.com/repos/esnme/ultrajson/issues/216","repository_url":"https://api.github.com/repos/esnme/ultrajson","labels_url":"https://api.github.com/repos/esnme/ultrajson/issues/216/labels{/name}","comments_url":"https://api.github.com/repos/esnme/ultrajson/issues/216/comments","events_url":"https://api.github.com/repos/esnme/ultrajson/issues/216/events","html_url":"https://github.com/esnme/ultrajson/pull/216","id":144367911,"number":216,"title":"Failing test about split multibyte character","user":{"login":"dlecocq","id":952157,"avatar_url":"https://avatars.githubusercontent.com/u/952157?v=3","gravatar_id":"","url":"https://api.github.com/users/dlecocq","html_url":"https://github.com/dlecocq","followers_url":"https://api.github.com/users/dlecocq/followers","following_url":"https://api.github.com/users/dlecocq/following{/other_user}","gists_url":"https://api.github.com/users/dlecocq/gists{/gist_id}","starred_url":"https://api.github.com/users/dlecocq/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dlecocq/subscriptions","organizations_url":"https://api.github.com/users/dlecocq/orgs","repos_url":"https://api.github.com/users/dlecocq/repos","events_url":"https://api.github.com/users/dlecocq/events{/privacy}","received_events_url":"https://api.github.com/users/dlecocq/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-03-29T19:49:16Z","updated_at":"2016-03-29T19:54:55Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/esnme/ultrajson/pulls/216","html_url":"https://github.com/esnme/ultrajson/pull/216","diff_url":"https://github.com/esnme/ultrajson/pull/216.diff","patch_url":"https://github.com/esnme/ultrajson/pull/216.patch"},"body":"When streaming JSON from a file-like object, it's possible for two successive reads to happen to split a multibyte character. In that case, remaining bytes must be read before attempting to interpret the sequence correctly.\r\n\r\nFor example, `ü` is encoded as `\\u00fc`. If `\\u00` should appear in one read, and the `fc` in the next, the current implementation attempts to interpret `\\u00`, throwing an exception:\r\n\r\n`ValueError: Unexpected character in unicode escape sequence when decoding 'string'`\r\n\r\n@vadim-moz mentioned this is a problem in some JSON implementations, which spurred my curiosity. Also, @benkirzhner, @sistawendy, and @b4hand may be interested."},"comment":{"url":"https://api.github.com/repos/esnme/ultrajson/issues/comments/203072197","html_url":"https://github.com/esnme/ultrajson/pull/216#issuecomment-203072197","issue_url":"https://api.github.com/repos/esnme/ultrajson/issues/216","id":203072197,"user":{"login":"vadim-moz","id":9726922,"avatar_url":"https://avatars.githubusercontent.com/u/9726922?v=3","gravatar_id":"","url":"https://api.github.com/users/vadim-moz","html_url":"https://github.com/vadim-moz","followers_url":"https://api.github.com/users/vadim-moz/followers","following_url":"https://api.github.com/users/vadim-moz/following{/other_user}","gists_url":"https://api.github.com/users/vadim-moz/gists{/gist_id}","starred_url":"https://api.github.com/users/vadim-moz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vadim-moz/subscriptions","organizations_url":"https://api.github.com/users/vadim-moz/orgs","repos_url":"https://api.github.com/users/vadim-moz/repos","events_url":"https://api.github.com/users/vadim-moz/events{/privacy}","received_events_url":"https://api.github.com/users/vadim-moz/received_events","type":"User","site_admin":false},"created_at":"2016-03-29T19:54:55Z","updated_at":"2016-03-29T19:54:55Z","body":"One more problem case: \r\n\r\nThe http is 8-bit protocol. So `ü` can be encoded (correctly) as a utf-8 sequence [C3, 9C]. If the read breaks between these bytes, it would also cause a failure."}}