{"push_id":1162008338,"size":1,"distinct_size":1,"ref":"refs/heads/string-decoder-simplify","head":"d11e41cecbab0d2ffa4491d6fb787282fe621fe1","before":"dc94308aa53adaffd327d6234c5408795fc51f44","commits":[{"sha":"d11e41cecbab0d2ffa4491d6fb787282fe621fe1","author":{"name":"Martin von Gagern","email":"0cf17f8b9b6fdceee6cf321dd6765200bf9be87f@gmx.net"},"message":"string_decoder: fix handling of malformed utf8\n\nThere have been problems with utf8 decoding in cases where the input\nwas invalid.  Some cases would give different results depending on\nchunking, while others even led to exceptions.  This commit simplifies\nthe code in several ways, reducing the risk of breakage.\n\nMost importantly, the `text` method is not only used for bulk\nconversion of a single chunk, but also for the conversion of the mini\nbuffer `lastChar` while handling characters spanning chunks.  That\nshould make most of the problems due to incorrect handling of special\ncases disappear.\n\nSecondly, that `text` method is now independent of encoding.  The\nencoding-dependent method `complete` now has a single well-defined\ntask: determine the buffer position up to which the input consists of\ncomplete characters.  The actual conversion is handled in a central\nlocation, leading to a cleaner and leaner internal interface.\n\nThirdly, we no longer try to remember just how many bytes we'll need\nfor the next complete character.  We simply try to fill up the\n`nextChar` buffer and perform a conversion on that.  This reduces the\nnumber of internal counter variables from two to one, namely `partial`\nwhich indicates the number of bytes currently stored in `nextChar`.\nA possible drawback of this approach is that there is chance of\ndegraded performance if input arrives one byte at a time and is from a\nscript using long utf8 sequences.  As a benefit, though, this allows us\nto emit a U+FFFD replacement character sooner in cases where the first\nbyte of an utf8 sequence is not followed by the expected number of\ncontinuation bytes.\n\nFixes: https://github.com/nodejs/node/issues/7308","distinct":true,"url":"https://api.github.com/repos/gagern/node/commits/d11e41cecbab0d2ffa4491d6fb787282fe621fe1"}]}
{"actor":{"display_login":"gagern"}}