When we have improperly encoded surrogates, we can still see the
        bits that they were meant to represent.

        The surrogates were meant to encode a 20-bit number, to which we
        add 0x10000 to get a codepoint. That 20-bit number now appears in
        this form:

          11101101 1010abcd 10efghij 11101101 1011klmn 10opqrst

        The CESU8_RE above matches byte sequences of this form. Then we need
        to extract the bits and assemble a codepoint number from them.

def _buffer_decode_surrogates(sup, input, errors, final):
        """
        When we have improperly encoded surrogates, we can still see the
        bits that they were meant to represent.

        The surrogates were meant to encode a 20-bit number, to which we
        add 0x10000 to get a codepoint. That 20-bit number now appears in
        this form:

          11101101 1010abcd 10efghij 11101101 1011klmn 10opqrst

        The CESU8_RE above matches byte sequences of this form. Then we need
        to extract the bits and assemble a codepoint number from them.
        """
        if len(input) < 6:
            if final:
                # We found 0xed near the end of the stream, and there aren't
                # six bytes to decode. Delegate to the superclass method to
                # handle it as normal UTF-8. It might be a Hangul character
                # or an error.
                return sup(input, errors, final)
            else:
                # We found a surrogate, the stream isn't over yet, and we don't
                # know enough of the following bytes to decode anything, so
                # consume zero bytes and wait.
                return '', 0
        else:
            if CESU8_RE.match(input):
                # Given this is a CESU-8 sequence, do some math to pull out
                # the intended 20-bit value, and consume six bytes.
                codepoint = (
                    ((input[1] & 0x0f) << 16) +
                    ((input[2] & 0x3f) << 10) +
                    ((input[4] & 0x0f) << 6) +
                    (input[5] & 0x3f) +
                    0x10000
                )
                return chr(codepoint), 6
            else:
                # This looked like a CESU-8 sequence, but it wasn't one.
                # 0xed indicates the start of a three-byte sequence, so give
                # three bytes to the superclass to decode as usual.
                return sup(input[:3], errors, False)