def check_invalid_utf8_sequences(utf8_string, meaning, conn_id=None):
    """
    Examine a UTF-8 encoded string and raise a `pywbem.XMLParseError` exception
    if the string contains invalid UTF-8 sequences (incorrectly encoded or
    ill-formed).

    This function works in both "wide" and "narrow" Unicode builds of Python
    and supports the full range of Unicode characters from U+0000 to U+10FFFF.

    This function is used to improve the error information raised from Python's
    `xml.dom.minidom` and `xml.sax` packages and should be called only after
    having catched an `ExpatError` from `xml.dom.minidom` or a
    `SAXParseException` from `xml.sax` .

    Parameters:

      utf8_string (:term:`byte string`):
        The UTF-8 encoded XML string to be examined.

      meaning (:term:`string`):
        Short text with meaning of the XML string, for messages in exceptions.

      conn_id (:term:`connection id`): Connection ID to be used in any
        exceptions that may be raised.

    Returns:

      :term:`unicode string`: The input string, converted to Unicode.

    Raises:

      TypeError: Invoked with incorrect Python object type for `utf8_xml`.

      pywbem.XMLParseError: `utf8_xml` contains invalid UTF-8 sequences.

    Notes on Unicode support in Python:

    (1) For internally representing Unicode characters in the unicode type, a
        "wide" Unicode build of Python uses UTF-32, while a "narrow" Unicode
        build uses UTF-16. The difference is visible to Python programs for
        Unicode characters assigned to code points above U+FFFF: The "narrow"
        build uses 2 characters (a surrogate pair) for them, while the "wide"
        build uses just 1 character. This affects all position- and
        length-oriented functions, such as `len()` or string slicing.

    (2) In a "wide" Unicode build of Python, the Unicode characters assigned to
        code points U+10000 to U+10FFFF are represented directly (using code
        points U+10000 to U+10FFFF) and the surrogate code points
        U+D800...U+DFFF are never used; in a "narrow" Unicode build of Python,
        the Unicode characters assigned to code points U+10000 to U+10FFFF are
        represented using pairs of the surrogate code points U+D800...U+DFFF.

    Notes on the Unicode code points U+D800...U+DFFF ("surrogate code points"):

    (1) These code points have no corresponding Unicode characters assigned,
        because they are reserved for surrogates in the UTF-16 encoding.

    (2) The UTF-8 encoding can technically represent the surrogate code points.
        ISO/IEC 10646 defines that a UTF-8 sequence containing the surrogate
        code points is ill-formed, but it is technically possible that such a
        sequence is in a UTF-8 encoded XML string.

    (3) The Python escapes ``\\u`` and ``\\U`` used in literal strings can
        represent the surrogate code points (as well as all other code points,
        regardless of whether they are assigned to Unicode characters).

    (4) The Python `encode()` and `decode()` functions successfully
        translate the surrogate code points back and forth for encoding UTF-8.

        For example, ``b'\\xed\\xb0\\x80'.decode("utf-8") = u'\\udc00'``.

    (5) Because Python supports the encoding and decoding of UTF-8 sequences
        also for the surrogate code points, the "narrow" Unicode build of
        Python can be (mis-)used to transport each surrogate unit separately
        encoded in (ill-formed) UTF-8.

        For example, code point U+10122 can be (illegally) created from a
        sequence of code points U+D800,U+DD22 represented in UTF-8:

          ``b'\\xED\\xA0\\x80\\xED\\xB4\\xA2'.decode("utf-8") = u'\\U00010122'``

        while the correct UTF-8 sequence for this code point is:

          ``u'\\U00010122'.encode("utf-8") = b'\\xf0\\x90\\x84\\xa2'``
    """

    context_before = 16    # number of chars to print before any bad chars
    context_after = 16     # number of chars to print after any bad chars

    try:
        assert isinstance(utf8_string, six.binary_type)
    except AssertionError:
        raise TypeError(
            _format("utf8_string parameter is not a byte string, but has "
                    "type {0}", type(utf8_string)))

    # Check for ill-formed UTF-8 sequences. This needs to be done
    # before the str type gets decoded to unicode, because afterwards
    # surrogates produced from ill-formed UTF-8 cannot be distinguished from
    # legally produced surrogates (for code points above U+FFFF).
    ifs_list = list()
    for m in _ILL_FORMED_UTF8_RE.finditer(utf8_string):
        ifs_pos = m.start(1)
        ifs_seq = m.group(1)
        ifs_list.append((ifs_pos, ifs_seq))
    if ifs_list:
        exc_txt = _format("Ill-formed (surrogate) UTF-8 Byte sequences found "
                          "in {0}:", meaning)
        for (ifs_pos, ifs_seq) in ifs_list:
            exc_txt += "\n  At offset {0}:".format(ifs_pos)
            for ifs_ord in six.iterbytes(ifs_seq):
                exc_txt += " 0x{0:02X}".format(ifs_ord)
            cpos1 = max(ifs_pos - context_before, 0)
            cpos2 = min(ifs_pos + context_after, len(utf8_string))
            exc_txt += _format(", CIM-XML snippet: {0!A}",
                               utf8_string[cpos1:cpos2])
        raise XMLParseError(exc_txt, conn_id=conn_id)

    # Check for incorrectly encoded UTF-8 sequences.
    # @ibm.13@ Simplified logic (removed loop).
    try:
        utf8_string_u = utf8_string.decode("utf-8")
    except UnicodeDecodeError as exc:
        # Only raised for incorrectly encoded UTF-8 sequences; technically
        # correct sequences that are ill-formed (e.g. representing surrogates)
        # do not cause this exception to be raised.
        # If more than one incorrectly encoded sequence is present, only
        # information about the first one is returned in the exception object.
        # Also, the stated reason (in _msg) is not always correct.

        # pylint: disable=unbalanced-tuple-unpacking
        unused_codec, unused_str, _p1, _p2, unused_msg = exc.args

        exc_txt = "Incorrectly encoded UTF-8 Byte sequences found in {0}". \
            format(meaning)
        exc_txt += "\n  At offset {0}:".format(_p1)
        ies_seq = utf8_string[_p1:_p2 + 1]
        for ies_ord in six.iterbytes(ies_seq):
            exc_txt += " 0x{0:02X}".format(ies_ord)
        cpos1 = max(_p1 - context_before, 0)
        cpos2 = min(_p2 + context_after, len(utf8_string))
        exc_txt += _format(", CIM-XML snippet: {0!A}",
                           utf8_string[cpos1:cpos2])
        raise XMLParseError(exc_txt, conn_id=conn_id)

    return utf8_string_u
def check_invalid_utf8_sequences ( utf8_string , meaning , conn_id = None ) : context_before = 16 context_after = 16 try : assert isinstance ( utf8_string , six . binary_type ) except AssertionError : raise TypeError ( _format ( "utf8_string parameter is not a byte string, but has " "type {0}" , type ( utf8_string ) ) ) ifs_list = list ( ) for m in _ILL_FORMED_UTF8_RE . finditer ( utf8_string ) : ifs_pos = m . start ( 1 ) ifs_seq = m . group ( 1 ) ifs_list . append ( ( ifs_pos , ifs_seq ) ) if ifs_list : exc_txt = _format ( "Ill-formed (surrogate) UTF-8 Byte sequences found " "in {0}:" , meaning ) for ( ifs_pos , ifs_seq ) in ifs_list : exc_txt += "\n  At offset {0}:" . format ( ifs_pos ) for ifs_ord in six . iterbytes ( ifs_seq ) : exc_txt += " 0x{0:02X}" . format ( ifs_ord ) cpos1 = max ( ifs_pos - context_before , 0 ) cpos2 = min ( ifs_pos + context_after , len ( utf8_string ) ) exc_txt += _format ( ", CIM-XML snippet: {0!A}" , utf8_string [ cpos1 : cpos2 ] ) raise XMLParseError ( exc_txt , conn_id = conn_id ) try : utf8_string_u = utf8_string . decode ( "utf-8" ) except UnicodeDecodeError as exc : unused_codec , unused_str , _p1 , _p2 , unused_msg = exc . args exc_txt = "Incorrectly encoded UTF-8 Byte sequences found in {0}" . format ( meaning ) exc_txt += "\n  At offset {0}:" . format ( _p1 ) ies_seq = utf8_string [ _p1 : _p2 + 1 ] for ies_ord in six . iterbytes ( ies_seq ) : exc_txt += " 0x{0:02X}" . format ( ies_ord ) cpos1 = max ( _p1 - context_before , 0 ) cpos2 = min ( _p2 + context_after , len ( utf8_string ) ) exc_txt += _format ( ", CIM-XML snippet: {0!A}" , utf8_string [ cpos1 : cpos2 ] ) raise XMLParseError ( exc_txt , conn_id = conn_id ) return utf8_string_u
Examine a UTF-8 encoded string and raise a `pywbem.XMLParseError` exception
    if the string contains invalid UTF-8 sequences (incorrectly encoded or
    ill-formed).

    This function works in both "wide" and "narrow" Unicode builds of Python
    and supports the full range of Unicode characters from U+0000 to U+10FFFF.

    This function is used to improve the error information raised from Python's
    `xml.dom.minidom` and `xml.sax` packages and should be called only after
    having catched an `ExpatError` from `xml.dom.minidom` or a
    `SAXParseException` from `xml.sax` .

    Parameters:

      utf8_string (:term:`byte string`):
        The UTF-8 encoded XML string to be examined.

      meaning (:term:`string`):
        Short text with meaning of the XML string, for messages in exceptions.

      conn_id (:term:`connection id`): Connection ID to be used in any
        exceptions that may be raised.

    Returns:

      :term:`unicode string`: The input string, converted to Unicode.

    Raises:

      TypeError: Invoked with incorrect Python object type for `utf8_xml`.

      pywbem.XMLParseError: `utf8_xml` contains invalid UTF-8 sequences.

    Notes on Unicode support in Python:

    (1) For internally representing Unicode characters in the unicode type, a
        "wide" Unicode build of Python uses UTF-32, while a "narrow" Unicode
        build uses UTF-16. The difference is visible to Python programs for
        Unicode characters assigned to code points above U+FFFF: The "narrow"
        build uses 2 characters (a surrogate pair) for them, while the "wide"
        build uses just 1 character. This affects all position- and
        length-oriented functions, such as `len()` or string slicing.

    (2) In a "wide" Unicode build of Python, the Unicode characters assigned to
        code points U+10000 to U+10FFFF are represented directly (using code
        points U+10000 to U+10FFFF) and the surrogate code points
        U+D800...U+DFFF are never used; in a "narrow" Unicode build of Python,
        the Unicode characters assigned to code points U+10000 to U+10FFFF are
        represented using pairs of the surrogate code points U+D800...U+DFFF.

    Notes on the Unicode code points U+D800...U+DFFF ("surrogate code points"):

    (1) These code points have no corresponding Unicode characters assigned,
        because they are reserved for surrogates in the UTF-16 encoding.

    (2) The UTF-8 encoding can technically represent the surrogate code points.
        ISO/IEC 10646 defines that a UTF-8 sequence containing the surrogate
        code points is ill-formed, but it is technically possible that such a
        sequence is in a UTF-8 encoded XML string.

    (3) The Python escapes ``\\u`` and ``\\U`` used in literal strings can
        represent the surrogate code points (as well as all other code points,
        regardless of whether they are assigned to Unicode characters).

    (4) The Python `encode()` and `decode()` functions successfully
        translate the surrogate code points back and forth for encoding UTF-8.

        For example, ``b'\\xed\\xb0\\x80'.decode("utf-8") = u'\\udc00'``.

    (5) Because Python supports the encoding and decoding of UTF-8 sequences
        also for the surrogate code points, the "narrow" Unicode build of
        Python can be (mis-)used to transport each surrogate unit separately
        encoded in (ill-formed) UTF-8.

        For example, code point U+10122 can be (illegally) created from a
        sequence of code points U+D800,U+DD22 represented in UTF-8:

          ``b'\\xED\\xA0\\x80\\xED\\xB4\\xA2'.decode("utf-8") = u'\\U00010122'``

        while the correct UTF-8 sequence for this code point is:

          ``u'\\U00010122'.encode("utf-8") = b'\\xf0\\x90\\x84\\xa2'``