<SYSTEM_TASK:>
Solve the following problem using Python, implementing the functions described below, one line at a time
<END_TASK>
<USER_TASK:>
Description:
 def parse(stream, with_text=False):
 # type: (Iterator[str], bool) -> Iterator[Union[Tuple[str, LexicalUnit], LexicalUnit]] """Generates lexical units from a character stream. Args: stream (Iterator[str]):
 A character stream containing lexical units, superblanks and other text. with_text (Optional[bool]):
 A boolean defining whether to output preceding text with each lexical unit. Yields: :class:`LexicalUnit`: The next lexical unit found in the character stream. (if `with_text` is False) \n *(str, LexicalUnit)* - The next lexical unit found in the character stream and the the text that seperated it from the prior unit in a tuple. (if with_text is True) """


    buffer = ''
    text_buffer = ''
    in_lexical_unit = False
    in_superblank = False

    for char in stream:
        if in_superblank:
            if char == ']':
                in_superblank = False
                text_buffer += char
            elif char == '\\':
                text_buffer += char
                text_buffer += next(stream)
            else:
                text_buffer += char
        elif in_lexical_unit:
            if char == '$':
                if with_text:
                    yield (text_buffer, LexicalUnit(buffer))
                else:
                    yield LexicalUnit(buffer)
                buffer = ''
                text_buffer = ''
                in_lexical_unit = False
            elif char == '\\':
                buffer += char
                buffer += next(stream)
            else:
                buffer += char
        else:
            if char == '[':
                in_superblank = True
                text_buffer += char
            elif char == '^':
                in_lexical_unit = True
            elif char == '\\':
                text_buffer += char
                text_buffer += next(stream)
            else:
                text_buffer += char