def _print_lexical_token_stream(self, data):  # pragma: no cover
        tokens = self._drop_syntactic_errors(self.get_tokens(data))
        for tk in tokens:
            if tk.is_tag:
                if tk.option:
                    print(tk.type.upper() + " " + tk.tag_name + ", option = \"" + tk.option + "\"")
                else:
                    print(tk.type.upper() + " " + tk.tag_name)
            elif tk.is_data:
                print(tk.type.upper() + " \"" + tk.text + "\"")
            elif tk.is_newline:
                print(tk.type.upper())
Given an input text, print out the lexical token stream.
dataset/ETHPy150Open ellmetha/django-precise-bbcode/precise_bbcode/bbcode/parser.py/BBCodeParser._print_lexical_token_stream