<SYSTEM_TASK:>
Solve the following problem using Python, implementing the functions described below, one line at a time
<END_TASK>
<USER_TASK:>
Description:
 def tokenize(text):
 r"""Generator for LaTeX tokens on text, ignoring comments. :param Union[str,iterator,Buffer] text: LaTeX to process \textbf { Do play \textit { nice } . } \begin { tabular } 0 & 1 \\ 2 & 0 \end { tabular } """

    current_token = next_token(text)
    while current_token is not None:
        yield current_token
        current_token = next_token(text)