def __init__(self, field):
        self.specials = '()<>@,:;.\"[]'
        self.pos = 0
        self.LWS = ' \t'
        self.CR = '\r\n'
        self.FWS = self.LWS + self.CR
        self.atomends = self.specials + self.LWS + self.CR
        # Note that RFC 2822 now specifies `.' as obs-phrase, meaning that it
        # is obsolete syntax.  RFC 2822 requires that we recognize obsolete
        # syntax, so allow dots in phrases.
        self.phraseends = self.atomends.replace('.', '')
        self.field = field
        self.commentlist = []
Test tokenization of the TGrep2 manual example patterns.
dataset/ETHPy150Open deanhiller/databus/webapp/play1.3.x/python/Lib/email/_parseaddr.py/AddrlistClass.__init__--crossover with--dataset/ETHPy150Open nltk/nltk/nltk/test/unit/test_tgrep.py/TestSequenceFunctions.test_tokenize_examples