    def build_tokenizer(self, args):
        """Build the pre-tokenizer for this task."""
        return encoders.build_tokenizer(args)
Build the pre-tokenizer for this task.
fairseq/tasks/fairseq_task.py