    def num_tokens(self, index: int):
        """
        Return an example's length (number of tokens), used for batching. Here
        we return the max across all examples at index across all underlying
        datasets.
        """
        return max(
            dataset.num_tokens(self._map_index_to_dataset(key, index))
            for key, dataset in self.datasets.items()
        )
Return an example's length (number of tokens), used for batching. Here
we return the max across all examples at index across all underlying
datasets.
fairseq/data/multi_corpus_sampled_dataset.py