    def num_tokens_vec(self, indices):
        """Return the number of tokens for a set of positions defined by indices.
        This value is used to enforce ``--max-tokens`` during batching."""
        sizes = self.src_sizes[indices]
        if self.tgt_sizes is not None:
            sizes = np.maximum(sizes, self.tgt_sizes[indices])
        return sizes
Return the number of tokens for a set of positions defined by indices.
This value is used to enforce ``--max-tokens`` during batching.
fairseq/data/language_pair_dataset.py