    def _get_sample_prob(self, dataset_lens):
        """
        Get smoothed sampling probability by languages. This helps low resource
        languages by upsampling them.
        """
        prob = dataset_lens / dataset_lens.sum()
        smoothed_prob = prob**self.cfg.multilang_sampling_alpha
        smoothed_prob = smoothed_prob / smoothed_prob.sum()
        return smoothed_prob
Get smoothed sampling probability by languages. This helps low resource
languages by upsampling them.
fairseq/tasks/multilingual_denoising.py