    def save_checkpoint(self, filename, extra_state):
        """Save all training state in a checkpoint file."""
        extra_state["rng_tracker_states"] = get_cuda_rng_tracker().get_states()
        super().save_checkpoint(filename, extra_state)
Save all training state in a checkpoint file.
fairseq/model_parallel/megatron_trainer.py