    def no_sync(self):
        """A context manager to disable gradient synchronization."""
        old_accumulate_grads = self.accumulate_grads
        self.accumulate_grads = True
        yield
        self.accumulate_grads = old_accumulate_grads
A context manager to disable gradient synchronization.
fairseq/distributed/legacy_distributed_data_parallel.py