    def zero_grad(self):
        """Clears the gradients of all optimized parameters."""
        for p in self.params:
            p.grad = None
        self.optimizer.zero_grad()
Clears the gradients of all optimized parameters.
fairseq/optim/fairseq_optimizer.py