To examine the reason behind the robustness of our method, Fig. 4 visualizes the error in the final parameter gradients used to update the model.
we take two models for CIFAR-100-at the start and end of training-and then compute gradients for a 100 batches with respect to the convolution kernels of all layers exactly, and using our approximate strategy.