University of Nevada, Reno; Microsoft
Heyang Qin, Samyam Rajbhandari, Olatunji Ruwase, Feng Yan, Lei Yang, Yuxiong He
adaptive batching;large batch size;gradient similarity measure;gradient variance;gradient noise;large scale training
SimiGrad: Fine-Grained Adaptive Batching for Large Scale Training using Gradient Similarity Measurement