Department of EECS, LIDS, MIT, Cambridge, MA, USA; Kim Jaechul Graduate School of AI, KAIST, Seoul, Korea; Department of EECS, UC Berkeley, Berkeley, CA, USA
David X. Wu, Chulhee Yun, Suvrit Sra
On the Training Instability of Shuffling SGD with Batch Normalization