Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning
Scalable training of deep learning machines by incremental block training with intra-block parallel optimization and blockwise model-update filtering