Stanford University, USA; Carnegie Mellon University; ETH Zürich, Switzerland
Jue WANG, Binhang Yuan, Luka Rimanic, Yongjun He, Tri Dao, Beidi Chen, Christopher Ré, Ce Zhang
Communication compression;Quantization;SGD
Fine-tuning Language Models over Slow Networks using Activation Quantization with Guarantees