Stanford University, USA; Carnegie Mellon University; ETH Zürich, Switzerland
Binhang Yuan, Yongjun He, Jared Davis, Tianyi Zhang, Tri Dao, Beidi Chen, Percy Liang, Christopher Ré, Ce Zhang
Decentralized training;Foundation model.
Decentralized Training of Foundation Models in Heterogeneous Environments