School of Information Science and Engineering, Yunnan University, Kunming, China
decentralized learning;federated learning;federated distillation;pre-trained language model
FedID: Federated Interactive Distillation for Large-Scale Pretraining Language Models