distributed training;model-parallel training;model parallelism;fault-tolerant training;communication efficiency;volunteer computing
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient