distributed training;model-parallel;model parallelism;pipeline;fault tolerance;communication efficiency;volunteer computing
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient