Distributed Large language models training;long context;sequence parallelism;recomputation;overlap communication
LightSeq: Sequence Level Parallelism for Distributed Training of Long Context Transformers