cs.LG cs.DC
Jinda Jia Cong Xie Hanlin Lu Daoce Wang Hao Feng Chengming Zhang Baixi Sun Haibin Lin Zhi Zhang Xin Liu Dingwen Tao
SDP4Bit: Toward 4-bit Communication Quantization in Sharded Data Parallelism for LLM Training