1FriendliAI Inc., Seoul, Korea, 2Department of Computer Science and Engineering, Seoul National University, Seoul, Korea
Taebum Kim, Hyoungjoo Kim, Gyeong-In Yu, Byung-Gon Chun
BPipe: Memory-Balanced Pipeline Parallelism for Training Large Language Models