Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM
Sainbayar Sukhbaatar Olga Golovneva Vasu Sharma Hu Xu Xi Victoria Lin Baptiste Roziere Jacob Kahn Shang-Wen Li Wen-tau Yih Jason E Weston Xian Li