Independent Researcher; AISIA Lab, University of Science, Vietnam National University Ho Chi Minh City; Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK, 38000 Grenoble, France; University of Tennessee at Chattanooga; Institute for Infocomm Research (I2R), A*STAR, Singapore; VinUniversity; Salesforce Research Asia; Singapore Management University
Sparse mixture of experts;hypernetwork;efficient training of LLMs;large language models
HyperRouter: Towards Efficient Training and Inference of Sparse Mixture of Experts