Google; Massachusetts Institute of Technology
Hussein Hazimeh, Zhe Zhao, Aakanksha Chowdhery, Maheswaran Sathiamoorthy, Yihua Chen, Rahul Mazumder, Lichan Hong, Ed Chi
Mixture of Experts;Sparse Mixture of Experts;Sparsity;Sparse Gate;Subset Selection;Multi-task Learning
DSelect-k: Differentiable Selection in the Mixture of Experts with Applications to Multi-Task Learning