Harvard University∗; UC Berkeley∗; UCLA∗; Google Research
mixture-of-experts;transformer;router;efficiency;conditional compute;sparsely activated models;theory
On the Benefits of Learning to Route in Mixture-of-Experts Models