Harvard University; The Swiss AI Lab IDSIA, USI & SUPSI; The Swiss AI Lab IDSIA, USI & SUPSI, AI Initiative, KAUST
transformers;moe;mixture of experts;pkm;product key memories;approximate computation;efficient transformers;language modelling
Approximating Two-Layer Feedforward Networks for Efficient Transformers