Department of Computer Science, Stanford University; Department of Computer Science and Engineering, University at Buffalo, SUNY; Department of Psychology, Stanford University
Dan Fu, Simran Arora, Jessica Grogan, Isys Johnson, Evan Sabri Eyuboglu, Armin Thomas, Benjamin Spector, Michael Poli, Atri Rudra, Christopher Ré
structured matrices;transformers;efficiency
Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture