Purdue University; University of Chicago and Together Computer; Mila and Université de Montréal; Carnegie Mellon University and Meta AI (FAIR); University of Buffalo, SUNY; The University of Tokyo; Stanford University; Vrije Universiteit Amsterdam
Stefano Massaroli, Michael Poli, Dan Fu, Hermann Kumbong, Rom Parnichkun, David Romero, Aman Timalsina, Quinn McIntyre, Beidi Chen, Atri Rudra, Ce Zhang, Ce Zhang, Christopher Ré, Stefano Ermon, Yoshua Bengio
Long convolutions;recurrence;attention;language models;signal processing;throughput;auto-regressive generation
Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions