FPT Software AI Center; Department of Statistics and Data Sciences, University of Texas at Austin; Department of ECE, Rice University; Department of Mathematics, University of California, Los Angeles
Tan Nguyen, Tam Nguyen, Hai Do, Khai Nguyen, Vishwanath Saragadam, Minh Pham, Khuong Duy Nguyen, Nhat Ho, Stanley Osher
transformer;admixture;attentions;redundant heads
Improving Transformer with an Admixture of Attention Heads