Department of Computer Science, Virginia Commonwealth University, Richmond, VA; Dept. of Electrical and Computer Engineering, University of California, Santa Barbara, CA; Department of Computer Science, Virginia Commonwealth University, Richmond, VA; C3 AI, Redwood City, CA; Department of Computer Science, Virginia Commonwealth University, Richmond, VA
Aliakbar Panahi, Seyran Saeedi, Tom Arodz
Transformer;matrix factorization;Kronecker product;compact models
Shapeshifter: a Parameter-efficient Transformer using Factorized Reshaped Matrices