University of Cambridge, Alan Turing Institute; Google; DeepMind; University of Cambridge; Google, University of Cambridge
Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Georgiana-Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Q Davis, Afroz Mohiuddin, Lukasz Kaiser, David Belanger, Lucy J Colwell, Adrian Weller
performer;transformer;attention;softmax;approximation;linear;bert;bidirectional;unidirectional;orthogonal;random;features;FAVOR;kernel;generalized;sparsity;reformer;linformer;protein;trembl;uniprot
Rethinking Attention with Performers