School of Informatics, University of Edinburgh; Centre for Artificial Intelligence, University College London
Jean Kaddour, Oscar Key, Piotr Nawrot, Pasquale Minervini, Matt Kusner
language models;transformers;efficient training
No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models