OpenAI; Microsoft Corporation
Ge Yang, Edward Hu, Igor Babuschkin, Szymon Sidor, Xiaodong Liu, David Farhi, Nick Ryder, Jakub Pachocki, Weizhu Chen, Jianfeng Gao
hyperparameter tuning;scaling law;transformer;language model pretraining;infinite-width neural networks
Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer