Georgia Institute of Technology; Microsoft Azure AI; Amazon; Microsoft Research
Chen Liang, Haoming Jiang, Simiao Zuo, Xz W, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Tuo Zhao
Training Large Transformer Models;Reducing Model Redundancy;Parameter Sensitivity;Adaptive Learning Rate Method;Model Generalization;Model Pruning
No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models