H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, U.S.A.; Microsoft, Redmond, U.S.A.
Chen Liang, Simiao Zuo, Qingru Zhang, Pengcheng He, Weizhu Chen, Tuo Zhao
Less is More: Task-aware Layer-wise Distillation for Language Model Compression