Microsoft Research; Penn State University; NC State University
Dongkuan (DK) Xu, Subhabrata Mukherjee, Xiaodong Liu, Debadeepta Dey, Wenhui Wang, Xiang Zhang, Ahmed Awadallah, Jianfeng Gao
Pre-trained Language Models;Knowledge Distillation;Neural Architecture Search
Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models