Georgia Institute of Technology; Amazon
Chen Liang, Haoming Jiang, Zheng Li, Xianfeng Tang, Bing Yin, Tuo Zhao
Knowledge Distillation;Structured Pruning;Pre-trained Transformer Language Models;Model Compression
HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers