University of Passau, Germany
Knowledge Distillation;Compression;NLP;Large Language Models;Feature Distillation
Learn From One Specialized Sub-Teacher: One-to-One Mapping for Feature-Based Knowledge Distillation