AAII, University of Technology Sydney; Institute for Artificial Intelligence, Peking University; Wangxuan Institute of Computer Technology, Peking University; School of Intelligence Science and Technology, Peking University; Microsoft Corporation
knowledge distillation;language model compression;token pruning
Length-Adaptive Distillation: Customizing Small Language Model for Dynamic Token Pruning