Huawei Noah’s Ark Lab.; School of Computer Science, Faculty of Engineering, The University of Sydney.; School of information and Electronics, Beijing Institute of Technology.
Zhiwei Hao, Jianyuan Guo, Kai Han, Han Hu, Chang Xu, Yunhe Wang
knowledge distillation;small-data pitfall;vanilla kd
Revisit the Power of Vanilla Knowledge Distillation: from Small Scale to Large Scale