 Knowledge distillation allows to transfer the knowledge learnt by a large model to a smaller one, that would not be able to easily learn it directly from the data itself, producing a new model that is faster to evaluate, and therefore deployable on less powerful hardware (such as a mobile device), while at the same time experiencing only a small loss of classification performance compared to the original, large model.Knowledge distillation has been successfully used in several applications of machine learning such as object detection, acoustic models, and natural language processing.What it is desired from a good model is good generalization capability, therefore transferring the knowledge from a large to a small model needs to somehow teach to the latter how to generalise in the same way as the former.