A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning

We introduce a novel technique for knowledge transfer, where knowledge from a pretrained deep neural network (DNN) is distilled and transferred to another DNN. As the DNN performs a mapping from the input space to the output space through many layers sequentially, we define the distilled knowledge t... View More