Deep Learning, Artificial Intelligence, Image Classification, Self Supervised Learning, Distillation.
ResNet-50×4 to ResNet-50 takes ~100 hours on four Titan-RTX GPUs while compressing from ResNet-50 to ResNet-18 takes ~90 hours on two 2080-TI GPUs. 2. Results 2.1. Distillation Comparisons on ImageNet Full ImageNet. Linear, Nearest Neighbor (NN), and Clsuter Assigment (CA) are used. The proposed