Solved by mailcorahul in post #2 
              
              
                I suppose you are referring to Knowledge Distillation/Model compression. 
A model’s output(called teacher model in this context) is what generally referred to as soft label. It is called soft because the output may not be strictly something like [1, 0, 0] for a 3-class classification task, instead i…