Hi,
By default pytorch will differentiate your whole program.
If there are parts where the gradients should not flow back, you should make it explicit by doing t_without_grad_tracking = t.detach().
Here in particular, if you teacher uses the output of your student’s net for training, but the gradients should not flow back to the student’s net (because you’re not training it), then you should .detach() the output of the student net before passing it to the teacher net.