Well that’s exactly my question.
I do “optimizer_student.step()”, which uses gradients computed from “err1 = torch.nn.MSELoss()(output,psudo_labels)”, which depends on the teacher’s parameters. The reason I am getting this error is because the optimizer optimizer_student updates the parameters of the student using in-place operations and so the teachers’parameters are broken from the computation graph at that point. I wonder how I can achieve this goal.