Hi,
Does PyTorch support the second-order derivative? I am trying to run the following code:
student_parameters = [param for param in Student.parameters()]
teacher_params = [param for param in Teacher.parameters()]
psudo_labels = Teacher(inputs)
output =  Student(inputs)
err1 = torch.nn.MSELoss()(output,psudo_labels)
err1.backward(retain_graph=True, create_graph=True,inputs=student_parameters)
optimizer_student.step()
output_new =  Student(inputs2)
new_loss = torch.nn.MSELoss()(output_new, GT)
teacher_grad = grad(new_loss, teacher_params)
But I get the following Runtime error:
RuntimeError: One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior.