Can someone clarify this? I have same doubt as above
This does not seem to work if the requires_grad is turned off after at least one forward/backward/step since in that case p.grad is not None anymore but 0. Or am I misunderstanding something?