Thanks, I think this is what I was searching for
Still, I have some silly doubts (I am still learning)


For the optimizer part, I am using 2 optimizers, so should I use optimizer.step() and optimizer.zero_grad() after calling loss.backward() for each model or do it in the end.


When we do this input.requires_grad_() what’s the difference between input.requires_grad = True


like suppose in the 2nd epoch, if the input.grad is still on, so if we find grad again wont it be grad of grads .


Can you explain more why we are doing this


if input.grad is not None:
    with torch.no_grad():
        input.grad.zero_()

Thanks, I am very grateful.