I am missing something here. if we zero out current gradient value with optimizer.zero_grad() then how does update happen to weights? For an update to happen we should have values in weight so we can do something like weight-=lr*grad?
Thanks.