Hello I have a model call M. when I call it and call backward() to calculate the gradient and then I zero the gradient the model has no gradient. but once I call optimizer.step() on zero gradient the model update its parameters on something I don’t know so the next iteration the model has new params.
output = M(input,GT)
error = L1(output,GT)
error.backward()
print(M.grad) —> non-zero.
M.zero_grad()
print(M.grad) —> zero.
optimizer.step() -> the model is update !!!
I don’t want to use with torch.no_grad() because I want to check each iteration.