Hello All,
I have just started learning this awesome tool called PyTorch but sadly I am stuck in an equivocal situation.
Below is a code snippet from one of the tutorials
 with torch.no_grad():
       weights -= weights.grad * lr
       bias -= bias.grad * lr
       weights.grad.zero_()
       bias.grad.zero_()

I am kind of confused that even if I will do parameter update without using torch.no_grad() ( i.e. only in-place ) and since the backward call has already been made in the code above this snippet( not included in the snippet above ) which basically means all the “grad” attributes are already computed and don’t require the “original” values again. Then, why is it illegal to do those operations without torch.no_grad()?
I know it will flag off the error in PyTorch but I just wanted to know where my line of thought is at fault?