You’re welcome! It can be a bit tricky to wrap your head around what’s happening at first under the hood, but very simply:

autograd creates a graph of all your model’s operations
when you call backward, it traverse this graph and collects gradients (using backpropagation 2). once it reaches the end of the graph, it is released from memory. that’s why you get the error on the second backward() call
when you call step(), every parameter is updated by the gradients collected in the backward() pass.

More detail on how autograd works 3