Hi,
The problem is that prev_output is linked to the previous forward you did. And so if you call backward after the new forward, it will try to backward through the previous forward as well. Hence the error you see.
You can use retain_graph=True when you call .backward() to avoid this issue but then you will backward all the way to the first foward you made (and will most likely OOM quite quickly).
If you don’t want gradients to flow back into the previous forward you should use .detach() on prev_output to stop the gradient.