Hi,
There are many questions here, I’ll try and answer in the order you asked them even if they overlap.
Yes the error is because here the operation t @ a is reused multiple times. I am not sure what you mean by " loss.backward() should detach the graph". backward does not change the graph, it just traverses it (and free un necessary buffers if retain_graph is not set).
In you case, the graph up to y is built once. Then each iteration extend this graph. The problem is that the first backward will free even the common part of the graph and so the second one will fail.
“I would expect the whole graph to be rebuilt on every iteration”: keep in mind that the computational graph is built ONLY buy doing forward operations in your python script. The backend will never change/built it.
In you case you indeed can use retain_graph=True to keep the whole graph during the backward. And the “lower” part (that was created inside the loop) will be freed when the loop exit. The upper part (t @ a) will still remain as the y variable still refers to it.
To free the upper part after the loop, simply delete everything that references it: del y in your case.
The graph is created every time your perform an operation on tensors (when gradient mode is enabled).
The graph contains both the functions that need to be applied to perform the backward pass and the buffers and temporary variable they need.
When you call backward, the graph is traversed in reverse order and each function is applied one after the other. After each function execution, all its buffers and temporary variables are deleted (if retain_graph=False).
If during the backward, a buffer does not exist, then the above error is raised as the only way this buffer does not exist anymore is because your already backproped.