Hi,
thanks for the prompt and thorough reply, it’s a lot clearer now  I have a few follow-up questions though:


So that basically means that y = t @ a creates a graph and each iteration in the loop extends this graph by attaching to it? Consequently when the variables in the loop are re-assigned (x_out and loss) then the previously attached part is removed again, since nothing refers to it anymore? Is that interpretation correct?


Then the keyword argument retain_graph actually only refers to the buffers (of the graph’s vertices) that are needed for performing the backpropagation? But the actual graph (i.e. the relations between tensors) are retained in any way (no matter the value of the retain_graph argument)? In other topics I read that it was called retain_variables before, why the change to retain_graph (retain_buffers would be clearer maybe)? At least to me it was confusing until I read your reply.


The following statements are not completely clear to me:



The graph contains both the functions that need to be applied to perform the backward pass […]
[…] each function is applied one after the other

What are these functions actually (are they related to the .grad_fn attribute)? And how are the backprop buffers related to them?