I’m not sure I understand the use case completely, so please correct me if I’m wrong.
retain_graph is used to keep the computation graph in case you would like to call backward using this graph again.
A typical use case would be multiple losses, where the second backward call still needs the intermediate tensors to compute the gradients.



 Harman_Singh:

simply because I need all the gradients of previous tensors in my code.


Each backward call computes the gradients for the parameters. If you don’t zero out the gradients, each following backward will accumulate the gradients in all parameters.
So I’m not sure, if you really need to retain the graph or if you just want to accumulate the grads.