But that does not modify the original Tensor!

Oh! I didn’t know that indexing like this returns a view. Then how do I detach those original Tensors? What modification do I need to make to detach only those particular x's?

You should only ever use it if you compute gradients on the same graph multiple times. Otherwise, there is something wrong.

This is a slightly rhetorical question but let me ask it anyway: But I do need to retain the derivatives, right? I need derivatives up to k steps in the ‘past’.