So, in that case, what I have mentioned about requiring  is correct, right? In which case I either need to use higher package as you mentioned or the other hack that I mentioned above I suppose.
But other than that, does the code snippet that I have put up make sense? My main confusion is about whether or not I have used retain_graph=True correctly. So, for instance, since I want to compute the loss sensitivity every k1 steps, I am detaching the older x's and keeping requires_grad=True only for the last k1 x's.