Hi, is ranked_tensor a list or a Tensor? if it’s a tensor, doing ranked_tensor[idx] creates a new Tensor from it every time you call it. This means that things that change the current Tensor (like retain_grad()) will be applied on this new Tensor. In your case the Tensor on which you call retain_grad and the one where you check the gradients are not the same.