When you do ipt[:,2].unsqueeze(1) it creates a new Tensor that is a slice of ipt. Bu that new Tensor is not used to compute the loss and that’s why you see this error.

Is there a way to calculate derivatives with respect to just one input if I don’t need all of the higher order derivatives?

You can only compute derivatives wrt to a Tensor. So if all your inputs are in a single Tensor, you have to compute the derivative for the whole thing. But this kind of batch computations are usually not much more expensive than doing a single element.