If I understand your case, you want something like:
input = # Some input
output = net(input)
loss = crit(output, labels)

The think the that loss.backward will update the weights in the network based on the loss value. If you want gradients of just output wrt inputs, then you only want the influence of output wrt to input and not the other weights in the network.
This means that you cannot do both in one single backward pass as you want in one case to consider the influence on the weights (for the loss) and you don’t want it on the other case (for the grad op).
Another point is that this autograd,grad call won’t give you the full jacobian of the network but the vector jacobian product with a vector of all ones. Or you can see it as the full jacobian of the function that does net(input).sum().