The gradient is input.grad
What do you understand by loss.grad?
input.grad is gradient of loss wrt input which is the cross entropy gradient.
there is no loss.grad as it is not involved in further opts
import torch

input = torch.randn(3, 5, requires_grad=True)
input.register_hook(lambda x: print(" \n input hook: ",x))
print(input)
target = torch.empty(3, dtype=torch.long).random_(5)
print(target)

criterion = torch.nn.CrossEntropyLoss()
criterion.requires_grad = True
loss0 = criterion(input,target)
loss0.register_hook(lambda x: print(" \n loss0 hook: ",x))
loss1 = 1*loss0
print("before backward loss0.grad :",loss0.grad)
print("loss0 :",loss0)
loss1.backward()
print("after backward loss0.grad :",loss0.grad)

what you can do is to register a backward hook in criterion rather than inspecting tensors or just do the identity op multiplying by 1 to see each term grad