It seems you are right.
I tested it with this small example:
loss = nn.CrossEntropyLoss(reduce=False)
input = torch.randn(2, 3, 4)
input = Variable(input, requires_grad=True)
target = torch.LongTensor(2,4).random_(3)
target = Variable(target)
output = loss(input, target)

loss1 = F.cross_entropy(input[0:1, :, 0], target[0, 0])
loss2 = F.cross_entropy(input[0:1, :, 1], target[0, 1])

loss1 and loss2 give the first two elements of output, so apparently it’s working.