I think I can see what’s happening now. I was a bit confused about how NLLLoss works. The calculation below shows that applying the negative log likelihood to an input processed through softmax produces the same result as running the input through log_softmax first, then just multiplying by -1. It also shows that applying CrossEntropyLoss to the raw_input is the same as applying NLLLoss to log_softmax(input). I’m guessing that the log_softmax approach may be more numerically stable than using softmax first, and calculating the log of the result separately.
>>> raw_input = torch.tensor([[0.7, 0.26, 0.04]])
>>> softmax_input = torch.softmax(raw_input, dim=1)
>>> softmax_input
tensor([[0.4628, 0.2980, 0.2392]])
>>> -torch.log(softmax_input)
tensor([[0.7705, 1.2105, 1.4305]])
>>> log_softmax_input = torch.log_softmax(raw_input, dim=1)
>>> log_softmax_input * -1
tensor([[0.7705, 1.2105, 1.4305]])
>>> loss = nn.NLLLoss()
>>> loss(log_softmax_input, torch.tensor([0]))
tensor(0.7705)
>>> cross_entropy_loss = nn.CrossEntropyLoss()
>>> cross_entropy_loss(raw_input, torch.tensor([0]))
tensor(0.7705)