Shisho_Sama:

basically, here the author is using a one-hot encoded output with the shape (batch, sequence_length, features) with a normal not-one-hot encoded target tensor of shape (batchsize, sequence_length) !


For this part what I can tell you is that nn.CrossEntropyLoss() does take output as one-hot encoded and targets with indices that isn’t encoded into one-hot. Internally it converts it into one-hot encoding and computes the loss.
You can find this easily on documentation and its source code.