Hi @BramVanroy,
I am using T5ForConditionalGeneration. The batch size is 8, and the vocabulary size is 32128.
What do you mean by a final layer? Like a softmax? I assumed that CrossEntropyLoss did that internally.
FYI, when I predict the probs for the test examples I use the following code:
logits = logits.squeeze(1)
selected_logits = logits[:, [12153, 2024]] 
print(selected_logits)
probs = torch.nn.Softmax(selected_logits)

Do you mean something like this?
Marcos