My friend who wishes to remain anonymous asked a good question about T5 that I couldn’t answer:
Say we have a model that predicts sentiment – answers are “positive/negative/neutral” – for something like RoBERTa we’d add a layer, slap on a softmax – and we get both argmax predictions, and some notion of probability amongst the three classes (as well as entropy).
For T5, we just get a text reply.
But of course if we looked at the outputs to the softmax in T5, we’d see p(“positive”) etc – assuming the response is 1 token.
Has anyone tried to do this already or seen examples/notebooks like this?
Ideally, we want  to ask our model several questions. Without worrying too much about the conditional logic, we’d like to be able to measure the probability of text outputs, including some rare categories (that nonetheless are present in our training set). As well as to look for low and high entropy predictions.
If nobody has done this, any code pointers where to look would be helpful.