You are absolutely right. That’s what I tried to do at first. However,  the T5 model has no T5ForSequenceClassification class or something similar. I think that the most suitable is to use the last logits taking into account padding. Do you have in mind any function that can be helpful?