Does the concept of "confidence" apply to text generation tasks? I'm currently using the T5 model to solve a text classification task as text generation. Specifically, I'm using the T5ForConditionalGeneration model from the HuggingFace Transformers library.
What I'm wondering is, is there a way that we can use the concept of "confidence" for this kind of setting? I'm asking because some samples I'm using are either mislabeled or don't have enough information to effectively make a classification decision. I was hypothesizing that the model's "confidence" for such samples would be relatively lower, and we could use such a measure to prevent the model from making predictions on those samples or something.
I don't think that the concept of "confidence" would apply to text generation, but was wondering if there's any other way we could make such decisions. Thanks!

The statistical concept of "confidence" arises in the context of confidence intervals, which arises when we want to estimate an unknown parameter in statistical analysis.  This can certainly occur in a text classification task so long as there is some unknown parameter in your model that you want to make an inference about.
Several of the things you say in your question suggest that you are not thinking about the meaning of a confidence interval in the correct way.  It is not a model or a sample that has a level of "confidence" --- the level of confidence is something that applies to a confidence interval for an inference about a parameter in the model.  In order to proceed sensibly, you need to step back and first ask yourself: (1) what is the form of my model; (2) what are the unknown parameters in my model; (3) what is an appropriate method to form a confidence interval for one or more of the parameters of interest; (4) what level of confidence do I wish to set for the formation of a confidence interval I wish to produce.
