Does the model support word level confidence scores?
I am using compute_word_confidence = True but almost all words get a confidence score of 0.0. Strangely, though there are some with apparently correct scores.
Does the model support word level confidence scores at all?
This is the added functionality of an external library (I'm assuming you are using whisper-timestamps). As such it's not something we are actively able to implement, support and debug. If you identify an issue it's better to open an issue in the external library's Github repo.
Word confidences are computed from a models logit outputs. In principle there should be nothing hindering this from working for our model as well.
Yes, I am using whisper-timestamped. The reason I asked about this here is that the confidence scores worked fine with openAI's whisper v3 so I assumed that if it doesn't work with the KB model, it must be something that the (finetuned) model doesn't support (for whatever reason).
Word confidences are computed from a models logit outputs.
Thanks for providing this additional information. This will help me understand the issue better.
Just a thought (since I was using the strict model with whisper-timestamped): could it be that the way the strict model works influences the generation of its logit outputs?