Hi,
I’ve currently fine-tuned this 1 Question-Answering model to fit a specific business use case we have (identifying the name of a company from a piece of text). When it comes to inference, I’ve found as @sgugger has very clearly explained in this notebook 1 that sometimes the best answer isn’t the one with the best start and end logits as sometimes the highest scoring combination can produce an answer that is too long or too short (just one character).
As such when I was predicting using this model locally I created a return_best_combinaton function that finds the most practical answer using the list of logit scores.
When I used this model using the SageMaker API I realised it just returns one single answer with a score assigned to it. I wanted to check how this answer is produced? (happy to just be directed to the source code if it’s available) and wether it’s possible to return n number of likely answers instead of just 1.
Thanks every so much,
Karim