Calculating Confidence score for Question Answering Models
For QA task (extractive QA) the pipeline provides/returns 4 values. 
1. a probability score
2.  start index
3. end index
4. extracted answer
(https://huggingface.co/transformers/main_classes/pipelines.html#transformers.QuestionAnsweringPipeline)

But If I am using the model class directly   (not using pipeline) like the code below then i am unable to find the probability score :

`  
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
model = AutoModelForQuestionAnswering.from_pretrained(MODEL_NAME,return_dict=True)
_encoding = tokenizer.encode_plus(question, docText, return_tensors="pt",max_length=4096)
input_ids = encoding["input_ids"]
attention_mask = encoding["attention_mask"]

start_scores, end_scores = model(input_ids, attention_mask=attention_mask).values()
all_tokens = tokenizer.convert_ids_to_tokens(input_ids[0].tolist())
answer_tokens = all_tokens[torch.argmax(start_scores) :torch.argmax(end_scores)+1]
answer = tokenizer.decode(tokenizer.convert_tokens_to_ids(answer_tokens))_
`
This answer object has only 3 values. extracted answer, start score and end score.I cant find any way or function to calculate the probability score. I want it because I want to sort multiple answers according to their probability/confidence score. 

This is a duplicate of Issue #5768 but that issue has been marked close without any answer.
This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.<|||||>model_name = "distilbert-base-uncased-finetuned-sst-2-english"

model = AutoModelForSequenceClassification.from_pretrained(model_name)

tokenizer = AutoTokenizer.from_pretrained(model_name)


train_encoding = tokenizer(X_train, truncation=True, padding=True, max_length=512, return_tensors="pt")


# Training the model with PyTorch


with torch.no_grad():

    outputs = model(**train_encoding)
    
    # Normalize logits and spans 
    start = F.softmax(outputs.start_logits, dim=1)
    end = F.softmax(outputs.end_logits, dim=1)


    # Getting the start and end index of the tensor to retrieve the answer
    start_index = torch.argmax((start), dim=1)
    end_index = torch.argmax((end), dim=1)



# Computing score

outer = np.matmul(np.expand_dims(start, -1), np.expand_dims(end, 1))
max_answer_len = 512
candidates = np.tril(np.triu(outer), max_answer_len - 1)
idx_sorts = [np.argmax(candidates[i].flatten()) for i in range(len(candidates))]
scores = [candidates[[i], start_indices[i], end_indices[i]][0] for i in range(len(candidates))]


# Extracting answer
tokenizer.decode(train_encoding["input_ids"][start_index:end_index], skip_special_tokens=True)



    
