How to speed up getting answers?
Hi ,

I'm facing the issues using BertForQuestionAnswering ,can you please help me fixing these:

1. I'm trying to use BertForQuestionAnswering pretrained model to get answers from news,
answer_question is the function where in you pass the context and the question and get the relevant answers,but if I have 100 contexts its taking around 100 seconds to get the answer,may I please know by any chance I can get the answers in much lesser time.

2. Some times I get the answer where in start index and end index are pointing to [SEP],that means the whole context,can I avoid this.


```
BERT_SQUAD = 'bert-large-uncased-whole-word-masking-finetuned-squad'

model = BertForQuestionAnswering.from_pretrained(BERT_SQUAD)
tokenizer = BertTokenizer.from_pretrained(BERT_SQUAD)

def answer_question(question, context):
    """
    Answer questions
    """
    try:
        print("Type:",type(context))
        print(context)

        encoded_dict = tokenizer.encode_plus(
                            question, context, # Sentence to encode.
                            add_special_tokens = True, # Add '[CLS]' and '[SEP]'
                            max_length = 256,  # Pad & truncate all sentences.
                            pad_to_max_length = True,
                            return_attention_mask = True,   # Construct attn. masks.
                            return_tensors = 'pt'     # Return pytorch tensors.
                       )
        print(encoded_dict)
        input_ids = encoded_dict['input_ids'].to(torch_device)
        token_type_ids = encoded_dict['token_type_ids'].to(torch_device) # segments

        start_scores, end_scores = model(input_ids, token_type_ids=token_type_ids)
        print('Start Scores:',start_scores)
        print('End Scores:',end_scores)
        all_tokens = tokenizer.convert_ids_to_tokens(input_ids[0])
        print(all_tokens)
        answer = tokenizer.convert_tokens_to_string(all_tokens[torch.argmax(start_scores) : torch.argmax(end_scores)+1])

        answer = answer.replace('[CLS]', '')
        #answer= answer.replace('[SEP]','')
        print("Start index:",all_tokens[torch.argmax(start_scores)])
        print("End index:",all_tokens[torch.argmax(end_scores)])
        print(answer)
    except ValueError:
        print("Error in fetching answer")
        answer=''
        
    
    return answer
```

Thanks in advance!!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
