Questions Answering Example
Hello folks! Can you provide simple example how to use pytorch bert with pretrained model for questions answering?
Hi!

You can check this file that implements question answering on the SQuAD dataset: https://github.com/huggingface/pytorch-pretrained-BERT/blob/master/examples/run_squad.py<|||||>Thank you very much!<|||||>How can we use pre trained BertForQuestionAnswering model? I have looked into BertForNextSentencePrediction and output of model makes sense given the input vector, but unable to find any good example on BertForQuestionAnswering.<|||||>Have you tried looking at the official [documentation](https://huggingface.co/pytorch-transformers/model_doc/bert.html#bertforquestionanswering) that provides a simple example for each model?<|||||>Hi @LysandreJik, the official example was not clear to me. I understood the part of encoding. But I am looking for something like, I will give a question and a paragraph which would contain the answer, and I need the model to predict the answer span. But in the example they have done it with a single sentence, which is quite confusing!<|||||>Hey @LysandreJik, Sorry my bad, didn't look at run_squad.py, it has been changed a lot since I saw it first during which BERT was only released! It is so good to see everything being integrated at a single place! Thanks for the great work you guys! ❤️ <|||||>@Arjunsankarlal Glad you could get what you were looking for!<|||||>@Arjunsankarlal @LysandreJik can you guys help me with the example. I got an error when I ran the example given in the [documentation](https://huggingface.co/pytorch-transformers/model_doc/bert.html#bertforquestionanswering) when encoding the sequence, that tokernizer doesn't have attribute "encode". So I updated the code as follows:

`from pytorch_pretrained_bert import BertTokenizer, BertForQuestionAnswering
import torch
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForQuestionAnswering.from_pretrained('bert-base-uncased')

tokenized_text = tokenizer.tokenize("Hello, my dog is cute")


indexed_tokens = tokenizer.convert_tokens_to_ids(tokenized_text)

input_ids = torch.tensor([indexed_tokens]) # Batch size 1
start_positions = torch.tensor([1])
end_positions = torch.tensor([3])
outputs = model(input_ids, start_positions=start_positions, end_positions=end_positions)
print(outputs)`

this is the output
tensor(1.7739, grad_fn=<DivBackward0>)

I believe it's a loss but I don't understand the example as in how does it answer the question. Also there isn't any start and end span. Can you please explain the example. Much appreciated.<|||||>Hi @adilmukhtar82 , could you give a look at the [`run_squad.py` example](https://github.com/huggingface/pytorch-transformers/blob/master/examples/run_squad.py), it shows how to use several models to do question answering.

You should probably update your repository version to `pytorch-transformers` too, most of the examples on our documentation won't work with `pytorch_pretrained_bert`.<|||||>@LysandreJik Thanks I have updated the repository and example is working fine. I am confused about the example mentioned in [documentation](https://huggingface.co/pytorch-transformers/model_doc/bert.html#bertforquestionanswering) ("hello, my dog is cute") as to how does it do with single sentence and not paragraph along with it. <|||||>A bit late but here you go - 
```
from transformers import DistilBertTokenizer, DistilBertForQuestionAnswering
import torch

tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased',return_token_type_ids = True)
model = DistilBertForQuestionAnswering.from_pretrained('distilbert-base-uncased-distilled-squad')

context = "The US has passed the peak on new coronavirus cases, President Donald Trump said and predicted that some states would reopen this month.The US has over 637,000 confirmed Covid-19 cases and over 30,826 deaths, the highest for any country in the world."
question = "What was President Donald Trump's prediction?"
encoding = tokenizer.encode_plus(question, context)

input_ids, attention_mask = encoding["input_ids"], encoding["attention_mask"]
start_scores, end_scores = model(torch.tensor([input_ids]), attention_mask=torch.tensor([attention_mask]))

ans_tokens = input_ids[torch.argmax(start_scores) : torch.argmax(end_scores)+1]
answer_tokens = tokenizer.convert_ids_to_tokens(ans_tokens , skip_special_tokens=True)

all_tokens = tokenizer.convert_ids_to_tokens(input_ids)

print ("\nAnswer Tokens: ")
print (answer_tokens)

answer_tokens_to_string = tokenizer.convert_tokens_to_string(answer_tokens)

print ("\nFinal Answer : ")
print (answer_tokens_to_string)

```

Output is : 
Answer Tokens:
['some', 'states', 'would', 're', '##open', 'this', 'month']

Final Answer :
some states would reopen this month
<|||||>@ramsrigouthamg Hey, could you maybe also provide a tensorflow example?<|||||>Thanks @ramsrigouthamg !

@mariusjohan there are PyTorch and TensorFlow examples in the [usage](https://huggingface.co/transformers/usage.html#extractive-question-answering) section of the documentation.<|||||>@LysandreJik The link is now updated to https://github.com/huggingface/transformers/blob/master/examples/question-answering/run_squad.py<|||||>@ramsrigouthamg  @LysandreJik 
Examples mentioned by you is giving following error:

```
Traceback (most recent call last):
  File "run_predict.py", line 30, in <module>
    answer_start_scores
TypeError: argmax(): argument 'input' (position 1) must be Tensor, not str

```
After doing investigation, I found that the following code is returning strings instead of integer indices:

@ramsrigouthamg 
start_scores, end_scores = model(torch.tensor([input_ids]), attention_mask=torch.tensor([attention_mask]))

@LysandreJik 
answer_start_scores, answer_end_scores = model(**inputs)

Values returned:
answer_start_scores = 'answer_start_scores'
answer_end_scores  = 'answer_end_scores '
start_scores = 'start_scores'
end_scores  = 'end_scores'

I have fine tuned bert-en-base model on squad v1.1 and want to write prediction code. Can you please help?<|||||>@saurabhhssaurabh I found the solution.
You just need to change  `answer_start_scores, answer_end_scores = model(**inputs)` to either 
`answer_start_scores, answer_end_scores = model(**inputs).values()` or `answer_start_scores, answer_end_scores = model(**inputs, return_dicts=True)`

I got it from here: https://stackoverflow.com/questions/64901831/huggingface-transformer-model-returns-string-instead-of-logits<|||||>Thank you, @JacobLoe 