Why does the BertForQuestionAnswering sample code duplicate the [CLS] token?
## ❓ Questions & Help

<!-- A clear and concise description of the question. -->
The BertForQuestionAnswering sample code creates duplicate [CLS] tokens. Wondering why:

```
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForQuestionAnswering.from_pretrained('bert-large-uncased-whole-word-masking-finetuned-squad')
question, text = "Who was Jim Henson?", "Jim Henson was a nice puppet"
input_text = "[CLS] " + question + " [SEP] " + text + " [SEP]"
input_ids = tokenizer.encode(input_text)
token_type_ids = [0 if i <= input_ids.index(102) else 1 for i in range(len(input_ids))]
start_scores, end_scores = model(torch.tensor([input_ids]), token_type_ids=torch.tensor([token_type_ids]))
all_tokens = tokenizer.convert_ids_to_tokens(input_ids)
print(' '.join(all_tokens[torch.argmax(start_scores) : torch.argmax(end_scores)+1]))
# a nice puppet
tokenizer.decode(input_ids)
#'[CLS] [CLS] who was jim henson? [SEP] jim henson was a nice puppet [SEP] [SEP]'  
```

If I remove the extra [CLS], the extraction doesn't work. It's exactly two tokens off:
```
input_ids = tokenizer.encode(input_text, add_special_tokens=False)
...rerun same code as above...
print(' '.join(all_tokens[torch.argmax(start_scores) : torch.argmax(end_scores)+1]))
# was a
```
What am I doing wrong? How can I get the extraction working without duplicate [CLS] tokens? (and duplicate final [SEP] tokens BTW).

The sample code comes right from the docs: https://huggingface.co/transformers/model_doc/bert.html#bertforquestionanswering
Indeed, this is a mistake, thank you for raising an issue. It should have been fixed with 74755c89b92e0c0c027221c13fd034afed4d2136.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
