Trainer bug in BertForQuestionAnswering when Pretrained model wasn't trained on NPS
## Environment info
 `transformers` version: 4.12.2
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.8.5
- PyTorch version (GPU?): 1.10.0+cpu (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no
### Who can help
not sure, it more to report on an unintuitive behaviour 
@LysandreJik (not sure its only in BERT or all across the othe model - really depends how they were trained )
- Trainer: @sgugger

The problem arises when using:
my own modified scripts: 
```
#some data prep...

BASE_MODEL = 'onlplab/alephbert-base'
train_dataset = SquadDataset(train_encodings)
val_dataset = SquadDataset(val_encodings)
model = BertForQuestionAnswering.from_pretrained(BASE_MODEL)

trainer = Trainer(
    model=model,                       
    args=training_args,             
    train_dataset=train_dataset,         
    eval_dataset=val_dataset           
)
```
But it wouldnt happen when I replace the bert model to  `'bert-base-cased'` for example or any othe rbert model which was trained on NSP  or specifically in its config as the `  "type_vocab_size": **2**` (and not 1)

The error you get is `Embedding Error Index out of Range in self` when you give `1` in some of indices in the `token_type_ids `embedding input. 

----

the unintuitive part that it doesn't happen on the same input, when replacing the `Trainer` code with the traditional training script - when it only give `0` to all `token_type_ids` which seem to be a bug in the `BertForQuestionAnswering` or at list it is very un intuitive especially today when people some time don't use this task when training Bert and use Work Masking.

The following script work when giving  `0` to all `token_type_ids` :
```
from torch.utils.data import DataLoader
from transformers import AdamW

device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')

model.to(device)
model.train()

train_loader = DataLoader(train_dataset, batch_size=1, shuffle=True)

optim = AdamW(model.parameters(), lr=5e-5)

for epoch in range(3):
    for batch in tqdm(train_loader):
        optim.zero_grad()
        input_ids = batch['input_ids'].to(device)
        attention_mask = batch['attention_mask'].to(device)
        start_positions = batch['start_positions'].to(device)
        end_positions = batch['end_positions'].to(device)
        outputs = model(input_ids, attention_mask=attention_mask, start_positions=start_positions, end_positions=end_positions)
        loss = outputs[0]
        loss.backward()
        optim.step()

model.eval()

```

## To reproduce

Steps to reproduce the behavior:

1.train any QA (squad like) dataset on Bert using   a model with "type_vocab_size": **1**`
2. fail to tain using the first snippet (with the trainer) 
3. succeed training using the "classic" snippet

## Expected behavior

mentioned above...
If you just drop the `token_type_ids` key from your dataset, or replace them all by zeros, then the training will succeed, so this is not a bug in the `Trainer`, just a mistake in the processing of the dataset.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.