wrong decoder_input_ids[:,0] for MarianMT models ?
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.3.1
- Platform: Linux
- Python version: 3.7.0
- PyTorch version (GPU?): 1.6.0
- Using GPU in script?: Yes

### Who can help: @sshleifer 

## Information

Model I am using (Bert, XLNet ...): MarianMTModel

The problem arises when using:
* [x] the official example scripts: (give details below)

```from transformers import MarianTokenizer
tok = MarianTokenizer.from_pretrained('Helsinki-NLP/opus-mt-en-de')
src_texts = [ "I am a small frog.", "Tom asked his teacher for advice."]
tgt_texts = ["Ich bin ein kleiner Frosch.", "Tom bat seinen Lehrer um Rat."]  # optional
batch_enc: BatchEncoding = tok.prepare_seq2seq_batch(src_texts, tgt_texts=tgt_texts)
# model(**batch) should work
```

model(**batch)  doesn't work as intended because [shift_tokens_right](https://github.com/huggingface/transformers/blob/03ec02a667d5ed3075ea65b9f89ef7135e97f6b4/src/transformers/modeling_bart.py#L226) adds eos token to generate the target sequence.

```
shift_tokens_right(batch["labels"], model.config.pad_token_id)
```
returns
```
[[0,   105,   495,    53,  5324, 17279,   649,     3],
 [0,  2136,  8818,   715,  5832,    91,   688,     3]]
```
instead of
```
[[58100,   105,   495,    53,  5324, 17279,   649,     3],
 [58100,  2136,  8818,   715,  5832,    91,   688,     3]]
```
Here, "58100" is the decoder_start_token_id.
This is the intended behavior.
It is very counterintuitive to me as well, but changing `prepare_seq2seq_batch` or `shift_tokens_right` to end up with `decoder_start_token_id` at the 0th position of `decoder_input_ids` seems to lead to worse fine-tuning performance.

If you have evidence to the contrary, I would be happy to change it.<|||||>I'm running the experiment now on wmt-en-ro, we'll see how it goes!<|||||>Thanks! I was using the pre-trained model for scoring (src, tgt) pairs and didn't actually get a chance to check the impact on finetuning yet. <|||||>I ran this last night, and finetuning loss was identical with and without the change.
Bleu was within 0.1 (`master` was slightly higher).
Here is the branch if you want to play with it: https://github.com/sshleifer/transformers_fork/tree/hack-batches-v2