Why does BART generate additional EOS token at the beginning?
## Environment info

- `transformers` version: 4.18.0.dev0
- Platform: Linux-5.11.0-1018-gcp-x86_64-with-glibc2.31
- Python version: 3.10.4
- Huggingface_hub version: 0.4.0
- PyTorch version (GPU?): 1.11.0+cu102 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): 0.4.1 (cpu)
- Jax version: 0.3.4
- JaxLib version: 0.3.2
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help

@patil-suraj @patrickvonplaten @sgugger

## Information

Model I am using: BART

## To reproduce

Steps to reproduce the behavior:

```python
from transformers import BartTokenizer, BartForConditionalGeneration

model_name = 'facebook/bart-base'
tokenizer = BartTokenizer.from_pretrained(model_name)
model = BartForConditionalGeneration.from_pretrained(model_name)

sentences = ('A flower.', 'Some good sentences.')

inputs = tokenizer(sentences, return_tensors='pt', max_length=8, padding='max_length', truncation=True)
output = model.generate(inputs.input_ids)

print('Input:', tokenizer.batch_decode(inputs.input_ids))
print('Output:', tokenizer.batch_decode(output))

print('Input IDs:', inputs.input_ids.tolist())
print('Output IDs:', output.tolist())
```

Output:

```
Input: ['<s>A flower.</s><pad><pad><pad>', '<s>Some good sentences.</s><pad><pad>']
Output: ['</s><s>A flower.</s><pad>', '</s><s>Some good sentences.</s>']
Input IDs: [[0, 250, 14214, 4, 2, 1, 1, 1], [0, 6323, 205, 11305, 4, 2, 1, 1]]
Output IDs: [[2, 0, 250, 14214, 4, 2, 1], [2, 0, 6323, 205, 11305, 4, 2]]
```

## Expected behavior

From my understanding, the model should be generating in this way:

![](https://user-images.githubusercontent.com/68557794/161550447-dbad465f-ac7d-4530-9812-e112c8109ac7.png)

```
250, 14214,     4, 2
  ↑      ↑      ↑  ↑
  0,   250, 14214, 4
```

So the first item of the output should be:

```
[250, 14214, 4, 2, 1, 1, 1, ...]
```

But actually it is:

```
[2, 0, 250, 14214, 4, 2, 1]
```

Therefore, I would like to know where the 2 (i.e. the EOS token) comes from?
This is because the way the model are trained. In BART `eos` (`</s>`) token is used as the `decoder_start_token_id`. So the format for the `decoder_input_ids` during training is `</s> <s> tokens ...`. So the model always needs to generate `<s><s>` as the first two tokens.<|||||>@patil-suraj Thank you! I am still a little confused. If the `decoder_input_ids` during training is `</s> <s> a b c ...`, according to my understanding, the model output should be `<s> a b c ...` (`</s> -> <s>`, `<s> -> a`, `a -> b`, `b -> c`, ...). But the model output is actually `</s> <s> a b c ...`.<|||||>I think I understand it now. We always add the decoder output to the input, and in the end we take the input as the final result.