shift_tokens_right function in XforConditionalGeneration classes
Hi there,

I am using BART for summarization and I noticed an issue with the `DataCollator` class when using it with multiple batches. The class uses the `prepare_decoder_input_ids_from_labels` function to create the `decoder_input_ids` for the labels (target sequence), which shifts the tokens one to the right and appends the `decoder_start_token_id` at the start of the shifted tensor. This works well when the batch size is 1, but it seems to be buggy when using multiple batches.

For example, I observed that the second example in my batch (which is padded with -100 in the labels tensor) has two `decoder_input_ids`, one to the left and one to the right. I would like to understand if this implementation is correct, and if so, what is the reasoning behind shifting the longer example while appending the `decoder_start_token_id` to the shorter example while keeping it at the end of the target sequence?

Here's an example with `batch_size` of 2:

(Examples are truncated for better visualization and with the `decoder_input_ids`s as `**2**`)

```
pdb>> decoder_input_ids

tensor([[    **2**,     0,   133,  6741, 31275,  2633,    41, 13433,     9,     5,
          3425,  6411,     9,    10,  6063,   797,  2187,     4, 50267,    ...
             0,   133,  6063,    40,    45],

        [    **2**,     0,   133, 27913, 39322, 31275,  3373,  1437,   103, 12720,
             8,  8047,  ... ,     7,    33,    10,  2200, 12628,
         50267,     0,  2264,  8089,     5,  6063,   197,   283,    11,     4,
         50267,     **2**,     1,     1,     1]], device='cuda:0')
```

P.S.: 

Given the reasoning behind shifting operation in decoding time, the second example makes more sense to me; as the first "2" denotes the start of the sequence and makes the decoder to predict the first word of the target sequence, and the last "2" is again predicted in the second example to denote the end of the sequence. However, I do not see any second "2" in the first example. Will the decoder be able to take care of both examples? 
cc @ArthurZucker and @younesbelkada <|||||>Hey for BART, the `eos_token_id=2`, which is why you have one at the end of a sequence, and a the beginning (for training purposes). Does that make sense?
