dpernes:

Is this the current behavior for BART?


Yes, this is to mimic other models such as BART and T5, which also automatically create the decoder_input_ids based on the labels.



 dpernes:

If no decoder_input_ids is provided, the model will create this tensor by shifting the input_ids to the right for denoising pre-training following the paper. (BART — transformers 4.11.3 documentation)


That’s actually a mistake in the documentation, it should be “by shifting the labels” instead of “by shifting the input_ids”. Can you open a PR to fix this?
And regarding the loss difference: that’s actually exactly what we want, right  we would like to get the exact same loss if users don’t provide decoder_input_ids themselves, but only labels. In that case, the model should create exactly the decoder_input_ids as the ones you provided. Seems like the implementation is correct