DataCollatorForLanguageModeling - Shift labels for left-to-right LM?
Hi,

Shouldn't the labels be shifted 1 step right in the data collation step at -https://github.com/huggingface/transformers/blob/09a2f40684f77e62d0fd8485fe9d2d610390453f/src/transformers/data/data_collator.py#L88

Actually never mind. At least GPT2 does the shifting internally. 