thanks, actually I’ve been digging through source code and found that forward method actually generates decoder_input_ids from labels through shift_tokens_right. Thank you for help, and sorry for being annoying, should have checked source code first