valhalla:

After modifying the adjust_logits_during_generation BLUE-4 score for my model went from 13.09 to 19.14. So this indeed is an issue.


In the config of  your finetuned model, what was decoder_start_token_id?
In your training code, what did a batch “look like”?