Hi folks,
Problem: fine-tuned model adopts peculiar behaviour with beam search.
Specifically, beam search outputs include 2 bos tokens and exclude the first word token. I have double checked my data feed and the inputs look fine, with one bos and the first word token is present. Even more strange, there are no such problems with nucleus sampling, which suggests that this is particular to beam search.
for example, here are two outputs from the same fine-tuned model;
beam search output: <s><s>rek likes to play football. </s>
nuleus sampling output: <s> Derek likes to go to the football pitch. </s>

Any clues?