can you show an example where you call generate twice with the same input_ids and model, but different kwargs, and get wierd beam search output?
I suspect the issue is in generate kwargs/decoder_start_token_id  if same model -> different outputs.