I’m not particularly familiar with BERT-based generation models but in general, .generate(input_ids, do_sample=True) is an easy way to diversify the output of the model. Bear in mind that the output text can still lack coherence but it will less likely be so repetitive. You can also experiment with the repetition_penalty argument of the generate method as explained here