@yjernite @patrickvonplaten @valhalla:
what are good kwargs to get 10 diverse summaries from bart? The top 10 beams are all > 98 ROUGE against each other. (aka barely different).
I am working a bit on pseudolabeling and have gotten huge gains from using the following strategy:

generate best summary with default bart parameters
if rouge(generated, train label) > 0.25: add (src example, generated example to dataset)

I modified the strategy a bit to consider all top 10 generations (adding at most 1 for each example), and that also works well, though I haven’t been controlled about whether it works better.
What I do know is that if you blindly add the pseudolabel to the dataset (remove bullet 2), it hurts performance.
Also if there is a systematic study on this stuff I would definitely read it!