Why is distillbart-cnn done with no teacher and distilbart-xsum has a teacher?
@sshleifer Can you expand on why distillbart-xsum is done with a teacher and distillbart-cnn is not?


It's purely empirical. `distilbart-xsum` variants perform about 1-2 ROUGE pts worse without a teacher, the gap is basically 0 for the `distilbart-cnn` variants.  For translation, it seems like teacher also helps a bit.
<|||||>In the future, you can tag me on discussion questions on discuss.huggingface.co !<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
