Hi @Hildweig, There is no paper for distilbart, the idea of distllbart came from @sshleifer’s great mind 
You can find the details of the distillation process here 35
For the CNN models, the distiiled model is created by copying the alternating layers from  bart-large-cnn . This is no teacher distillation i.e you just copy layers from teacher model and then fine-tune the student model in stander way.
And for XSUM it uses ombination of Distillbert’s ce_loss and the hidden states MSE loss used in the tinybert paper.
DistilBERT paper 13
tinybert paper 4