Pegasus distillation is now supported in the seq2seq examples.

  
      
      GitHub
  
  
    

huggingface/transformers 15

🤗Transformers: State-of-the-art Natural Language Processing for Pytorch and TensorFlow 2.0. - huggingface/transformers


  
  
    
    
  
  


You’ll need to pass pegaus model path instead of bart and multi-news directory formatted according to the format specified in the readme.