Also you could make use of this notebook - it should be applicable 1-to-1, when you change the tokenizer correctly and switch EncoderDecoderModel with ProphetNetForConditionalGeneration:

  
      github.com
  
  
    patrickvonplaten/notebooks/blob/master/BERT2BERT_for_CNN_Dailymail.ipynb 24
{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "BERT2BERT for CNN/Dailymail",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyMYchE5beUGXDKxJoG2JsLp",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU",
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "e13b8fdb8a564e8699c77890e29995bf": {
          "model_module": "@jupyter-widgets/controls",


  This file has been truncated. show original