Excuse me to bother you again.
I think now I understood what has caused the problem.
ProphetNetConfig has max_position_embeddings=512 as a parameter,
but the default --max_source_length of finetune_trainer.py is 1024.
It was natural that an error would occur unless the value was at least “less than 512”.
(Maybe more precisely, I also have to consider the input will be added special tokens, right?)
On GPU, I could run the script as below without CUDA error.
# For GPU

!python finetune_trainer.py \
    --learning_rate=3e-5 \
    --do_train --do_eval --evaluate_during_training \
    --max_source_length 500 \
    --per_device_train_batch_size 2 \
    --predict_with_generate \
    --n_train 100 \
    --n_val 10 \
    --model_name_or_path microsoft/prophetnet-large-uncased \
    --data_dir $XSUM_DIR \
    --output_dir tmp \
    --overwrite_output_dir

("max_source_length=500 & per_device_train_batch_size=8 (default batch size)” seemed to need too large memory size to run on the GPU I could use on Colab.)