Trucated Outputs while finetuning 'bart-base' on XSUM [Summarization Task]
Hey!
I am trying to finetune bart-base model on the XSUM using standard commands. In the test-generations.txt file, the outputs I am getting after a few epochs (2-3) are truncated arbitrarily. Here is the exact command I am using:
`./finetune.sh --data_dir $XSUM_DIR --train_batch_size=8 --eval_batch_size=8 --output_dir=xsum_results --num_train_epochs 1 --model_name_or_path facebook/bart-base`
xsum_results is the directory I created and I am running this inside the examples/seq2seq directory. 

I referred to these issues but could not find anything that could help me:
https://github.com/huggingface/transformers/issues/5656
https://github.com/huggingface/transformers/issues/6502

Some examples of the outputs I am getting:
German carmaker Daimler has reported a rise in sales of its cars and trucks in
Angelina Jolie has been honoured at a film festival in Bosnia, where she was the
Cuba's President Raul Castro has said he will introduce a series of reforms to the country
People who are suicidal should be given more help to stop them from jumping, a charity has

My pip freeze:
`absl-py==0.10.0
cachetools==4.1.1
certifi==2020.6.20
chardet==3.0.4
click==7.1.2
dill==0.3.2
filelock==3.0.12
future==0.18.2
gitdb==4.0.5
GitPython==3.1.8
google-auth==1.21.3
google-auth-oauthlib==0.4.1
grpcio==1.32.0
idna==2.10
joblib==0.16.0
Markdown==3.2.2
nlp==0.4.0
nltk==3.5
numpy==1.19.2
oauthlib==3.1.0
packaging==20.4
pandas==1.1.2
Pillow==7.2.0
portalocker==2.0.0
protobuf==3.13.0
pyarrow==1.0.1
pyasn1==0.4.8
pyasn1-modules==0.2.8
pyparsing==2.4.7
python-dateutil==2.8.1
pytorch-lightning==0.9.0
pytz==2020.1
PyYAML==5.3.1
regex==2020.9.27
requests==2.24.0
requests-oauthlib==1.3.0
rouge-score==0.0.4
rsa==4.6
sacrebleu==1.4.14
sacremoses==0.0.43
sentencepiece==0.1.91
six==1.15.0
smmap==3.0.4
tensorboard==2.2.0
tensorboard-plugin-wit==1.7.0
tokenizers==0.8.1rc2
torch==1.6.0+cu101
torchvision==0.7.0+cu101
tqdm==4.49.0
transformers @ git+https://github.com/yashgupta-7/transformers@9e68d075a4100906509170498480823e7e61874a
urllib3==1.25.10
Werkzeug==1.0.1
xxhash==2.0.0
zipp==3.2.0`

Here is the recommended setting for XSUM.
`--max_target_length=60 --val_max_target_length=60 --test_max_target_length=100`
But the default values are all higher than these, so this cannot be a problem in my opinion.

It would be great if someone can point me to the potential problems that may be the reason. I am looking forward to fine-tuning on a custom dataset and really want the standard XSUM to get working!
Try adding `decoder_start_token_id=2` to `best_tfmr/config.json` and let me know if that changes anything!<|||||>I just did the above-mentioned change and decoded (run_eval.py) without training any further. Still, the issue persists. <|||||>Ok, you could try re-training/training more with new training code. We can't reproduce this on `master`<|||||>okay, thanks!