I have this problem, too. I am running finetune_t5.sh (or finetune bart) as given in transformers/examples/seq2seq. The error is given even when running the example script as given:
From transformers/examples/seq2seq, run ./finetune_bart_tiny.sh. Observe the following output with warning about scheduler:
cnn_tiny.tgz                  100%[================================================>]  22.59K  --.-KB/s    in 0.08s
2020-10-11 22:53:22 (299 KB/s) - ‘cnn_tiny.tgz’ saved [23131/23131]
x cnn_tiny/
x cnn_tiny/train.target
x cnn_tiny/train.source
x cnn_tiny/val.source
x cnn_tiny/val.target
x cnn_tiny/test.source
x cnn_tiny/test.target
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
Epoch 1: 100%|█████████████████████████████████████████████████████| 4/4 [00:48<00:00, 12.22s/it, loss=10.838, v_num=6]/usr/local/Caskroom/miniconda/base/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:200: UserWarning: Please also save or load the state of the optimzer when saving or loading the scheduler.
warnings.warn(SAVE_STATE_WARNING, UserWarning)
Epoch 1: 100%|█████████████████████████████████████████████████████| 4/4 [00:59<00:00, 14.81s/it, loss=10.838, v_num=6]
(cryptic)