I just tested with microsoft/deberta-v3-large and I don’t get any error : the training starts well for me.
I just added 2 more arguments --per_device_train_batch_size 2 --per_device_eval_batch_size 2 because otherwise I didn’t have enough VRAM.
I don’t have exactly the same setting as you, though. Would it be possible for you to try again with the same settings as me (first trying with the last transformers version on master, then with Python version: 3.8.10 and finally with PyTorch 1.9.0)?
My setting:
- `transformers` version: 4.13.0.dev0
- Python version: 3.8.10
- PyTorch version (GPU?): 1.9.0+cu102 (True)

Can’t wait to see how it turns out on your end!