Solved by SaulLu in post #12 
              
              
                I just tested with microsoft/deberta-v3-large and I don’t get any error : the training starts well for me. 
I just added 2 more arguments --per_device_train_batch_size 2 --per_device_eval_batch_size 2 because otherwise I didn’t have enough VRAM. 
I don’t have exactly the same setting as you, though.…