TypeError: zero_grad() got an unexpected keyword argument 'set_to_none'
### System Info

Getting below error while runing Bert pretraining using Huggingface trainer with deepspeed 0.8.2 version and pytorch 2.0RC

Traceback (most recent call last):
  File "pretrain_glue.py", line 124, in <module>
    result = trainer.train()
  File "/opt/conda/envs/env/lib/python3.8/site-packages/transformers/trainer.py", line 1631, in train
    return inner_training_loop(
  File "/opt/conda/envs/env/lib/python3.8/site-packages/transformers/trainer.py", line 1814, in _inner_training_loop
    model.zero_grad(set_to_none=True)
TypeError: zero_grad() got an unexpected keyword argument 'set_to_none'

@sgugger , could this be due to below change?
Enforce same behavior as PyTorch 2.0 for older versions (https://github.com/huggingface/transformers/pull/22136)

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Run Bert pretrain with HF trainer and deepspeed 0.8.2 version on Pytorch 2.0 RC build

### Expected behavior

run with no errors
Yes, this has been fixed already on main. You just need to pull from source :-)<|||||>ah, thankyou for quick fix. :)