trocr deepspeed
### System Info

- `transformers` version: 4.24.0
- Platform: Linux-5.10.14-1.el7.elrepo.x86_64-x86_64-with-glibc2.35
- Python version: 3.10.6
- Huggingface_hub version: 0.10.1
- PyTorch version (GPU?): 1.13.0+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

Hi @stas00,


### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I am using huggingface integrated deepspeed to train TROCR in jupyterlab.

For both load method are the same issue.
`model = VisionEncoderDecoderModel.from_encoder_decoder_pretrained()`
`model = VisionEncoderDecoderModel.from_pretrained()`


### Expected behavior

However, I met an issue
`AttributeError: 'VisionEncoderDecoderConfig' object has no attribute 'hidden_size'` 

Do we have any configuration to fix this?
issue resolved by
`hidden_size = 768`
`setattr(model.config, "hidden_size", hidden_size)`

refer to this #15526