torch.jit support
### Feature request

Hi there, is torch.jit used by default for model inference in the transformers library, for example, in the Auto series APIs? If not, why isn't it used by default? Thank you.

### Motivation

None

### Your contribution

None
No it's not used by default since the Transformers library can't know whether you are going to use your model for training or inference. It also comes with constraints (like duplicating shared weights) so it's up to the user to activate it if the situation suits their needs. You can pass `torchscript=True` when loading your model to have the jit-compilation done for you.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.