torch compile changes model output in half precision
### System Info

- `transformers` version: 4.31.0
- Platform: Linux-5.15.0-1034-oracle-x86_64-with-glibc2.29
- Python version: 3.8.10
- Huggingface_hub version: 0.15.1
- Safetensors version: 0.3.1
- Accelerate version: 0.20.3
- Accelerate config:    not found
- PyTorch version (GPU?): 2.0.0+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?:  does not matter (provide GPU results, same on CPU)
- Using distributed or parallel set-up in script?: No

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction



```
import torch
import transformers

if __name__ == "__main__":
    device = torch.device('cuda')
    model = transformers.AutoModelForTokenClassification.from_pretrained(
        "Jean-Baptiste/roberta-large-ner-english").to(device)

    model.eval()
    a = torch.randint(100, 2000, (128, 256), device=device)

    with torch.no_grad(), torch.cuda.amp.autocast():
        out_not_compiled = model(input_ids=a, attention_mask=torch.ones_like(a)).logits

    model = torch.compile(model)
    with torch.no_grad(), torch.cuda.amp.autocast():
        out_compiled = model(input_ids=a, attention_mask=torch.ones_like(a)).logits

    print(
        torch.sum(torch.abs(out_compiled.to(torch.float32) - out_not_compiled)) /
        (128 * 256))

>> tensor(0.0120, device='cuda:0')  # note that actual difference is > 410. 
```

Autocast to both `float16` and `bfloat16`  produces the same difference. 

(Commenting out model compilation results into the same output)

### Expected behavior

Small difference in output vectors. 
That seems like an issue for PyTorch more than Transformers :-) Also note that there is a special order for context manager autocast and compiled model to respect (can't remember right now) which also may be the cause.