[LEDModel, Longformer] Make_fx compatibility
### System Info

- transformers version: 4.29.2
- Platform: Linux
- Python version: 3.8.16
- PyTorch version: 2.0.1

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Related to #23907

To reproduce:
```python
from torch.fx.experimental.proxy_tensor import make_fx
from transformers import LEDModel

model = LEDModel.from_pretrained('allenai/led-base-16384', torchscript=True)
inp = model.dummy_inputs['input_ids']
model.eval()
fx_g = make_fx(model)(inp)
```

The presence of `.item()` and `torch.div` cause make_fx to fail for LEDModel and Longformer with the following error:
```
RuntimeError: It appears that you're trying to get value out of a tracing tensor with aten._local_scalar_dense.default - erroring out! It's likely that this is caused by data-dependent control flow or similar.  It may be possible to trace this with dynamic shapes; try setting tracing_mode='symbolic' in your make_fx call.
```

The calls to `item()` could be removed without side effects, and I believe the same is true for replacing `torch.div` with regular python divisions.

Even with these adjustments, make_fx seems to fail in both models because of `is_global_attn`:
https://github.com/huggingface/transformers/blob/f1660d7e23d4432513fe060bde4f9b7b29f05204/src/transformers/models/led/modeling_led.py#L233
https://github.com/huggingface/transformers/blob/f1660d7e23d4432513fe060bde4f9b7b29f05204/src/transformers/models/longformer/modeling_longformer.py#L601

I am not sure if it would be possible to have a workaround for that.

One thing to note is that the presence of `.item()` and `torch.div` also causes graph breaks when using torch dynamo to get the FX representations of these models. It seems that `is_global_attn` is not an issue in that case.

### Expected behavior

The full FX representation of LEDModel/Longformer using make_fx
@Giuseppe5 As with #23907, happy to look at a PR with a fix! 

cc @ArthurZucker @younesbelkada <|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.