KeyError: 'eval_loss' when doing LukeForMaskedLM
### System Info

- `transformers` version: 4.30.2
- Platform: Linux-5.14.21-150400.24.55-default-x86_64-with-glibc2.31
- Python version: 3.10.10
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 1.13.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: **Yes**
- Using distributed or parallel set-up in script?: **No**

### Who can help?

@sgugger

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I'm using [run_mlm.py](https://github.com/huggingface/transformers/blob/main/examples/pytorch/language-modeling/run_mlm.py) to evaluate some models on a custom dataset (one text by line). In some models (BERT, XLM-RoBERTa, DeBERTa) I managed to run the evaluations successfully and even on LUKE I can start the evaluation, but after all the evaluation steps on [studio-ousia/mluke-base-lite](https://huggingface.co/studio-ousia/mluke-base-lite), I got the following:

```
Traceback (most recent call last):
  File "/cfs/home/u021274/higo/./language-modeling/run_mlm.py", line 658, in <module>
    main()
  File "/cfs/home/u021274/higo/./language-modeling/run_mlm.py", line 629, in main
    perplexity = math.exp(metrics["eval_loss"])
KeyError: 'eval_loss'
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /cfs/home/u021274/higo/./language-modeling/run_mlm.py:658 in <module>                            │
│                                                                                                  │
│   655                                                                                            │
│   656                                                                                            │
│   657 if __name__ == "__main__":                                                                 │
│ ❱ 658 │   main()                                                                                 │
│   659                                                                                            │
│                                                                                                  │
│ /cfs/home/u021274/higo/./language-modeling/run_mlm.py:629 in main                                │
│                                                                                                  │
│   626 │   │   max_eval_samples = data_args.max_eval_samples if data_args.max_eval_samples is n   │
│   627 │   │   metrics["eval_samples"] = min(max_eval_samples, len(eval_dataset))                 │
│   628 │   │   try:                                                                               │
│ ❱ 629 │   │   │   perplexity = math.exp(metrics["eval_loss"])                                    │
│   630 │   │   except OverflowError:                                                              │
│   631 │   │   │   perplexity = float("inf")                                                      │
│   632 │   │   metrics["perplexity"] = perplexity                                                 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
KeyError: 'eval_loss'
```

To do that, i entered the following prompt:

```
CUDA_VISIBLE_DEVICES=5 python3 ./language-modeling/run_mlm.py \
    --model_name_or_path studio-ousia/mluke-base-lite \
    --validation_file ./data/test_data.txt \
    --max_seq_length 512 \
    --line_by_line True \
    --do_eval \
    --output_dir ./other_models/mluke-base-lite \
    --fp16 True
```

The configuration above was functional on all other before mentioned models, only on LUKE I'm having this issue.

### Expected behavior

Evaluation of mluke-base-lite on a given text dataset.
This example might need adjustments to be used on Luke, as the model has an API that is slightly different from BERT.