'ReduceLROnPlateau' object has no attribute 'get_last_lr'
### System Info

- `transformers` version: 4.29.2
- Platform: Linux-4.18.0-425.19.2.el8_7.x86_64-x86_64-with-glibc2.28
- Python version: 3.9.5
- Huggingface_hub version: 0.14.1
- Safetensors version: not installed
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes (cuda 11.7)
- Using distributed or parallel set-up in script?: No


### Who can help?

@sgug

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I run the trainer in the default way (training a LM from scratch); using a cosine LR scheduler works fine, but the recently added `'reduce_lr_on_plateau'` seems to be incompatible with my current setup.

```python
from datasets import DatasetDict
from transformers import (
    DataCollatorForLanguageModeling,
    PreTrainedTokenizerFast,
    Trainer,
    TrainingArguments,
)


def initialize_trainer(
    model: AutoModelForMaskedLM,
    tokenizer: PreTrainedTokenizerFast,
    data_collator: DataCollatorForLanguageModeling,
    datasets: DatasetDict,
    model_init = None,
    **config,
):
    args = TrainingArguments(**config)

    trainer = Trainer(
        model=model,
        tokenizer=tokenizer,
        args=args,
        data_collator=data_collator,
        train_dataset=datasets["train"],
        eval_dataset=datasets["valid"],
    )

    return trainer


trainer = initialize_trainer(
    model, 
    tokenizer, 
    data_collator, 
    datasets, 
    output_dir='checkpoints', 
    save_steps=10_000, 
    eval_steps=100, 
    logging_steps=100,
    per_device_train_batch_size=64,
    per_device_eval_batch_size=64,
    gradient_accumulation_steps=8,
    weight_decay=0.1,
    lr_scheduler_type='reduce_lr_on_plateau',
    learning_rate=5e-4,
    num_train_epochs=1,
    fp16=True,
    max_grad_norm=0.5,
    group_by_length=True,
    auto_find_batch_size=False,
    do_eval=True,
    evaluation_strategy='steps',
    report_to="wandb",
)

trainer.train()
```

```python
>>>
│ ~/.local/lib/python3.9/site-packages/transformers/trainer_pt_utils.py:854 in              │
│ _get_learning_rate                                                                        │
│                                                                                           │
│    851 │   │   │   else:                                                                  │
│    852 │   │   │   │   raise                                                              │
│    853 │   else:                                                                          │
│ ❱  854 │   │   last_lr = self.lr_scheduler.get_last_lr()[0]                               │
│    855 │   │   if torch.is_tensor(last_lr):                                               │
│    856 │   │   │   last_lr = last_lr.item()                                               │
│    857 │   return last_lr                                                                 │
╰───────────────────────────────────────────────────────────────────────────────────────────╯
AttributeError: 'ReduceLROnPlateau' object has no attribute 'get_last_lr'
```

### Expected behavior

I expect the scheduler to work without any additional steps, but maybe an extra argument is required? Or is my torch version incompatible with the way it is currently implemented?
This might be a bug indeed. Will have a look when I have some time but if someone wants to open a PR to fix this, they are more than welcome to do so!