 No inf checks were recorded for this optimizer
### System Info

transformers 4.30.0.dev0 

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

`trainer = transformers.Trainer(
    model = model,
    train_dataset=data,
    args =training_args,
    data_collator=transformers.DataCollatorForLanguageModeling(tokenizer, mlm=False),   
)

trainer.train()
`

### Expected behavior

```
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
/tmp/ipykernel_1639/4032920361.py in <cell line: 1>()
----> 1 trainer.train()

~/.local/lib/python3.8/site-packages/transformers/trainer.py in train(self, resume_from_checkpoint, trial, ignore_keys_for_eval, **kwargs)
   1659             self._inner_training_loop, self._train_batch_size, args.auto_find_batch_size
   1660         )
-> 1661         return inner_training_loop(
   1662             args=args,
   1663             resume_from_checkpoint=resume_from_checkpoint,

~/.local/lib/python3.8/site-packages/transformers/trainer.py in _inner_training_loop(self, batch_size, args, resume_from_checkpoint, trial, ignore_keys_for_eval)
   2013                         optimizer_was_run = scale_before <= scale_after
   2014                     else:
-> 2015                         self.optimizer.step()
   2016 
   2017                     if optimizer_was_run:

~/.local/lib/python3.8/site-packages/accelerate/optimizer.py in step(self, closure)
    132             elif self.scaler is not None:
    133                 scale_before = self.scaler.get_scale()
--> 134                 self.scaler.step(self.optimizer, closure)
    135                 self.scaler.update()
    136                 scale_after = self.scaler.get_scale()

~/.local/lib/python3.8/site-packages/torch/cuda/amp/grad_scaler.py in step(self, optimizer, *args, **kwargs)
    370             self.unscale_(optimizer)
    371 
--> 372         assert len(optimizer_state["found_inf_per_device"]) > 0, "No inf checks were recorded for this optimizer."
    373 
    374         retval = self._maybe_opt_step(optimizer, optimizer_state, *args, **kwargs)

AssertionError: No inf checks were recorded for this optimizer.


```
Miss configuration LoraConfig caused the issue