Training Transformer XL from scratch for CLM
I am training Transformer XL using the **run_clm.py** script. I was able to train GPT2, XLNet, CTRL etc without any issue. But with Transformer XL, I get the error

  ```
File "../lib/python3.8/site-packages/transformers/trainer.py", line 1272, in train
    tr_loss += self.training_step(model, inputs)
  File "../lib/python3.8/site-packages/transformers/trainer.py", line 1734, in training_step
    loss = self.compute_loss(model, inputs)
  File "../lib/python3.8/site-packages/transformers/trainer.py", line 1776, in compute_loss
    loss = outputs["loss"] if isinstance(outputs, dict) else outputs[0]
  File "../lib/python3.8/site-packages/transformers/file_utils.py", line 1738, in __getitem__
    return inner_dict[k]
KeyError: 'loss'

```
I am using the same input format as in the other case of other models.  Can anyone tell me what is the issue here ?
Transformer XL is not compatible with the Trainer API, and won't work with any of the example scripts. You should use another model, or a modified version of the `run_clm_no_trainer` script.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.