TransfoXLCorpus requires pytorch to tokenize files
## 🐛 Bug

The current TransfoXLCorpus code requires pytorch, and fails if it is not installed.

Model I am using (Bert, XLNet....): Transformer-XL

Language I am using the model on (English, Chinese....): Other

The problem arise when using:
* [ X ] my own modified scripts: I'm using a very simple script to read in text files, see code below

The tasks I am working on is:
* [ X ] my own task or dataset: I am attempting to build a corpus from my own dataset of long text sentences.

## To Reproduce

Steps to reproduce the behavior:

corpus = TransfoXLCorpus(lower_case=True, delimiter=" ")
corpus.build_corpus(EXAMPLE_DIR, "text8")

Traceback (most recent call last):
  File "build_xl_corpus.py", line 26, in <module>
    corpus.build_corpus(EXAMPLE_DIR, "text8")
  File "/home/tom/.local/lib/python3.7/site-packages/transformers/tokenization_transfo_xl.py", line 521, in build_corpus
    os.path.join(path, 'train.txt'), ordered=True, add_eos=False)
  File "/home/tom/.local/lib/python3.7/site-packages/transformers/tokenization_transfo_xl.py", line 187, in encode_file
    encoded.append(self.convert_to_tensor(symbols))
  File "/home/tom/.local/lib/python3.7/site-packages/transformers/tokenization_transfo_xl.py", line 246, in convert_to_tensor
    return torch.LongTensor(self.convert_tokens_to_ids(symbols))
NameError: name 'torch' is not defined

## Expected behavior

I did not expect this behavior to require pytorch

## Environment

* OS: Ubuntu
* Python version:
* PyTorch version: None
* PyTorch Transformers version (or branch): 2.0.0
* Using GPU ? Yes
* Distributed of parallel setup ? No
* Any other relevant information:

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
