Replaced torch.load for loading the pretrained vocab of TransformerXL tokenizer to pickle.load
TransformerXL tokenizer requires torch to work because it uses torch.load to load the vocabulary. This means that if I'm using the TF2 implementation, I have to add torch as a dependency just for that. So I replaced the call to a call to pickle.load (which is what torch.load internally uses) to solve the issue.
Tested an all the TransformerXL related tests (also the slow ones) and they all passed.
Hi @w4nderlust that's a good idea!

The CI seems not happy with the change though and it seems related to your changes.

Do you think you could take a look?<|||||>@thomwolf trying to see the details of the failing tests, but circleci wat me to login with github and grant access to all my repos and orgs, I prefer to avoid it.

If you can point out the failing tests I'm happy to take a look at it.<|||||>@thomwolf I inspected  further and this is what i discovered:
1. i was running the `modeling_transfo_xl.py` tests, but I should have been running the `tokenization_transfo_xl.py` test. I Imagine the errors in CI are coming from there.
2. upon further inspection, I noticed that inside `tokenization_transfo_xl.py` torch is used everywhere. There probably was a design decision that led to that which I'm not aware of, but as a user i would ask you to reconsider, because if the tokenizer uses torch, the TF version of TransformerXL can neve be used without installing torch. The extent to which torch is used goes beyond my current familiarity with the library, so i will refrain to propose modifications to it, a part from what i propose in the next point.
3. in my commit I replaced the loading of the vocab, but upon inspection i realized that, yes, torch uses pickle, but it does that in a way that is peculiar, including magic numbers and protocol versions and some custom logic that will take some time to reverse engineer ( https://github.com/pytorch/pytorch/blob/0c01f136f3c8d16f221d641befcb5a74142bbeb1/torch/serialization.py#L764-L774 ). It doesn't seem you can directly load the vocab dictionary without re-implementing quite some load code from torch, plus this doesn't sound like a sound approach because PyTorch can itself start using a new load mechanism in the future. So, what I tried to do is to replace ALSO `torch.save` usages within the context of vocabulary save with `pickle.dump`.( line 260-262) in my last commit). The effect is that now all `test_tokenize_transfo_cl.py` tests pass, the vocab can be saved and loaded, but, because  the vocab that ships with the pretrained models  was saved originally with torch, If it try to load from pretrained model, loading doesn't work (what is loaded is just the torch magic number). So here I guess you have to make a call about what you want to do: if you want to use pickle to load and save vocab, this PR does it for you, but you have to change the TransformerXL pretrained model that you ship by replacing the vocab file saved with PyTorch with one saved with pickle (the code to do it from the current vocab file is straightforward `pickle.dump(torch.load(vocab_file), vocab_file)`).

As I realized the issue is bigger than I originally thought, it would be great if someone could look at it in more detail from the HF side.<|||||>Hi @w4nderlust ok, I'm reaching this PR now.

So the original tokenizer for Transformer-XL was copied from the original research work to be able to import the trained checkpoints. The reliance on PyTorch is thus not really a design decision of us but more of the original author.

We can definitely reconsider it and if you don't mind, I'll try to build upon your PR to relax this reliance on PyTorch while keeping backward compatibility if possible.<|||||># [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6935?src=pr&el=h1) Report
> Merging [#6935](https://codecov.io/gh/huggingface/transformers/pull/6935?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/aba4e22944f0c985bebdcde51d47a565dd4f551d?el=desc) will **increase** coverage by `1.96%`.
> The diff coverage is `79.16%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/6935/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/6935?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #6935      +/-   ##
==========================================
+ Coverage   74.71%   76.67%   +1.96%     
==========================================
  Files         194      181      -13     
  Lines       39407    35738    -3669     
==========================================
- Hits        29441    27401    -2040     
+ Misses       9966     8337    -1629     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/6935?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6935/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `83.09% <60.00%> (+0.17%)` | :arrow_up: |
| [src/transformers/tokenization\_transfo\_xl.py](https://codecov.io/gh/huggingface/transformers/pull/6935/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdHJhbnNmb194bC5weQ==) | `42.48% <84.21%> (+0.74%)` | :arrow_up: |
| [src/transformers/configuration\_reformer.py](https://codecov.io/gh/huggingface/transformers/pull/6935/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX3JlZm9ybWVyLnB5) | `21.62% <0.00%> (-78.38%)` | :arrow_down: |
| [src/transformers/modeling\_reformer.py](https://codecov.io/gh/huggingface/transformers/pull/6935/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19yZWZvcm1lci5weQ==) | `16.71% <0.00%> (-77.89%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_xlm.py](https://codecov.io/gh/huggingface/transformers/pull/6935/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl94bG0ucHk=) | `18.73% <0.00%> (-74.53%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_flaubert.py](https://codecov.io/gh/huggingface/transformers/pull/6935/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9mbGF1YmVydC5weQ==) | `21.53% <0.00%> (-68.15%)` | :arrow_down: |
| [src/transformers/tokenization\_marian.py](https://codecov.io/gh/huggingface/transformers/pull/6935/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fbWFyaWFuLnB5) | `32.75% <0.00%> (-66.38%)` | :arrow_down: |
| [src/transformers/trainer\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6935/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyX3V0aWxzLnB5) | `60.63% <0.00%> (-20.14%)` | :arrow_down: |
| [src/transformers/trainer.py](https://codecov.io/gh/huggingface/transformers/pull/6935/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyLnB5) | `55.70% <0.00%> (-15.11%)` | :arrow_down: |
| [src/transformers/integrations.py](https://codecov.io/gh/huggingface/transformers/pull/6935/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9pbnRlZ3JhdGlvbnMucHk=) | `29.00% <0.00%> (-5.66%)` | :arrow_down: |
| ... and [71 more](https://codecov.io/gh/huggingface/transformers/pull/6935/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/6935?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6935?src=pr&el=footer). Last update [aba4e22...cd57922](https://codecov.io/gh/huggingface/transformers/pull/6935?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
<|||||>Thank you for the work on this! Much appreciated! :)