OOM risk in RobertaTokenizer/GPT2Tokenizer
# 🐛 Bug

## Information

Model I am using: Roberta (_roberta-base_)

Language I am using the model on: English 

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts

I am using a modified version of the [examples/distillation/scripts/binarized_data.py](https://github.com/huggingface/transformers/blob/master/examples/distillation/scripts/binarized_data.py)  file.

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset

I am tokenizing a gzipped JSON file that contains 2B inputs resulting in 250Gb of uncompressed data.
The tokenization function is divided across _n_ processes to make the tokenization part faster.
The resulting _token_ids_ are written as a list of integers in an output file.
While tokenization was done batch by batch, I noticed that my RAM was increasing. 
It caused an OOM error (I have 64GB or RAM) while only 1.5B inputs had been processed.

I identified the problem to be the `cache` attribute of the `GPT2Tokenizer` ([link](https://github.com/huggingface/transformers/blob/master/src/transformers/tokenization_gpt2.py#L191)) that is never flushed so its size will potentially grow infinitely.
Tokenizers inheriting from `GPT2Tokenizer` (such as `RobertaTokenizer` ) are thus also impacted.

## To reproduce

Steps to reproduce the behavior:

Run tokenization: `tokenizer.encode()` on a very big file using `GPT2Tokenizer` or `RobertaTokenizer`.

## Expected behavior

The memory footprint of the tokenizer should be constant while processing an infinite stream of inputs.

## Suggestion
I made a quick and dirty fix in my script by flushing the `cache` (tokenizer.cache.clear()) if its size reaches an arbitrarily set threshold (100k in my case) with no significant loss in terms of performance.
However I think that there are smarter solutions rather than flushing the whole cache content. One can use a LRU cache instead of a Python dict. You can also define a private method that checks if cache size reaches a threshold and perform flushing in an "elegant" way.

I know that for production purposes [Tokenizers](https://github.com/huggingface/tokenizers) lib would be more appropriate but I wanted to notice you this behavior.

## Environment info
     
- `transformers` version: 2.4.1
- Platform: Ubuntu 16.04 LTS
- Python version: 3.6.9
- PyTorch version: 1.4.0 (with GPU)
- Tensorflow version : 2.0.0 (with GPU)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: using python 3.6 [multiprocessing](https://docs.python.org/3.6/library/multiprocessing.html) lib

If lru_cache is used, the max size couldn't be configured at runtime or disabled completely. Trying to go around this with anonymous functions will cause pickling problems and is generally ugly.

A more elegant and straightforward solution is to use a custom cache with ordered dict and a max size checked at each insertion.

Both have a considerable performance impact versus the current unbounded dict though, about 10% more processing time which can really add up in a high throughput objects like tokenizers.

Pretty much any more advanced cache will come with a performance hit. Thoughts?<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
