Autotokenizer error "Already borrowed" when used on thread pool
## Environment info
- `transformers` version: 4.8.2
- Platform: Databricks
- Python version: 3.7.10
- PyTorch version (GPU?): GPU
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: Yes

## Information

Model I am using : camembert-base:

The problem arises when i try to use a tokenizer (from whatever model in my experiments) on multiple thread pools with an Autotokenizer, the error **RuntimeError: Already borrowed** get raised, i haven't tried if the same issues occure whith AutoModel, but i suspect it would, this makes it completly inefficient as it requires to duplicate the tokenizer on each thread (same for model), and is a real problem for packages like Petastorm  /  Horovod.

## To reproduce
Below you'll find a simple snippet of code to reproduce the error:
Steps to reproduce the behavior:
```Python
 from multiprocessing.dummy import Pool as ThreadPool
  
tokenizer =   AutoTokenizer.from_pretrained(pretrained_model_name_or_path="camembert-base")
def tokenizer_test(text):
    print(tokenizer(text))
    
pool = ThreadPool(10)  
data_list = ['this is a test'] * 10
pool.map(tokenizer_test, data_list)
pool.close()
pool.join()
```
However this works fine if i switch the Autotokenizer with CamembertTokenizer.from_pretrained("camembert-base") for example.
Duplicate of https://github.com/huggingface/tokenizers/issues/537<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.