Running tokenizer on dataset -- Hangs
### System Info

transformers              4.20.0.dev0               dev_0
python                    3.7.11               h12debd9_0
linux os.

### Who can help?

@SaulLu when I use the wikitext-103 dataset the tokenizer hangs with `Running tokenizer on dataset` and shows no progress. This was not always an issue but as of today has become one. It will either hang at the end of tokenizing or at the very beginning. 

Any idea why this would be hanging here? I am not out of memory and am using 16 or 8 CPUs with the `--preprocessing_num_workers` and `run_clm.py` interface with a GPT-2 model. Using an A100. 

Thanks, 
Trenton 

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
python run_clm.py --train_name testing --fp16 True --dataloader_num_workers 4 --dataloader_pin_memory True --per_device_train_batch_size 8 --per_device_eval_batch_size 8 --optim adamw_hf --seed 42 --model_type gpt2 --dataset_name wikitext --dataset_config_name wikitext-103-v1 --tokenizer_name gpt2 --preprocessing_num_workers 16 --do_train --do_eval --evaluation_strategy steps --eval_steps 10000 --save_steps 10000 --save_total_limit 1 --num_train_epochs 175
```

### Expected behavior

No hanging. 
Setting `--preprocessing_num_workers 1` removes this issue...<|||||>This looks like an issue that should be reported to the [Datasets repo](https://github.com/huggingface/datasets/).<|||||>Ok thanks I've just cross linked it.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.