Pre-processing re-runs for each process
### System Info

- `transformers` version: 4.22.0.dev0
- Platform: Linux-4.15.0-180-generic-x86_64-with-glibc2.27
- Python version: 3.10.4
- Huggingface_hub version: 0.8.1
- PyTorch version (GPU?): 1.12.1+cu102 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: yes

### Who can help?

@muellerzr @sgugger @patil-suraj 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

In `examples/summarization_no_trainer.py`, we use the following code to pre-process the data:
```
    with accelerator.main_process_first():
        processed_datasets = raw_datasets.map(
            preprocess_function,
            batched=True,
            num_proc=args.preprocessing_num_workers,
            remove_columns=column_names,
            load_from_cache_file=not args.overwrite_cache,
            desc="Running tokenizer on dataset",
        )
```
In a multi-GPU (process) setup, the main process pre-processes the data first and saves it in the cache. Ideally, the other processes should just pick it up from there. But that's not happening. Every process is re-pre-processing the data. This is a problem when the data is large.

I have tried to check if something changes during the run with `Hasher.hash(preprocess_func)`. But the hash remains the same.

### Expected behavior

Processes other than the main process should read the processed data from cache.the 
Do you have a reproducer of the behavior you are seeing? On my side data is processed once.<|||||>Wow, ok. So looks like the example scripts always set `--overwrite_cache` to `True`.

It is currently `parser.add_argument("--overwrite_cache", type=bool, default=None)` instead of `parser.add_argument("--overwrite_cache", action="store_true")`.

Will make a PR soon.