Trainer().train() just idles, neccesitates kernel restart
### System Info

- `transformers` version: 4.22.1
- Platform: Linux-4.19.0-21-cloud-amd64-x86_64-with-debian-10.13
- Python version: 3.7.12
- Huggingface_hub version: 0.9.1
- PyTorch version (GPU?): 1.11.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help?

@sgugger because I think you're the original author of the example

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Run https://github.com/huggingface/notebooks/blob/main/examples/language_modeling.ipynb, Section *Fine-tuning a language model*. (change `push_to_hub=False).
2. trainer.train() prints epoch and step counts, then doesn't seem to do anything useful (python process cpu usage stays at +-0, so does GPU utilization), and also process interrupt doesn't work


### Expected behavior

Either training starts or some kind of error is shown
How are you running the notebook? It works fine on my side.<|||||>I'm unable to upload it here, but it's literally the original example, sans the `notebook_login` and `push_to_hub=False`. 
And it *works* on my local machine (installed the same libraries as listed above, except my kernel is `5.19.8-200.fc36.x86_64`)


As reported, it fails when I'm running on a Vertex AI instance. That's what the instance says about itself: 

```
Environment
    PyTorch 1.11 (with Intel® MKL-DNN/MKL)
Environment version
    M94
Machine type
    n1-standard-8 (8 vCPUs, 30 GB RAM)
GPU
    NVIDIA Tesla V100 x 1
```
I also tried running it (on the instance) with `os.environ['CUDA_VISIBLE_DEVICES'] = '-1'`, but that didn't help.
<|||||>This Vertex AI instance type *works*: 
```
Environment
    Python 3 (with Intel® MKL and CUDA 11.0)
Environment version
    M97
Machine type
    n1-standard-4 (4 vCPUs, 15 GB RAM)
GPU
    NVIDIA Tesla T4 x 1
```
So I can now get my work done, but I think the reported issue is still valid<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.