transformers-cli serve not working
### System Info

System info
``` bash
- `transformers` version: 4.27.0
- Platform: macOS-12.3.1-arm64-arm-64bit
- Python version: 3.8.12
- Huggingface_hub version: 0.13.2
- PyTorch version (GPU?): 2.0.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```


### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction


The following command fails for `transformers[serving]==4.27.0`
```bash
transformers-cli serve --task=fill-mask --model=bert-base-uncased
```
this is the traceback
```bash
Traceback (most recent call last):
  File "venv/bin/transformers-cli", line 8, in <module>
    sys.exit(main())
  File "venv/lib/python3.8/site-packages/transformers/commands/transformers_cli.py", line 54, in main
    service = args.func(args)
  File "venv/lib/python3.8/site-packages/transformers/commands/serving.py", line 49, in serve_command_factory
    nlp = pipeline(
  File "venv/lib/python3.8/site-packages/transformers/pipelines/__init__.py", line 976, in pipeline
    return pipeline_class(model=model, framework=framework, task=task, **kwargs)
  File "venv/lib/python3.8/site-packages/transformers/pipelines/base.py", line 773, in __init__
    self.model = self.model.to(device=device)
  File "venv/lib/python3.8/site-packages/transformers/modeling_utils.py", line 1811, in to
    return super().to(*args, **kwargs)
  File "venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1126, in to
    device, dtype, non_blocking, convert_to_format = torch._C._nn._parse_to(*args, **kwargs)
RuntimeError: Device index must not be negative
```

### Expected behavior

However, downgrading to `transformers[serving]==4.26.1` fixes the issue

```bash
INFO:     Started server process [22054]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://localhost:8888 (Press CTRL+C to quit)
```
cc @Narsil <|||||>This will be patched very soon, thanks for reporting!<|||||>> This will be patched very soon, thanks for reporting!

Thank you for fixing it so quickly:)