BatchEncoding.to accepted types too restrictive
## Environment info
     
- `transformers` version: 4.1.1
- Platform: Linux-4.14.81.bm.15-amd64-x86_64-with-debian-9.11
- Python version: 3.7.9
- PyTorch version (GPU?): 1.7.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: yes

### Who can help

## Information

In `BatchEncoding.to`, the only accepted class types are `str` and `torch.device`. I think some libraries like pytorch-lightning call `.to` with the integer value for the GPU number, and HF complains about this when it is perfectly valid:

```
>>> x = torch.zeros(1)
>>> x.to(0)
tensor([0.], device='cuda:0')
```

## Expected behavior

Also allow int values in BatchEncoding.to