Fix a TF Vision Encoder Decoder test
# What does this PR do?

Simple fix for the following CI failure: 

```
tests.vision_encoder_decoder.test_modeling_tf_vision_encoder_decoder.TFViT2GPT2EncoderDecoderModelTest.test_pt_tf_equivalence
```
This is due to a few PyTorch device issue (need `pt_inputs` to be on the correct device).

Tested on GCP GPU VM - it pass now.

@patrickvonplaten @sgugger 
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_15896). All of your documentation changes will be reflected on that endpoint.<|||||>Probably wait @patrickvonplaten too, but once ready, should I merge this PR myself ? (or still should be one of you to do so?)<|||||>You can merge your own PRs as long as you have the approvals you want :-) 