[`ViTHybrid`] fix last `accelerate` slow test
# What does this PR do?

FIxes the last test that I forgot to run on multi gpu setup ! Sorry for the multiple iterations

Fixes: 
1- https://github.com/huggingface/transformers/actions/runs/3650247864
2- in `test_model_parallelism` the test splits the model into different sub-modules, with respect to the class attribute `_no_split_modules`. Sometimes a manual device assignment is needed to avoid errors such as `tensors not on the same device`
3- Similar to https://github.com/huggingface/transformers/pull/19912 (check the last modification)

cc @sgugger @ydshieh 
_The documentation is not available anymore as the PR was closed or merged._