[ROC_BERT] Make CI happy
# What does this PR do?

Fixes a small slow test that was failing when trying to play with 8-bit conversion for BERT models

cc @ydshieh 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20175). All of your documentation changes will be reflected on that endpoint.<|||||>Hi @younesbelkada Thank you for trying make CI happy.

But I am not sure why we need `torch.allclose` for 2 integer tensors. This should be used for float tensors. For the integer outputs (here token ids), they should be equal, so `self.assertEqual` would be the one to use I think.<|||||>Thanks @ydshieh for the heads up, indeed I believe we can use that. Let me update it<|||||>The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20175). All of your documentation changes will be reflected on that endpoint.<|||||>ahah merci beaucoup @ydshieh !! 