[Tests] Fix slow opt tests
Fixes failing circle ci tests: 

```
tests/models/opt/test_modeling_opt.py::OPTEmbeddingsTest::test_logits
tests/models/opt/test_modeling_opt.py::OPTModelIntegrationTests::test_inference_no_head
```
_The documentation is not available anymore as the PR was closed or merged._<|||||>Hey, will review that ASAP (~1-2h) 