test_run_glue_with_pabee failing
examples/bert-loses-patience/test_run_glue_with_pabee.py::PabeeTests::test_run_glue

https://app.circleci.com/pipelines/github/huggingface/transformers/10373/workflows/0c9f2e61-2732-4857-84f0-71b59ddf10a9/jobs/71646


@JetRunner 
@sshleifer Thanks for the issue. I noticed that this test fails sometimes. Do you have any idea how long has this problem been? Is it broken at the beginning or recently?<|||||>Started breaking within the last few days. It is breaking fairly consistently at this point.<|||||>It's I guess just flaky, for example it's not broken on master right now.<|||||>But it's not really a flaky "type of error". It's hitting IndexError on an embedding lookup.
```
IndexError: index out of range in self
```
<|||||>It also fails with the assert: `self.assertGreaterEqual(value, 0.75)` in my case (three times out of four right now).