Failing bart-base slow test
```
=================================== FAILURES ===================================
____________ BartModelIntegrationTests.test_bart_base_mask_filling _____________
[gw0] linux -- Python 3.7.6 /home/hf/actions-runner_transformers/_work/transformers/transformers/.env/bin/python

self = <tests.test_modeling_bart.BartModelIntegrationTests testMethod=test_bart_base_mask_filling>

    @slow
    def test_bart_base_mask_filling(self):
        pbase = pipeline(task="fill-mask", model="facebook/bart-base")
        src_text = [" I went to the <mask>."]
        results = [x["token_str"] for x in pbase(src_text)]
        expected_results = ["Ġbathroom", "Ġrestroom", "Ġhospital", "Ġkitchen", "Ġcar"]
>       self.assertListEqual(results, expected_results)
E       AssertionError: Lists differ: ['Ġlibrary', 'Ġhospital', 'Ġbathroom', 'Ġmovies', 'Ġpolice'] != ['Ġbathroom', 'Ġrestroom', 'Ġhospital', 'Ġkitchen', 'Ġcar']
E       
E       First differing element 0:
```

https://github.com/huggingface/transformers/runs/996031882?check_suite_focus=true