Failing slow RobertaModelIntegrationTest
```
RUN_SLOW=1 pytest tests/test_modeling_roberta.py::RobertaModelIntegrationTest::test_inference_masked_lm
```
Have not investigated at all, but wanted to record.

Traceback:
```
self = <tests.test_modeling_roberta.RobertaModelIntegrationTest testMethod=test_inference_masked_lm>

    @slow
    def test_inference_masked_lm(self):
        model = RobertaForMaskedLM.from_pretrained("roberta-base")

        input_ids = torch.tensor([[0, 31414, 232, 328, 740, 1140, 12695, 69, 46078, 1588, 2]])
        output = model(input_ids)[0]
        expected_shape = torch.Size((1, 11, 50265))
        self.assertEqual(output.shape, expected_shape)
        # compare the actual values for a slice.
        expected_slice = torch.Tensor(
            [[[33.8843, -4.3107, 22.7779], [4.6533, -2.8099, 13.6252], [1.8222, -3.6898, 8.8600]]]
        )
>       self.assertTrue(torch.allclose(output[:, :3, :3], expected_slice, atol=1e-3))
E       AssertionError: False is not true
```
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
