[Good first issue] LXMERT PyTorch Integration tests
The PyTorch implementation of the LXMERT model currently has no integration tests. This is problematic as the behavior can diverge without being noticed.

The [test_modeling_lxmert.py](https://github.com/huggingface/transformers/blob/master/tests/test_modeling_lxmert.py) file should be updated to include integration testing.

An example of a good modeling integration test is visible in the [test_modeling_bert.py#L552-L565](https://github.com/huggingface/transformers/blob/master/tests/test_modeling_bert.py#L552-L565) file:

https://github.com/huggingface/transformers/blob/1809de5165804666ba6c6a02a9d177f6683869cc/tests/test_modeling_bert.py#L552-L565

Some additional tips:
- The test must be marked as slow using the `@slow` decorator, so as to be run *daily*, and not on every commit of every branch/pull request of this repository.
- The test must be decorated with the `@require_torch` decorator so as to only be run in environments using PyTorch.
- A single test is necessary. If you feel like implementing multiple of these, then sharing the same checkpoint would be ideal so as to reduce download time.
@LysandreJik I'll tackle this one if that's cool?<|||||>Hi @jmwoloso! @sadakmed has already contributed a proposal in #9989 that unfortunately slipped through the net - could you give it a look and let me know if that's what you had in mind?<|||||>Hi @LysandreJik! I originally saw #9954 and was going to make mine based upon that, but I see there have been some updates via #9989 so I'll check that out and adjust if needed, but yes, was going to essentially modify the TF integration test to be PT compatible. @sadakmed does mention adding a context manager to deal with the random seed in #10143 so not sure if that is of interest, but the idea is that I'll use the TF implementation and make it PT compatible.<|||||>Hi @jmwoloso, DO you mean TF implementation of lxmert integration test, I committed for both tf [#10052](https://github.com/huggingface/transformers/pull/10052) and pt [#9989](https://github.com/huggingface/transformers/pull/9989) (since Feb nd I still have this open tabs in my mind of something is unfinished, really a pain). Both has the same issue of the inability to hardcode input coz it's too large, other details u already know,  How it's implemented know, I dont think that fixing seeds locally will impact others in different classes, not to mention in other tests, a Context manager is a safe way to get around it, constant input without affecting anything else, nd I saw somewhere in a torch library that they use this technique (so it's not a crazy idea).<|||||>Seems like this issue might be ready to be closed based on @sadakmed previously [merged PR](https://github.com/huggingface/transformers/pull/9989) in July of 21'. 

cc @LysandreJik <|||||>Indeed! Thanks!