Fix Blip-2 CI again
# What does this PR do?

The fix added in #21566 have to be applied to a later merged commit #21624
_The documentation is not available anymore as the PR was closed or merged._<|||||>@gante Don't worry. You added 2 new tests, and we just need to use FP16 (for those 2 new tests) to avoid GPU OOM. The original 2 tests are not undo by your PR.<|||||>Oh, I see! haha that makes more sense now :)