VQA model inferences
There are a bunch of models fine tuned for VQA and NLVR tasks based on LXMERT, ViLT and Visual BERT, CLIP vision BERT and so on. Is there a resource to see how to run inferences on them ? (and possibly benchmark them )
I tried the LXMERT and VisualBERT demos from [here](https://github.com/huggingface/transformers/blob/master/examples/research_projects/lxmert/demo.ipynb) and [here](https://github.com/huggingface/transformers/blob/master/examples/research_projects/visual_bert/demo.ipynb) 

There seems to be dependency issues and a mismatch with tokeniser version in the Req file. Tried fixing it with no luck. 

Question: We need FRCNN models for LXMERT and visual BERT as both uses RoIs as a supervision signal along with text encoder ? 

Does that mean ViLT based VQA won't need FRCNN as they don't use ROI based supervision ? 

Please clarify  <|||||>Hi,

I'm working on adding the ViLT model, see #14895. It will be much easier to use compared to LXMERT and VisualBERT. It indeed doesn't require an external model like Faster R-CNN. Instead, it just turns an image into a sequence of patches, which are fed to the model (similar to ViT). <|||||>Awesome thanks ! @patil-suraj  was mentioning you are working on this. Closing this for now. 

Looking forward to it !