Hi @gchhablani ,
I think the following two are interesting ones to go with :

ViT + mBERT - Multilingual Visual Question Answering (WIT/COCO pre-train, test on VQA/GQA)
modifications of CLIP/VQGAN: Scene generation - CLIP + VQGAN, trained on text-scene dataset for scene (movie/landscape) generation / video+text, following the example of VideoBERT

I am just thinking what can be the dataset for scene generation. Should we start looking at VideoBERT as well for possible datasets used there? Are there any references/material which we can start looking into ? Please advise.