Solved by ynjiun_wang in post #29 
              
              
                two observations: 


“My model is a simple bi-encoder (vit for the images, bert for the linguisric component) with a late fusion” => if we check the table the “late fusion” performance is 64.75% (if  your model is the same category of this entry, then your 64% already achieved almost the limit of th…