5

Associable discriminationAssociable GenerationOriginal Instruction(a)Please read thepicture and writea funny sentence.Let's think outsidethe box.Is my shadow standing up?∑(°A°ノ)ノPlease read thepicture and writea funny sentence.Let's think outsidethe box.Is my shadow standing up?∑(°A°ノ)ノ(b)Instruction with ConditionConditon: shadowA.aaInstruction for Selection(c)Please select theoption that ......:Let's think outside the box.Options: A.aa  B.bb C.cc(d)Please rank theoption that ......Let's think outside the box.Options: A.aa  B.bb C.ccInstruction for RankingRank1:B  Rank2:A  Rank3:CModel

GPT4v [71]
LLaVA-1.5 [34]
MiniGPT-v2 [35]
mPLUG-OwlMultilingual [12]
VisualGLM-6B [74]
Qwen-VL [1]
Qwen-VL+AIT (Ours)
Qwen-VL+CLoT (Ours)

Size

-

3T1
19.3
13B 13.2
7B
6.1
7B 28.1
6B 24.1
7B 30.2
7B 39.7
7B 41.8

Image&Text to Text (IT2T)
5T2 Rank Avg.
4T1
23.5
3.2
14.9
27.2
13.9
13.7
18.6
4.0
3.4
26.0
10.5
32.2
30.9
9.7
22.5
33.6
10.4
26.0
38.9
40.4+ 6.8
15.7
21.6
42.7+ 9.1
38.7
Let’sThinkOutsidetheBox