For a choice question, as shown in Fig. 6 (c), the options
in “Task-specific Prompt” contain the random permutations
of ground truth response (GTR), image captions generated
by BLIP2 [73], GTR from other images, rewrites of GTR by
Qwen-14B [5]. See details in Appendix. For “task-specific
responses”, it is the GTR. This design is to train LLM to
improve its LoT selection ability. For a ranking question,
as shown in Fig. 6 (d), it is to enforce LLM to rank multi-
ple distinct responses of a given input to match their human
preferences. By training on the choice and ranking ques-
tions, LLM is encouraged to distinguish LoT responses and
align human creative preferences, improving its LoT dis-
criminative selection and ranking abilities.
(2) Associable Instruction Learning. By using the above
instruction templates, we augment the 130,000 samples in
the Oogiri-GO dataset to more than 500,000 instructions
whose formulation is in Fig. 5. During training, LLM is
Let’sThinkOutsidetheBox