ity datasets becomes more and more important.
The proposed multi-modal LLMs usually construct
their own instruction-tuning datasets based on man-
ually designed (Dai et al., 2023; Zhu et al., 2023) or
GPT-aided instructions (Liu et al., 2023; Zhao et al.,
2023b) and data collected from benchmark adapta-
tion (Zhang et al., 2023a; Gao et al., 2023) or self-
instruction (Pi et al., 2023; Yang et al., 2023). It is
interesting to see the impacts of multimodal data
management on the performance of fine-tuned mul-
timodal LLMs such as the data scaling law in mul-
timodal instruction fine-tuning, the quality-control
techniques in multimodal dataset construction, the
impact of task balancing in multitask multimodal
training, and so on.
DataManagementForLargeLanguageModels-ASurvey