in both the pretraining and SFT stages can be a
promising solution to hallucination.
Social Biases and Fairness The problem of so-
cial biases and unfairness existing in current pre-
training datasets and their impacts on pre-trained
LLMs have been addressed in several works as dis-
cussed in Section 2.2.4. However, there is still a
large gap between current prominent LLMs and
ideal LLMs without social biases. Many questions
are worth exploring, such as how to mitigate the
potential biases in pretraining datasets, the exis-
tence of bias in the SFT datasets, and whether it is
feasible to reduce social bias through SFT.
Multimodal Data Management Current re-
search in data management mostly focuses on nat-
ural language processing. With the application of
LLMs extending to multimodalities like vision, au-
dio, and so on, the construction of multimodal-
ity datasets becomes more and more important.
The proposed multi-modal LLMs usually construct
DataManagementForLargeLanguageModels-ASurvey