Data plays a fundamental role in the training
of Large Language Models (LLMs). Effective
data management, particularly in the formula-
tion of a well-suited training dataset, holds sig-
nificance for enhancing model performance and
improving training efficiency during pretrain-
ing and supervised fine-tuning phases. Despite
the considerable importance of data manage-
ment, the current research community still falls
short in providing a systematic analysis of the
rationale behind management strategy selec-
tion, its consequential effects, methodologies
for evaluating curated datasets, and the ongoing
pursuit of improved strategies. Consequently,
the exploration of data management has at-
tracted more and more attention among the re-
search community. This survey provides a com-
prehensive overview of current research in data
management within both the pretraining and
supervised fine-tuning stages of LLMs, cover-
ing various noteworthy aspects of data man-
DataManagementForLargeLanguageModels-ASurvey