strategies. Therefore, this survey aims to provide a
comprehensive overview of current research in data
management as shown in Figure 1. In Section 2, we
focus on pretraining data management, including
the research on data quantity, data quality, domain
composition, and data management systems. In
Section 3, we discuss the data quantity, data qual-
ity, task composition, and data-efficient learning in
the SFT stage of LLMs. In Section 4, looking into
the future, we present the existing challenges and
promising future directions in training data man-
agement for LLMs. Through this survey, we are
devoted to offering a guiding resource to practi-
tioners attempting to build powerful LLMs with
effective and efficient data management practices.

2 Pretraining of LLM
DataManagementForLargeLanguageModels-ASurvey