Towards a Theoretical Understanding of Synthetic Data in LLM Post-Training: A Reverse-Bottleneck Perspective
<|reference_start|>Towards a Theoretical Understanding of Synthetic Data in LLM Post-Training: A Reverse-Bottleneck Perspective: Synthetic data has become a pivotal resource in post-training tasks for large language models (LLMs) due to the scarcity of high-quality, specific data. While various methods have been developed to generate synthetic data, there remains a discernible gap between the practical effects of synthetic data and our theoretical comprehension. To address this challenge, we commence by presenting a detailed modeling of the prevalent synthetic data generation process. Building upon this modeling, we demonstrate that the generalization capability of the post-trained model is critically determined by the information gain derived from the generative model, as analyzed from a novel reverse-bottleneck perspective. Moreover, we introduce the concept of Generalization Gain via Mutual Information (GGMI) and elucidate the relationship between generalization gain and information gain. This analysis serves as a theoretical foundation for synthetic data generation and further highlights its connection with the generalization capability of post-trained models, offering an understanding about the design of synthetic data generation techniques and the optimization of the post-training process. We open source our code through an anonymous GitHub repository at https://anonymous.4open.science/r/Understanding-Synthetic.<|reference_end|>
@article{gan2024towards,
    title={Towards a Theoretical Understanding of Synthetic Data in LLM
  Post-Training: A Reverse-Bottleneck Perspective},
    author={Zeyu Gan, Yong Liu},
    journal={arXiv preprint arXiv:2410.01720},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.01720},
    primaryClass={cs.AI cs.CL cs.LG}
}