title: Large Language Models for Data Synthesis
summary: Generating synthetic data that faithfully captures the statistical structure
  of real-world distributions is a fundamental challenge in data modeling.
  Classical approaches often depend on strong parametric assumptions or manual
  structural design and struggle in high-dimensional or heterogeneous domains.
  Recent progress in Large Language Models (LLMs) reveals their potential as
  flexible, high-dimensional priors over real-world distributions. However, when
  applied to data synthesis, standard LLM-based sampling is inefficient,
  constrained by fixed context limits, and fails to ensure statistical alignment.
  Given this, we introduce LLMSynthor, a general framework for data synthesis
  that transforms LLMs into structure-aware simulators guided by distributional
  feedback. LLMSynthor treats the LLM as a nonparametric copula simulator for
  modeling high-order dependencies and introduces LLM Proposal Sampling to
  generate grounded proposal distributions that improve sampling efficiency
  without requiring rejection. By minimizing discrepancies in the summary
  statistics space, the iterative synthesis loop aligns real and synthetic data
  while gradually uncovering and refining the latent generative structure. We
  evaluate LLMSynthor in both controlled and real-world settings using
  heterogeneous datasets in privacy-sensitive domains (e.g., e-commerce,
  population, and mobility) that encompass both structured and unstructured
  formats. The synthetic data produced by LLMSynthor shows high statistical
  fidelity, practical utility, and cross-data adaptability, positioning it as a
  valuable tool across economics, social science, urban studies, and beyond.
paper_authors._id: 6832c2c8ba29b909f4013a6d
paper_authors.name: Yihong Tang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-26T08:12:00.941Z
paper_authors._id: 67569b1860146dd8c9c8008f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/67569b1860146dd8c9c8008f/f5Tz2yVTry4LGyQE2VC6-.jpeg
paper_authors.fullname: Yihong Tang
paper_authors.type: user
paper_authors.user: HYTYH
paper_authors._id: 6832c2c8ba29b909f4013a6e
paper_authors.name: Menglin Kong
paper_authors._id: 6832c2c8ba29b909f4013a6f
paper_authors.name: Lijun Sun
paper_title: Large Language Models for Data Synthesis
paper_summary: Generating synthetic data that faithfully captures the statistical structure
  of real-world distributions is a fundamental challenge in data modeling.
  Classical approaches often depend on strong parametric assumptions or manual
  structural design and struggle in high-dimensional or heterogeneous domains.
  Recent progress in Large Language Models (LLMs) reveals their potential as
  flexible, high-dimensional priors over real-world distributions. However, when
  applied to data synthesis, standard LLM-based sampling is inefficient,
  constrained by fixed context limits, and fails to ensure statistical alignment.
  Given this, we introduce LLMSynthor, a general framework for data synthesis
  that transforms LLMs into structure-aware simulators guided by distributional
  feedback. LLMSynthor treats the LLM as a nonparametric copula simulator for
  modeling high-order dependencies and introduces LLM Proposal Sampling to
  generate grounded proposal distributions that improve sampling efficiency
  without requiring rejection. By minimizing discrepancies in the summary
  statistics space, the iterative synthesis loop aligns real and synthetic data
  while gradually uncovering and refining the latent generative structure. We
  evaluate LLMSynthor in both controlled and real-world settings using
  heterogeneous datasets in privacy-sensitive domains (e.g., e-commerce,
  population, and mobility) that encompass both structured and unstructured
  formats. The synthetic data produced by LLMSynthor shows high statistical
  fidelity, practical utility, and cross-data adaptability, positioning it as a
  valuable tool across economics, social science, urban studies, and beyond.
paper_ai_summary: LLMSynthor enhances LLMs for efficient and statistically accurate data synthesis through distributional feedback and proposal sampling.
paper_ai_keywords: Large Language Models LLMSynthor nonparametric copula simulator LLM Proposal Sampling summary statistics space synthetic data statistical fidelity practical utility cross-data adaptability