Systematic Assessment of Tabular Data Synthesis Algorithms
<|reference_start|>Systematic Assessment of Tabular Data Synthesis Algorithms: Data synthesis has been advocated as an important approach for utilizing data while protecting data privacy. A large number of tabular data synthesis algorithms (which we call synthesizers) have been proposed. Some synthesizers satisfy Differential Privacy, while others aim to provide privacy in a heuristic fashion. A comprehensive understanding of the strengths and weaknesses of these synthesizers remains elusive due to drawbacks in evaluation metrics and missing head-to-head comparisons of newly developed synthesizers that take advantage of diffusion models and large language models with state-of-the-art marginal-based synthesizers.   In this paper, we present a systematic evaluation framework for assessing tabular data synthesis algorithms. Specifically, we examine and critique existing evaluation metrics, and introduce a set of new metrics in terms of fidelity, privacy, and utility to address their limitations. Based on the proposed metrics, we also devise a unified objective for tuning, which can consistently improve the quality of synthetic data for all methods. We conducted extensive evaluations of 8 different types of synthesizers on 12 real-world datasets and identified some interesting findings, which offer new directions for privacy-preserving data synthesis.<|reference_end|>
@article{du2024systematic,
    title={Systematic Assessment of Tabular Data Synthesis Algorithms},
    author={Yuntao Du, Ninghui Li},
    journal={arXiv preprint arXiv:2402.06806},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.06806},
    primaryClass={cs.CR cs.DB cs.LG}
}