TabPFGen -- Tabular Data Generation with TabPFN
<|reference_start|>TabPFGen -- Tabular Data Generation with TabPFN: Advances in deep generative modelling have not translated well to tabular data. We argue that this is caused by a mismatch in structure between popular generative models and discriminative models of tabular data. We thus devise a technique to turn TabPFN -- a highly performant transformer initially designed for in-context discriminative tabular tasks -- into an energy-based generative model, which we dub TabPFGen. This novel framework leverages the pre-trained TabPFN as part of the energy function and does not require any additional training or hyperparameter tuning, thus inheriting TabPFN's in-context learning capability. We can sample from TabPFGen analogously to other energy-based models. We demonstrate strong results on standard generative modelling tasks, including data augmentation, class-balancing, and imputation, unlocking a new frontier of tabular data generation.<|reference_end|>
@article{ma2024tabpfgen,
    title={TabPFGen -- Tabular Data Generation with TabPFN},
    author={Junwei Ma, Apoorv Dankar, George Stein, Guangwei Yu, Anthony Caterini},
    journal={arXiv preprint arXiv:2406.05216},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.05216},
    primaryClass={cs.LG}
}