Unmasking Trees for Tabular Data
<|reference_start|>Unmasking Trees for Tabular Data: Despite much work on advanced deep learning and generative modeling techniques for tabular data generation and imputation, traditional methods have continued to win on imputation benchmarks. We herein present UnmaskingTrees, a simple method for tabular imputation (and generation) employing gradient-boosted decision trees which are used to incrementally unmask individual features. This approach offers state-of-the-art performance on imputation, and on generation given training data with missingness; and it has competitive performance on vanilla generation. To solve the conditional generation subproblem, we propose a tabular probabilistic prediction method, BaltoBot, which fits a balanced tree of boosted tree classifiers. Unlike older methods, it requires no parametric assumption on the conditional distribution, accommodating features with multimodal distributions; unlike newer diffusion methods, it offers fast sampling, closed-form density estimation, and flexible handling of discrete variables. We finally consider our two approaches as meta-algorithms, demonstrating in-context learning-based generative modeling with TabPFN.<|reference_end|>
@article{mccarter2024unmasking,
    title={Unmasking Trees for Tabular Data},
    author={Calvin McCarter},
    journal={arXiv preprint arXiv:2407.05593},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.05593},
    primaryClass={cs.LG stat.ML}
}