RandALO: Out-of-sample risk estimation in no time flat
<|reference_start|>RandALO: Out-of-sample risk estimation in no time flat: Estimating out-of-sample risk for models trained on large high-dimensional datasets is an expensive but essential part of the machine learning process, enabling practitioners to optimally tune hyperparameters. Cross-validation (CV) serves as the de facto standard for risk estimation but poorly trades off high bias ($K$-fold CV) for computational cost (leave-one-out CV). We propose a randomized approximate leave-one-out (RandALO) risk estimator that is not only a consistent estimator of risk in high dimensions but also less computationally expensive than $K$-fold CV. We support our claims with extensive simulations on synthetic and real data and provide a user-friendly Python package implementing RandALO available on PyPI as randalo and at https://github.com/cvxgrp/randalo.<|reference_end|>
@article{nobel2024randalo:,
    title={RandALO: Out-of-sample risk estimation in no time flat},
    author={Parth T. Nobel, Daniel LeJeune, Emmanuel J. Cand`es},
    journal={arXiv preprint arXiv:2409.09781},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.09781},
    primaryClass={math.ST cs.LG math.OC stat.CO stat.ML stat.TH}
}