Early Stopping Based on Repeated Significance
<|reference_start|>Early Stopping Based on Repeated Significance: For a bucket test with a single criterion for success and a fixed number of samples or testing period, requiring a $p$-value less than a specified value of $\alpha$ for the success criterion produces statistical confidence at level $1 - \alpha$. For multiple criteria, a Bonferroni correction that partitions $\alpha$ among the criteria produces statistical confidence, at the cost of requiring lower $p$-values for each criterion. The same concept can be applied to decisions about early stopping, but that can lead to strict requirements for $p$-values. We show how to address that challenge by requiring criteria to be successful at multiple decision points.<|reference_end|>
@article{bax2024early,
    title={Early Stopping Based on Repeated Significance},
    author={Eric Bax, Arundhyoti Sarkar, and Alex Shtoff},
    journal={arXiv preprint arXiv:2408.00908},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.00908},
    primaryClass={stat.ME cs.LG}
}