Bootstrap SGD: Algorithmic Stability and Robustness
<|reference_start|>Bootstrap SGD: Algorithmic Stability and Robustness: In this paper some methods to use the empirical bootstrap approach for stochastic gradient descent (SGD) to minimize the empirical risk over a separable Hilbert space are investigated from the view point of algorithmic stability and statistical robustness. The first two types of approaches are based on averages and are investigated from a theoretical point of view. A generalization analysis for bootstrap SGD of Type 1 and Type 2 based on algorithmic stability is done. Another type of bootstrap SGD is proposed to demonstrate that it is possible to construct purely distribution-free pointwise confidence intervals of the median curve using bootstrap SGD.<|reference_end|>
@article{christmann2024bootstrap,
    title={Bootstrap SGD: Algorithmic Stability and Robustness},
    author={Andreas Christmann and Yunwen Lei},
    journal={arXiv preprint arXiv:2409.01074},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.01074},
    primaryClass={stat.ML cs.AI cs.LG}
}