Scalable Subsampling Inference for Deep Neural Networks
<|reference_start|>Scalable Subsampling Inference for Deep Neural Networks: Deep neural networks (DNN) has received increasing attention in machine learning applications in the last several years. Recently, a non-asymptotic error bound has been developed to measure the performance of the fully connected DNN estimator with ReLU activation functions for estimating regression models. The paper at hand gives a small improvement on the current error bound based on the latest results on the approximation ability of DNN. More importantly, however, a non-random subsampling technique--scalable subsampling--is applied to construct a `subagged' DNN estimator. Under regularity conditions, it is shown that the subagged DNN estimator is computationally efficient without sacrificing accuracy for either estimation or prediction tasks. Beyond point estimation/prediction, we propose different approaches to build confidence and prediction intervals based on the subagged DNN estimator. In addition to being asymptotically valid, the proposed confidence/prediction intervals appear to work well in finite samples. All in all, the scalable subsampling DNN estimator offers the complete package in terms of statistical inference, i.e., (a) computational efficiency; (b) point estimation/prediction accuracy; and (c) allowing for the construction of practically useful confidence and prediction intervals.<|reference_end|>
@article{wu2024scalable,
    title={Scalable Subsampling Inference for Deep Neural Networks},
    author={Kejin Wu and Dimitris N. Politis},
    journal={arXiv preprint arXiv:2405.08276},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.08276},
    primaryClass={stat.ML cs.LG stat.CO}
}