Improving the performance of Stein variational inference through extreme sparsification of physically-constrained neural network models
<|reference_start|>Improving the performance of Stein variational inference through extreme sparsification of physically-constrained neural network models: Most scientific machine learning (SciML) applications of neural networks involve hundreds to thousands of parameters, and hence, uncertainty quantification for such models is plagued by the curse of dimensionality. Using physical applications, we show that $L_0$ sparsification prior to Stein variational gradient descent ($L_0$+SVGD) is a more robust and efficient means of uncertainty quantification, in terms of computational cost and performance than the direct application of SGVD or projected SGVD methods. Specifically, $L_0$+SVGD demonstrates superior resilience to noise, the ability to perform well in extrapolated regions, and a faster convergence rate to an optimal solution.<|reference_end|>
@article{padmanabha2024improving,
    title={Improving the performance of Stein variational inference through extreme
  sparsification of physically-constrained neural network models},
    author={Govinda Anantha Padmanabha, Jan Niklas Fuhg, Cosmin Safta, Reese E.
  Jones, Nikolaos Bouklas},
    journal={arXiv preprint arXiv:2407.00761},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.00761},
    primaryClass={cs.LG cs.CE}
}