Density-Regression: Efficient and Distance-Aware Deep Regressor for Uncertainty Estimation under Distribution Shifts
<|reference_start|>Density-Regression: Efficient and Distance-Aware Deep Regressor for Uncertainty Estimation under Distribution Shifts: Morden deep ensembles technique achieves strong uncertainty estimation performance by going through multiple forward passes with different models. This is at the price of a high storage space and a slow speed in the inference (test) time. To address this issue, we propose Density-Regression, a method that leverages the density function in uncertainty estimation and achieves fast inference by a single forward pass. We prove it is distance aware on the feature space, which is a necessary condition for a neural network to produce high-quality uncertainty estimation under distribution shifts. Empirically, we conduct experiments on regression tasks with the cubic toy dataset, benchmark UCI, weather forecast with time series, and depth estimation under real-world shifted applications. We show that Density-Regression has competitive uncertainty estimation performance under distribution shifts with modern deep regressors while using a lower model size and a faster inference speed.<|reference_end|>
@article{bui2024density-regression:,
    title={Density-Regression: Efficient and Distance-Aware Deep Regressor for
  Uncertainty Estimation under Distribution Shifts},
    author={Ha Manh Bui and Anqi Liu},
    journal={arXiv preprint arXiv:2403.05600},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.05600},
    primaryClass={cs.LG stat.ML}
}