Given a regression model, is there a way to evaluate confidence of a prediction? What are methods for evaluating the predictive probability or something equivalent in a regression problem? In classification problems a predictive probability will be outputted given an input. For example in a neural network or a logistic regression if the problem is to classify a cat or a dog given a flattened image as a vector input into the model, it would output 70% dog and 30% cat and that could be used as a way to know how "confident" the model is in it's prediction. What is the equivalent in a non-classification regression problem and how is it calculated?

*

*One method for regression is MDN, mixture density network. The idea is to model the output of the network as mixture of gaussians. i.e. Instead of outputting a single number each time, the network output is several numbers describe the gaussian distribution for example: output $\mu$, $\sigma$ of the normal distribution. Then your goal is to maximize the average log likelihood (i.e. your loss is the negative log likelihood). In this case for each prediction you will get different $\sigma$ that will describe the confidence in the prediction.

You can read more about it here:
https://towardsdatascience.com/a-hitchhikers-guide-to-mixture-density-networks-76b435826cca

*

*Another method which is good both for regression and for classification is bayesian neural network. The basic idea here, is to model the weights as normal distribution, instead of simple numbers.These weights will describe the posterior distribution. Then in order to get the confidence you sample from the posterior weights and calculate the standard deviation of the results.

