Given a regression model, is there a way to evaluate confidence of a prediction? What are methods for evaluating the predictive probability or something equivalent in a regression problem? In classification problems a predictive probability will be outputted given an input. For example in a neural network or a logistic regression if the problem is to classify a cat or a dog given a flattened image as a vector input into the model, it would output 70% dog and 30% cat and that could be used as a way to know how "confident" the model is in it's prediction. What is the equivalent in a non-classification regression problem and how is it calculated?

A simpler answer -
you could communicate confidence in a regression prediction using any of the following.
For clarity, let's say your regression model is

*

*$\mu = \alpha + \beta X$ (Expected value)

*$y = \mu + \epsilon$ (Observed values are expected values plus residuals)

*$\epsilon \sim N(0, \sigma)$ (Residuals are Normally distributed with SD $\sigma$)

You could report

*

*Mean absolute error, mean(abs(ε)) : In the training data, how far, on average, was each point from the regression line.

*Mean squared error, $\sigma$: In the training data, what's the standard deviation of distances from the line.

*Confidence Interval: Reflects uncertainty in the the expected value $\mu$.

*Prediction Interval: Reflects both uncertainty in $\mu$, and the spread of the data due to $\sigma$.

