Ranking the Outputs of Statistical Models I have the following question about Ranking the Outputs of Statistical Models.

*

*In the case of a regression model with a continuous response variable : Suppose the model makes predictions on 3 new observations (e.g. failure time for these 3 observations : observation #1 expected to fail in 10 days plus minus 5 days, observation #2 expected to fail in 50 days plus minus 20 days, observation #3 expected to fail in  200 days plus minus 100 days) - based on the confidence intervals of the predicted values of these 3 observations, are we allowed to rank these times? For example, could we say that observation #3 is likely to fail after observation #1?



*

*In the case of a classification model with a categorical response variable: Suppose the model predicts that observation #1 belongs to "class 1" with a probability of 0.9 and that observation #2 belongs to "class 0" with a probability of 0.53 - based on the confidence intervals, are we allowed to rank these probabilities? For example, could we say that the prediction made for observation #1 is more "certain" than the prediction made for observation #2?

My Question: Can the predictions made by statistical models be ranked as the way I described?
Thanks!

Your question has two separate aspects - one is the interpretation of the outputs of statistical models and the other one is the ranking of random variables based on their probability distribution.
It is easiest to answer the kind of questions you are describing in a Bayesian framework, since in that case you can describe everything probabilistically - that is, you actually get a probability distribution for every prediction (as depicted in your plot). so for example if $t_1$ and $t_2$ are the failure times of two observations you can use those distributions to calculate for example $P(t_1 > t_2) = \int_{t_1>t_2}f_1(t_1)f_2(t_2)dt_1dt_2$ which will tell you how likely it is for $t_1$ to happen after $t_2$.
However just as with any random variables there is no unique way of ranking the outputs. You could also calculate for example the expected values $\mathbb E[t_1]$ and $\mathbb E[t_2]$, and it is possible to have at the same time $\mathbb E[t_1] > \mathbb E[t_2]$ and $P(t_2 > t_1) > 1/2$, that is, the expected value of $t_1$ can be larger but $t_2 > t_1$ is more likely than $t_1 > t_2$. So, you will have to decide based on which criterion you want to rank them (this is somewhat similar to the analysis of risk vs. expected value in
finance, see e.g. Mean-variance analysis).
In a classification case where the model outputs probabilities this is more straightforward, and it does makes sense to rank based on the level of certainty. However keep in mind that usually those models don't perform a full Bayesian calculation so there can be issues with the Calibration of those probabilities.
