Ranking the Outputs of Statistical Models I have the following question about Ranking the Outputs of Statistical Models.

*

*In the case of a regression model with a continuous response variable : Suppose the model makes predictions on 3 new observations (e.g. failure time for these 3 observations : observation #1 expected to fail in 10 days plus minus 5 days, observation #2 expected to fail in 50 days plus minus 20 days, observation #3 expected to fail in  200 days plus minus 100 days) - based on the confidence intervals of the predicted values of these 3 observations, are we allowed to rank these times? For example, could we say that observation #3 is likely to fail after observation #1?



*

*In the case of a classification model with a categorical response variable: Suppose the model predicts that observation #1 belongs to "class 1" with a probability of 0.9 and that observation #2 belongs to "class 0" with a probability of 0.53 - based on the confidence intervals, are we allowed to rank these probabilities? For example, could we say that the prediction made for observation #1 is more "certain" than the prediction made for observation #2?

My Question: Can the predictions made by statistical models be ranked as the way I described?
Thanks!


based on the confidence intervals of the predicted values of these 3 observations, are we allowed to rank these times? For example, could we say that observation #3 is likely to fail after observation #1?

The marginal distributions can give you some idea about which variable is larger but you do not know everything. Below you see an example with two different joint distributions for a pair of variables x,y. The marginal distributions are the same but the number of times that you have $x>y$ is not the same.
In the example you get that more often $x>y$ than $x<y$. But in some situations it might be possible that the mean or median of one distribution $x$ is larger than the the mean or median of the distribution of $y$ but that you get more often $y>x$.


For example, could we say that the prediction made for observation #1 is more "certain" than the prediction made for observation #2?

You could indeed say that the predictions are more "certain" if the values are closer to zero or one. A related concept is information entropy which would be computed as
$$H = \sum_{\text{for all classes $i$}} p_i \log p_i = p \log(p) + (1-p) \log(1-p)$$
This means that the model makes a more 'certain' prediction. But, the model could be bad and make a lot of false positive or false negative errors. We may not be certain about the model.
