Does logistic regression try to predict the true conditional P(Y|X)? Consider a binary classification dataset (X, Y), generated according to some unknown distribution $P(X, Y)$. I have a question about models which output probabilities by minimizing the cross-entropy loss (logistic regression and deep models using a final softmax layer).

*

*do these models attempt to predict the true conditional probability $P(Y|X)$?

*or do they aim for a weaker result, like for example trying to get the order between the classes right?


Minimizing crossentropy loss is equivalent to maximum likelihood estimation of the regression coefficients and thus to maximum likelihood estimation of the conditional probabilities. (Since the probabilities are a decent function of the regression coefficients, the MLE of the probabilities is that same decent function of the regression coefficient MLE.)
Consequently, yes, the models seek out something like $P(Y\vert X)$, but they go beyond that. When you have multiple classes (more than just two), these models are supposed to seek out the true probabilities of all classes, not just the one with the highest probability.
Neural networks are known for giving poor probability predictions, however, so I will leave a link to a related question of mine about calibration of the probabilities of the non-dominant classes.
