Could we explain the disadvantage of imbalanced data mathematically? Simple setup:

*

*observed response is binary (yes/no, 0/1, positive/negative).

*use logistic regression to model the probability of the response being, say, 1: $P(Y=1|X)$.

*the MLE of the model coefficients enable us to predict the probability by $\frac{1}{1+exp(-X\hat{\beta})}$.

*By checking ROC and AUC, we find the most appropriate threshold of $p_{0}$ such that $\hat{y}=1$ if $\hat{p}>p_{0}$, otherwise $\hat{y}=0$.

In this process, could we mathematically show that $P(Y=1|X)$ is underestimated if we have 10000 0's and 100 1's? How is this bias quantified in formula?

Not a formal proof, but an intuition: Unbalanced data is not a problem per se, the problem is that you don't have many samples to represent the minority class. Imagine a trivial model, where you would use logistic regression with only the intercept for your data. In such a case, the model would correctly estimate the probability to be $\tfrac{100}{10100}$ (try it yourself on different datasets). Now imagine that you use a more complicated model, the model would start struggling a little bit for the minority class because it doesn't have much data for it. Imagine a different problem, where you take only the minority class data ($n=100$ here) and try fitting some complicated model to this data. It will fail simply because you don't have enough data. For exactly the same reason, your predictions regarding the minority class would be not as precise as with the majority class, because you don't have enough data to represent it.
