Probability Calibration, what causes simple linear models like logistic regression to be over-confident and diverge from the true class probabilities? I've recently studied probability calibration and have investigated many examples and revisited old models of mine to find that they are all poorly calibrated. The idea of stacking also has been cast in a new light.
However intuitive the "calibration" process is, I find it hard to understand why even very simple models become so over-confident.
I'd be super grateful if you can share some insights into how this happens, as I would have guessed that many simpler (especially linear) models would reflect the true class probabilities.
Many thanks in advance!

Suppose there are for example many noise covariates and a very small sample. The linear model will by chance estimate nonzero coefficients on some of these (overfit).  This can lead to higher confidence in the class probabilities.
If for example you have 2 variables and both are noise, if the model correctly estimates (doesn't overfit) by chance, you will get zero coefficients on both, and your estimated prob will be 0.5 each.  Suppose you accidentally estimate a nonzero coefficient due to a spurious correlation between the response and a covariate (overfit); the estimated prob will then be greater or less than 0.5 (more confident).
This is a question of bias variance tradeoff, I expect, which varies by sample size, signal, etc. In other words, a linear model might be a complex model for a dataset without much signal; model complexity is relative, not absolute - a fact that is often overlooked in the applied literature.
