Why do we prefer unbiased estimators instead of minimizing MSE? I was thinking about why, usually, $\hat{\sigma}^2=\hat{p}(1-\hat{p})$ is used to estimate the variance in a Bernoulli population instead of $s^2=\hat{p}(1-\hat{p})\frac{n}{n-1}$.
$s^2$ is unbiased, but in some cases would lead to strange situations.
Suppose you find $\hat{p}=0.5$. $s^2$ would be grater than $0.25$, which is obviously wrong (since $0<=\sigma^2<=0.25$).
In this situation, it would be fair to cap $s^2$ at $0.25$ to avoid results that don't make sense at all. This would also reduce the $MSE$ but would make the estimator biased again. Instead, it makes more sense to just use $\hat{\sigma}^2$.
This made me think about why unbiased estimators are usually preferred, even if they have a higher MSE.
I have 3 questions:

*

*Is this the reason why biased $\hat{\sigma}^2$ is usually used instead of udjusted $s^2$ for Bernoulli populations? Are there any other reasons?

*Why would you want to choose an esimator which is further away from the truth on average, just to make it unbiased?

*Why is udjusted $s^2$ preferred over $\hat{\sigma}^2$ for other distributions, even if its $MSE$ is higher? (The estimator that minimizes MSE divides the sum of the squared residuals by $n+1$, but I've never seen it used, why?)

Moreover, $s^2$ is unbiased, but $\sqrt{s^2}$ is biased for estimating the standard deviation, so it isn't that obvious why we should be choosing it (why isn't a different adjusted estimator used
just for the standard deviation?).
I'm editing the question based on the comments I've received:
It seems like the second question is too general because in the majority of cases an unbiased estimator doesn't exists, also there isn't usually a single estimator that minimizes MSE. So, please, consider the second point I've asked only for situations where there is an unbiased estimator which isn't the best one in terms of MSE.

I think one historical reason $s^2$ with an $n-1$ denominator is used is the analogy to regression settings where the denominator is $n-p$, and especially to the Neyman-Scott problem.  The difference between $n$ and $n-1$ doesn't matter much but the difference between $n$ and $n-p$ may.
The Neyman-Scott problem is the extreme case of $n$ pairs of Normal observations: $Y_{ij}=\mu_i+\epsilon_{ij}$, where $\epsilon_{ij}\sim N(0,\sigma^2)$.  In this model, the MLE of $\mu_i$ is the pairwise mean $\bar Y_i$ and of $\sigma^2$ is
$$\hat\sigma^2_\textrm{MLE}=\frac{\sum_i\sum_j (Y_{ij}-\bar Y_{i})^2}{2n}$$
The MLE has expectation $\sigma^2/2$, which is Not Good.
A 'degrees of freedom' correction gives
$$\hat\sigma^2_\textrm{df}=\frac{\sum_i\sum_j (Y_{ij}-\bar Y_{i})^2}{2n-n}$$
which is unbiased, but in this case is clearly better than the MLE.
So, the actual advantage of the df-corrected estimator in various structured experimental designs lends some weight to also using the df-corrected estimator where it doesn't matter.
