Correlated Residuals - The Big Bad Wolf of Statistics? In a probability and statistics class, we are often told about the requirement of residuals being uncorrelated when fitting a regression model.
We are often told that the reason for this requirement is that the absence of this usually suggests low quality data.
But mathematically, is there a mathematical reason as to why residuals need to be uncorrelated? What happens if the residuals are in fact correlated?
Can someone please comment on this?

The requirement of residuals being uncorrelated is typically a heuristic, which is based on the following chain of assumptions:

*

*If the regression model itself can be assumed to be approximately linear (actually affine due to the constant term), and the errors of deviation from this model are uncorrelated with zero mean and equal variance, then according to the Gauss-Markov theorem, the ordinary least squares (OLS) estimator will produce the vector of unbiased regression coefficients with lowest variance "envelope", within the class of linear unbiased estimators.


*Since the model is assumed to be approximately linear (affine), then the residuals from using the estimated vector of regression coefficients will be linearly related to the vector of residuals, and these residuals should also enjoy small uncorrelated values.
Please note that the residuals computed from an OLS estimate in a linear (affine) model will not be identically distributed, due to the effect of Leverage.
Without the "nice" linearity and OLS assumptions mentioned above, i.e., for any general perhaps nonlinear model and/or nonlinear estimation procedure, the requirement of residuals being uncorrelated is typically a heuristic based on the notion of having extracted the most information out of the data. Uncorrelatedness is a heuristic proxy for this notion, not a rigorous mathematical one.
