If a fitted OLS regression model is mis-specified, is it possible to produce a second model that is unbiased? Let’s say I want to build a linear regression model to conduct some sort of statistical inference. I plan on using the least squares method to fit the model. My understanding is that you need to specify a functional form of the model without looking at your Y values. If you were to look at the Y values your model would be biased. So let’s say I make my best guess as to what the functional form of the true data generating process is. Then I fit the model. Then I look at some diagnostic plots to test the least squares method assumptions. I see that the assumptions are not met. Say the residuals exhibit heteroskedasticity. At this point it is clear that I don’t have a great model. I could use this information to devise a new model. But if I were to fit that, the model would be biased.
Is it possible to build an unbiased model at this point? If so, what are my options? Or should I just accept that any new models I devise will be biased?

The basic linear regression model is $y=X\beta+\epsilon$, with $\epsilon\sim N(0, \sigma^2I)$. We also assume that $\forall i\neq j, Cov(\epsilon_i,\epsilon_j)=0$.
There's a few methods for assessing model assumptions, some of them are explained here. I'll try to give you some key ideas, per violation, but first let us define some terms:

*

*Projection matrix of $X$: $P_X=X(X^TX)^{-1}X^T$

*Error: $e_i=y_i-\hat{y}_i$

*Standardized error: $e^*_i=e_i/\sigma\sqrt{I_n-P_X}$
So, possible violations:

*

*$E[e]\neq 0$

*Non-linear model

*$Var(\epsilon)\neq \sigma^2I_n$

*Non-normal error

A common method for diagnosing model violations is using plots. A very handy plot is the SRVF (standardized residuals vs. fit): We can see whether violations 1,2,3 occur:

Another nice plot is the Q-Q plot, from which we can deduce regarding violation 4.
Now, what do we do if a violation was found?

*

*$E[e]\neq 0$ - adapt your intercept.

*Non-linear model - for each covariate $x_j$, plotting $y_i$ vs. $x_j$. If the relation seems like (for example) $y_i \propto x_2^2$, simply add $x_2^2$ as another covariate. Should you end up adding this covariate, you might want to consider using a variable selection method.

*$Var(\epsilon)\neq \sigma^2I_n$ - use a variance-stabilizing transformation.

*Non-normal error - same as 3.

You might also like to test for outliers and/or influential points in your data and handle them properly.
