<p>To prove the unbiasedness of <span class="math-container">$\hat{\sigma}^2$</span> when conditioned on <span class="math-container">$X$</span>, that is, <span class="math-container">$\mathbb{E}(\hat{\sigma}^2|X)=\sigma^2$</span>, consider <span class="math-container">$\hat{u}= y- X \hat{\beta}=y-X(X'X)^{-1} X'y = [I_n -X(X'X)^{-1} X'X]y = My$</span>, and using <span class="math-container">$y=X\beta +u$</span>, write <span class="math-container">$\hat{u}=Mu$</span>, since <span class="math-container">$MX=0$</span>.</p>
<p>Just a remark about the <span class="math-container">$M$</span> matrix: <span class="math-container">$M$</span> is a matrix that is both symmetric (<span class="math-container">$M=M'$</span>) and idempotent (<span class="math-container">$M=M^2$</span>). The matrix <span class="math-container">$M$</span>, named the residual maker, can be interpreted as a matrix producing the (column) vector of least squares residuals  in the regression of <span class="math-container">$y$</span> on <span class="math-container">$X$</span> when it premultiplies any vector <span class="math-container">$y$</span>.</p>
<p>Then,
<span class="math-container">\begin{equation*}
    \hat{u}' \hat{u} = u'M' M u = u' M u
\end{equation*}</span>
Thus, <span class="math-container">$u' M u$</span> is a scalar and equals its trace that we denote as <span class="math-container">$tr(u' M u)$</span>.</p>
<p>Then, recall that tr(ABC)=tr(BCA), and under the homoskedasticity assumption <span class="math-container">$\mathbb{E}(uu'|X)=\sigma^2$</span>,
<span class="math-container">\begin{align*}
    \mathbb{E}(u'M u |X) = &amp; \mathbb{E}(tr(u'Mu|X)) = \mathbb{E}[tr(Muu')|X] \\
    = &amp; tr[M \mathbb{E}(u u'|X)] = tr(M \sigma^2 I_n) \\
    = &amp; \sigma^2 tr(M)= \sigma^2 (n-k-1)
    \end{align*}</span>
The last equality comes from the fact that <span class="math-container">$tr(M)=tr(I_n)-tr[X(X'X)^{-1} X'] = n - tr[(X'X)^{-1}X'X]$</span>, and this yelds <span class="math-container">$n-tr(I_{k+1})=n-k-1$</span>. Eventually,
<span class="math-container">\begin{equation}
    \mathbb{E}(\hat{\sigma}^2|X)= \mathbb{E}(u'Mu|X)/(n-k-1)=\sigma^2
\end{equation}</span></p>
<p>To find references where the properties of trace in linear algebra are discussed in a simple manner, consult the appendix of &quot;Introductory Econometrics: A Modern Approach&quot; by Wooldridge (latest edition)</p>

CC BY-SA 4.0
2023-12-22T10:27:36.053Z
2023-12-22T10:38:50.190Z
2023-12-22T10:38:50.190Z
Tony
<p>Consider the following linear model: <span class="math-container">$Y = X\beta + u$</span>
.</p>
<p>If Gauss-Markov assumptions hold, I'm trying to prove that <span class="math-container">$\hat{\sigma}^2 = \frac{\hat{u}'\hat{u}}{N-K-1}$</span> is an unbiased estimator for the error variance <span class="math-container">$\sigma^2$</span>. How can I prove this?</p>
<p>My attempt:</p>
<p>I tried with:</p>
<p><span class="math-container">$E(\hat{\sigma}^2) = \frac{E(\hat{u}'\hat{u})}{N-K-1} =\frac{E[(Y- X\hat{\beta})'(Y- X\hat{\beta})]}{N-K-1} = \frac{E(Y'Y) - E(Y'X \hat{\beta}) -E(\hat{\beta}X'Y + E(\hat{\beta}X'X \hat{\beta})}{N-K-1}$</span>.</p>
<p>How can I calculate those expectations?</p>

CC BY-SA 4.0
2023-12-22T10:10:05.343Z
Pls, see the updated question above
Mark
<econometrics><regression>
Linear Regression Model and estimators
Mark
# Linear Regression Model and estimators

(Asked by: username_0 on 2023-12-22)

Consider the following linear model: $Y = X\beta + u$
.

If Gauss-Markov assumptions hold, I'm trying to prove that $\hat{\sigma}^2 = \frac{\hat{u}'\hat{u}}{N-K-1}$ is an unbiased estimator for the error variance $\sigma^2$. How can I prove this?

My attempt:

I tried with:

$E(\hat{\sigma}^2) = \frac{E(\hat{u}'\hat{u})}{N-K-1} =\frac{E[(Y- X\hat{\beta})'(Y- X\hat{\beta})]}{N-K-1} = \frac{E(Y'Y) - E(Y'X \hat{\beta}) -E(\hat{\beta}X'Y + E(\hat{\beta}X'X \hat{\beta})}{N-K-1}$.

How can I calculate those expectations?

**Tags:** econometrics, regression

> **Comments:**
> **username_0**: Pls, see the updated question above  

---

## 1 answers

### Answer by username_1 on 2023-12-22. Score: 3 | Accepted answer

To prove the unbiasedness of $\hat{\sigma}^2$ when conditioned on $X$, that is, $\mathbb{E}(\hat{\sigma}^2|X)=\sigma^2$, consider $\hat{u}= y- X \hat{\beta}=y-X(X'X)^{-1} X'y = [I\_n -X(X'X)^{-1} X'X]y = My$, and using $y=X\beta +u$, write $\hat{u}=Mu$, since $MX=0$.

Just a remark about the $M$ matrix: $M$ is a matrix that is both symmetric ($M=M'$) and idempotent ($M=M^2$). The matrix $M$, named the residual maker, can be interpreted as a matrix producing the (column) vector of least squares residuals in the regression of $y$ on $X$ when it premultiplies any vector $y$.

Then,
\begin{equation\*}
\hat{u}' \hat{u} = u'M' M u = u' M u
\end{equation\*}
Thus, $u' M u$ is a scalar and equals its trace that we denote as $tr(u' M u)$.

Then, recall that tr(ABC)=tr(BCA), and under the homoskedasticity assumption $\mathbb{E}(uu'|X)=\sigma^2$,
\begin{align\*}
\mathbb{E}(u'M u |X) = & \mathbb{E}(tr(u'Mu|X)) = \mathbb{E}[tr(Muu')|X] \\
= & tr[M \mathbb{E}(u u'|X)] = tr(M \sigma^2 I\_n) \\
= & \sigma^2 tr(M)= \sigma^2 (n-k-1)
\end{align\*}
The last equality comes from the fact that $tr(M)=tr(I\_n)-tr[X(X'X)^{-1} X'] = n - tr[(X'X)^{-1}X'X]$, and this yelds $n-tr(I\_{k+1})=n-k-1$. Eventually,
\begin{equation}
\mathbb{E}(\hat{\sigma}^2|X)= \mathbb{E}(u'Mu|X)/(n-k-1)=\sigma^2
\end{equation}

To find references where the properties of trace in linear algebra are discussed in a simple manner, consult the appendix of "Introductory Econometrics: A Modern Approach" by Wooldridge (latest edition)


---

