<p>Given regression equation Y=Xβ + ε which satisfies all classical assumptions including those of homoscadasticity </p>

<p>Estimated regression equation using OLS is the following - </p>

<p>Y=Xb + e </p>

<p>Where, b is the OLS estimator for β and e is the estimated residual term for equation where E(e|x) =0</p>

<p>Note: b=(X'X)<span class="math-container">$^{-1}$</span>X'Y</p>

<p>We can find, 
Cov(b,e|x) = E(b.e|x) - E(b).E(e|x)</p>

<p>=E(b.e|x) = E((X'X)<span class="math-container">$^{-1}$</span>X'Y.e|x)=E((X'X)<span class="math-container">$^{-1}$</span>X'(Xβ + ε).e|x)</p>

<p>=E((β.e + (X'X)<span class="math-container">$^{-1}$</span>X'ε.e)|x)</p>

<p>=(X'X)<span class="math-container">$^{-1}$</span>X'E(ε.e) </p>

<p>Solving for E(ε.e) gives σ²(n-k) [I wrote e= Y-Xb, substituting for b and some manipulation gave me e= ε + X(X'X)<span class="math-container">$^{-1}$</span>X'ε] </p>

<p>This gives Cov(b,e|x)=(X'X)<span class="math-container">$^{-1}$</span>X'σ²(n-k) </p>

<p>I think in this expression - (X'X)<span class="math-container">$^{-1}$</span>X' - I shouldn't have X' in the right (as the formula of var(b) suggests). What am I missing? </p>

<p><img src="https://i.sstatic.net/hq69d.jpg" alt="Question"></p>

CC BY-SA 4.0
2019-09-10T13:48:32.110Z
In your last equality E(β.e|x + (X'X)−1X'ε.e|x) is kind of vaudou for most humans.
Bertrand
CC BY-SA 4.0
2019-09-10T14:24:15.460Z
@Bertrand Please have a look at the edited question. Does this make sense now?
Elina Gilbert
CC BY-SA 4.0
2019-09-10T19:22:12.457Z
i) You should be careful with the dimensions of your vectors: $b$ is $K \times 1$ and $e$ is $N \times 1$, so the covariance is $K \times N$. It is better to transpose $e$ in $E[be^T]$.   ii) After taking carefully into account the dimensions, it turns out that $E[\varepsilon e^T]$ is a $N \times N$ matrix, and is different from what you mention.
Bertrand
CC BY-SA 4.0
2019-09-10T23:48:45.850Z
@Bertrand Thank you! Doing so, I am getting 0 covariance. Is it right? I mean why does it be so?
Elina Gilbert
CC BY-SA 4.0
2019-09-11T06:41:19.787Z
The 0 answer seems to be right. You may also want to prove it using projections matrices $P_X$ and $M_X$.
Bertrand
CC BY-SA 4.0
2019-09-11T09:44:55.707Z
Thanks! Yes, I redid it using M matrix. It made the proof quite small.
Elina Gilbert
<econometrics><regression><variance>
Covariance between regression coefficient and residual
Elina Gilbert
# Covariance between regression coefficient and residual

(Asked by: username_0 on 2019-09-10)

Given regression equation Y=Xβ + ε which satisfies all classical assumptions including those of homoscadasticity

Estimated regression equation using OLS is the following -

Y=Xb + e

Where, b is the OLS estimator for β and e is the estimated residual term for equation where E(e|x) =0

Note: b=(X'X)$^{-1}$X'Y

We can find,
Cov(b,e|x) = E(b.e|x) - E(b).E(e|x)

=E(b.e|x) = E((X'X)$^{-1}$X'Y.e|x)=E((X'X)$^{-1}$X'(Xβ + ε).e|x)

=E((β.e + (X'X)$^{-1}$X'ε.e)|x)

=(X'X)$^{-1}$X'E(ε.e)

Solving for E(ε.e) gives σ²(n-k) [I wrote e= Y-Xb, substituting for b and some manipulation gave me e= ε + X(X'X)$^{-1}$X'ε]

This gives Cov(b,e|x)=(X'X)$^{-1}$X'σ²(n-k)

I think in this expression - (X'X)$^{-1}$X' - I shouldn't have X' in the right (as the formula of var(b) suggests). What am I missing?

![Question](https://i.sstatic.net/hq69d.jpg)

**Tags:** econometrics, regression, variance

> **Comments:**
> **username_1**: In your last equality E(β.e|x + (X'X)−1X'ε.e|x) is kind of vaudou for most humans.  
> **username_0**: @username_1 Please have a look at the edited question. Does this make sense now?  
> **username_1**: i) You should be careful with the dimensions of your vectors: $b$ is $K \times 1$ and $e$ is $N \times 1$, so the covariance is $K \times N$. It is better to transpose $e$ in $E[be^T]$.   ii) After taking carefully into account the dimensions, it turns out that $E[\varepsilon e^T]$ is a $N \times N$ matrix, and is different from what you mention.  
> **username_0**: @username_1 Thank you! Doing so, I am getting 0 covariance. Is it right? I mean why does it be so?  
> **username_1**: The 0 answer seems to be right. You may also want to prove it using projections matrices $P_X$ and $M_X$.  
> **username_0**: Thanks! Yes, I redid it using M matrix. It made the proof quite small.  

---

## 0 answers

