<p>The $\mathbf M = \mathbf I-\mathbf X(\mathbf X'\mathbf X)^{-1}\mathbf X'$ matrix is the "annihilator" or "residual maker" matrix associated with matrix $\mathbf X$. It is called "annihilator" because $\mathbf M\mathbf X =0$ (for its own $X$ matrix of course). Is is called "residual maker" because $\mathbf M \mathbf y =\mathbf {\hat e}$, in the regression $\mathbf y = \mathbf X \beta + \mathbf e$.  </p>

<p>It is a symmetric and idempotent matrix. It is used in the proof of the Gauss-Markov theorem.  </p>

<p>Also, it is used in the <a href="http://en.wikipedia.org/wiki/Frisch%E2%80%93Waugh%E2%80%93Lovell_theorem" rel="nofollow noreferrer">Frisch–Waugh–Lovell theorem</a>, from which one can obtain results for the "partitioned regression", that says that in the model (in matrix form)</p>

<p>$$\mathbf y = \mathbf X_1\beta_1 + \mathbf X_2\beta_2 + \mathbf u$$</p>

<p>we have that</p>

<p>$$\hat \beta_1 = (\mathbf X_1'\mathbf M_2\mathbf X_1)^{-1}(\mathbf X_1'\mathbf M_2)\mathbf y $$</p>

<p>Since $\mathbf M_2$ is idempotent we can re-write the above by</p>

<p>$$\hat \beta_1 = (\mathbf X_1'\mathbf M_2\mathbf M_2\mathbf X_1)^{-1}(\mathbf X_1'\mathbf M_2\mathbf M_2)\mathbf y$$</p>

<p>and since $M_2$ is also symmetric we have</p>

<p>$$\hat \beta_1 = ([\mathbf M_2\mathbf X_1]'[\mathbf M_2\mathbf X_1])^{-1}([\mathbf M_2\mathbf X_1]'[\mathbf M_2\mathbf y]$$</p>

<p>But this is the least-squares estimator from the model</p>

<p>$$[\mathbf M_2\mathbf y] = [\mathbf M_2\mathbf X_1]\beta_1 + \mathbf M_2\mathbf u$$</p>

<p>and also $\mathbf M_2\mathbf y$ are the residuals from regressing $\mathbf y$ on the matrix $\mathbf X_2$ only.  </p>

<p>In other words:
1) If we regress $\mathbf y$ on the matrix $\mathbf X_2$ only, and then regress the <em>residuals</em> from this estimation on the matrix $\mathbf M_2\mathbf X_1$ only, the $\hat \beta_1$ estimates we will obtain will be <em>mathematically</em> equal to the estimates we will obtain if we regress $\mathbf y$ on both $\mathbf X_1$ and $\mathbf X_2$ together at the same time, as a usual multiple regression.  </p>

<p>Now, assume that $\mathbf X_1$ is not a matrix but just one regressor, say $\mathbf x_1$. Then $\mathbf M_2 \mathbf x_1$ is the residuals from regressing the variable $X_1$ on the regressor matrix $\mathbf X_2$. And this provides the intuition here: $\hat \beta_1$ gives us the effect that "the part of $X_1$ that is unexplained by $\mathbf X_2$" has on "the part of $Y$ that is left unexplained by $\mathbf X_2$".</p>

<p>This is an emblematic part of classic Least-Squares Algebra.</p>

CC BY-SA 3.0
2015-01-28T20:58:36.720Z
Started answering, but I had a lot of overlap with this answer.  You can find a lot of this information in Chapter 3.2.4 of the 7th edition of "Econometric Analysis" by Bill Greene.
cc7768
CC BY-SA 3.0
2015-01-28T21:01:19.653Z
@cc7768 Yes, that's a good source for least-squares algebra. But don't hesitate to post additional material. For example, essentially my answer covers only the second question of the OP.
Alecos Papadopoulos
CC BY-SA 3.0
2015-01-30T06:29:49.593Z
@AlecosPapadopoulos you say that if we regress $\mathbf M_2y$ on $\mathbf X_1$, we also get $\hat \beta_1$. But isn't the equation saying, regress $\mathbf M_2y$ on $\mathbf M_2\mathbf X_1$ instead?
Heisenberg
CC BY-SA 3.0
2015-01-30T07:02:12.477Z
@Heisenberg Correct. Typo. Fixed it, and added a bit more.
Alecos Papadopoulos
CC BY-SA 4.0
2015-01-28T20:55:39.050Z
2018-07-23T22:41:32.737Z
2018-07-23T22:41:32.737Z
Alecos Papadopoulos
<p>In <a href="https://economics.stackexchange.com/questions/3194/what-happens-if-the-control-variables-are-also-endogenous">another question of mine</a>, an answerer used the following derivation of OLS coefficient:</p>

<blockquote>
  <p>We have a model: $$ Y = X_1 \beta + X_2 \beta_2 + Z \gamma + \varepsilon, $$ where $Z$ is unobserved. Then we have: $$\text{plim}\, \hat \beta_{1} = \beta_1 + \gamma \frac{Cov(X_1^*, Z)}{Var(X_1^*)} = \beta_1, $$ where $X_1^* = M_2 X_1$ and $M_2 = [I - X_2(X_2'X_2)^{-1}X_2']$.</p>
</blockquote>

<p>This looks different from the usual $\beta = (X'X)^{-1}X'Y$ that I've seen in Econometrics. Is there a more explicit exposition of this derivation? Is there a name for the $M_2$ matrix?</p>

CC BY-SA 3.0
2015-01-28T20:13:01.790Z
Im pretty sure its described in Hansen's lecture notes, but I don't have them at my hands right now.
FooBar
<econometrics><regression>
Alternative way of deriving OLS coefficients
Heisenberg
# Alternative way of deriving OLS coefficients

(Asked by: username_0 on 2015-01-28)

In [another question of mine](https://economics.stackexchange.com/questions/3194/what-happens-if-the-control-variables-are-also-endogenous), an answerer used the following derivation of OLS coefficient:

> We have a model: $$ Y = X\_1 \beta + X\_2 \beta\_2 + Z \gamma + \varepsilon, $$ where $Z$ is unobserved. Then we have: $$\text{plim}\, \hat \beta\_{1} = \beta\_1 + \gamma \frac{Cov(X\_1^\*, Z)}{Var(X\_1^\*)} = \beta\_1, $$ where $X\_1^\* = M\_2 X\_1$ and $M\_2 = [I - X\_2(X\_2'X\_2)^{-1}X\_2']$.

This looks different from the usual $\beta = (X'X)^{-1}X'Y$ that I've seen in Econometrics. Is there a more explicit exposition of this derivation? Is there a name for the $M\_2$ matrix?

**Tags:** econometrics, regression

> **Comments:**
> **username_1**: Im pretty sure its described in Hansen's lecture notes, but I don't have them at my hands right now.  

---

## 1 answers

### Answer by username_2 on 2015-01-28. Score: 10 | Accepted answer

The $\mathbf M = \mathbf I-\mathbf X(\mathbf X'\mathbf X)^{-1}\mathbf X'$ matrix is the "annihilator" or "residual maker" matrix associated with matrix $\mathbf X$. It is called "annihilator" because $\mathbf M\mathbf X =0$ (for its own $X$ matrix of course). Is is called "residual maker" because $\mathbf M \mathbf y =\mathbf {\hat e}$, in the regression $\mathbf y = \mathbf X \beta + \mathbf e$.

It is a symmetric and idempotent matrix. It is used in the proof of the Gauss-Markov theorem.

Also, it is used in the [Frisch–Waugh–Lovell theorem](http://en.wikipedia.org/wiki/Frisch%E2%80%93Waugh%E2%80%93Lovell_theorem), from which one can obtain results for the "partitioned regression", that says that in the model (in matrix form)

$$\mathbf y = \mathbf X\_1\beta\_1 + \mathbf X\_2\beta\_2 + \mathbf u$$

we have that

$$\hat \beta\_1 = (\mathbf X\_1'\mathbf M\_2\mathbf X\_1)^{-1}(\mathbf X\_1'\mathbf M\_2)\mathbf y $$

Since $\mathbf M\_2$ is idempotent we can re-write the above by

$$\hat \beta\_1 = (\mathbf X\_1'\mathbf M\_2\mathbf M\_2\mathbf X\_1)^{-1}(\mathbf X\_1'\mathbf M\_2\mathbf M\_2)\mathbf y$$

and since $M\_2$ is also symmetric we have

$$\hat \beta\_1 = ([\mathbf M\_2\mathbf X\_1]'[\mathbf M\_2\mathbf X\_1])^{-1}([\mathbf M\_2\mathbf X\_1]'[\mathbf M\_2\mathbf y]$$

But this is the least-squares estimator from the model

$$[\mathbf M\_2\mathbf y] = [\mathbf M\_2\mathbf X\_1]\beta\_1 + \mathbf M\_2\mathbf u$$

and also $\mathbf M\_2\mathbf y$ are the residuals from regressing $\mathbf y$ on the matrix $\mathbf X\_2$ only.

In other words:
1) If we regress $\mathbf y$ on the matrix $\mathbf X\_2$ only, and then regress the *residuals* from this estimation on the matrix $\mathbf M\_2\mathbf X\_1$ only, the $\hat \beta\_1$ estimates we will obtain will be *mathematically* equal to the estimates we will obtain if we regress $\mathbf y$ on both $\mathbf X\_1$ and $\mathbf X\_2$ together at the same time, as a usual multiple regression.

Now, assume that $\mathbf X\_1$ is not a matrix but just one regressor, say $\mathbf x\_1$. Then $\mathbf M\_2 \mathbf x\_1$ is the residuals from regressing the variable $X\_1$ on the regressor matrix $\mathbf X\_2$. And this provides the intuition here: $\hat \beta\_1$ gives us the effect that "the part of $X\_1$ that is unexplained by $\mathbf X\_2$" has on "the part of $Y$ that is left unexplained by $\mathbf X\_2$".

This is an emblematic part of classic Least-Squares Algebra.

> **Comments:**
> **username_3**: Started answering, but I had a lot of overlap with this answer.  You can find a lot of this information in Chapter 3.2.4 of the 7th edition of "Econometric Analysis" by Bill Greene.  
> **username_2**: @username_3 Yes, that's a good source for least-squares algebra. But don't hesitate to post additional material. For example, essentially my answer covers only the second question of the OP.  
> **username_0**: @username_2 you say that if we regress $\mathbf M_2y$ on $\mathbf X_1$, we also get $\hat \beta_1$. But isn't the equation saying, regress $\mathbf M_2y$ on $\mathbf M_2\mathbf X_1$ instead?  
> **username_2**: @username_0 Correct. Typo. Fixed it, and added a bit more.  

---

