<p><strong>OLS</strong></p>

<p>The OLS part is clear: <span class="math-container">$\sum_{i=1}^n y_{2i} (y_{1i} - \hat\gamma_{12} y_{2i} - \hat\beta_{11} x_{1i}) = 0$</span> and <span class="math-container">$\sum_{i=1}^n x_{1i} (y_{1i} - \hat\gamma_{12} y_{2i} - \hat\beta_{11} x_{1i}) = 0$</span>, where <span class="math-container">$\hat\gamma_{12}$</span> and <span class="math-container">$\hat\beta_{11}$</span> are the OLS estimators.</p>

<p><strong>2SLS</strong></p>

<p>For 2SLS, I will assume that <span class="math-container">$(x_{1i}, x_{2i}, x_{3i})$</span> is used as IV. The corresponding 2SLS is identical to the IV estimator using <span class="math-container">$\hat{y}_{2i}$</span> and <span class="math-container">$x_{1i}$</span> as IV. Thus, (I think) the "normal equations" are
<span class="math-container">\begin{align}
\sum_{i=1}^n \hat{y}_{2i} (y_{1i} - \tilde\gamma_{12} y_{2i} - \tilde\beta_{11} x_{1i}) &amp;= 0,\\
\sum_{i=1}^n x_{1i} (y_{1i} - \tilde\gamma_{12} y_{2i} - \tilde\beta_{11} x_{1i}) &amp;= 0,
\end{align}</span>
where <span class="math-container">$\tilde\gamma_{12}$</span> and <span class="math-container">$\tilde\beta_{11}$</span> are the 2SLS estimators using <span class="math-container">$x_{1i}, x_{2i}, x_{3i}$</span> as IV, and <span class="math-container">$\hat{y}_{2i}$</span> is the fitted value obtained by regressing <span class="math-container">$y_{2i}$</span> on the instruments <span class="math-container">$x_{1i}$</span>, <span class="math-container">$x_{2i}$</span> and <span class="math-container">$x_{3i}$</span>.</p>

<p>In the above, I understood 2SLS as the IV estimator using <span class="math-container">$(\hat{y}_{2i}, x_{1i})$</span> as IV. We can also understand 2SLS as the OLS estimator of <span class="math-container">$y_{1i}$</span> on <span class="math-container">$(\hat{y}_{2i}, x_{1i})$</span>. In that case, the normal equations can be written as
<span class="math-container">\begin{align}
\sum_{i=1}^n \hat{y}_{2i} (y_{1i} - \tilde\gamma_{12} \hat{y}_{2i} - \tilde\beta_{11} x_{1i}) &amp;= 0,\\
\sum_{i=1}^n x_{1i} (y_{1i} - \tilde\gamma_{12} \hat{y}_{2i} - \tilde\beta_{11} x_{1i}) &amp;= 0,
\end{align}</span>
where <span class="math-container">$y_{2i}$</span> is replaced with <span class="math-container">$\hat{y}_{2i}$</span>. The two sets of normal equations for 2SLS (one in terms of <span class="math-container">$y_{2i}$</span> and the other in terms of <span class="math-container">$\hat{y}_{2i}$</span>) are identical.</p>

<p><strong>Intercept</strong></p>

<p>I have noticed that the intercept is excluded in the question. If the model is <span class="math-container">$y_{1i} = \beta_{10} + \gamma_{12} y_{2i} + \beta_{11} x_{1i} + \epsilon_i$</span> instead, you need equations for the intercept as well.</p>

<p><strong>Matrix notations</strong></p>

<p>Using matrix notations, let <span class="math-container">$y$</span> be the <span class="math-container">$n\times 1$</span> matrix of the LHS variable, <span class="math-container">$X$</span> the <span class="math-container">$n\times 2$</span> matrix of <span class="math-container">$(y_{2i}, x_{1i})$</span> and <span class="math-container">$Z$</span> the <span class="math-container">$n\times 3$</span> matrix of <span class="math-container">$(x_{i1}, x_{i2}, x_{i3})$</span>. Let <span class="math-container">$\beta = (\gamma_{12}, \beta_{11})'$</span>. (I am considering the model without the intercept. For the model with the intercept, <span class="math-container">$X$</span> and <span class="math-container">$Z$</span> contain a column of ones.) Then the OLS estimator is <span class="math-container">$\hat\beta = (X'X)^{-1} X'y$</span>, which solves the normal equations <span class="math-container">$X'(y-X\hat\beta)=\boldsymbol 0$</span>. To see this, note that <span class="math-container">$$X'(y-X\hat\beta)=\boldsymbol 0 \\
X'y - X'X\hat\beta = \boldsymbol 0\\
X'X\hat\beta = X'y\\
\hat\beta = (X'X)^{-1}X'y$$</span></p>

<p>The 2SLS (using <span class="math-container">$Z$</span> as instruments) estimator is <span class="math-container">$$\tilde\beta = [X'Z(Z'Z)^{-1} Z'X]^{-1} X'Z(Z'Z)^{-1} Z'y = (\hat{X}'X)^{-1} \hat{X}'y$$</span> where <span class="math-container">$\hat{X} = Z(Z'Z)^{-1} Z'X$</span>. This 2SLS solves <span class="math-container">$\hat{X}' (y-X\tilde\beta)=\boldsymbol 0$</span>, the normal equations for this 2SLS. Again, to see this, note that 
<span class="math-container">$$\hat{X}' (y-X\tilde\beta)=\boldsymbol 0\\
\hat X'y - \hat X'X\tilde\beta = \boldsymbol 0\\
\hat X'X\tilde\beta = \hat X' y\\
\tilde\beta = (\hat X'X)^{-1} \hat X'y $$</span>
You will see that <span class="math-container">$\hat{X}$</span> is the <span class="math-container">$n\times 2$</span> matrix of <span class="math-container">$(\hat{y}_{2i}, x_{1i})$</span>. It is also true that <span class="math-container">$\tilde\beta = (\hat{X}'\hat{X})^{-1} \hat{X}'y$</span>, and the normal equations are also written as <span class="math-container">$\hat{X}' (y-\hat{X}\tilde\beta)=0$</span>. The two different expressions are identical because <span class="math-container">$\hat{X}'X = \hat{X}'\hat{X}$</span>.</p>

CC BY-SA 4.0
2019-10-12T06:23:37.183Z
Note that $\boldsymbol {X'}(\boldsymbol y-\boldsymbol{X\hat\beta})=\boldsymbol 0$ is a $K\times 1$ vector. This corresponds to the equations at the beginning of the answer.
ahorn
CC BY-SA 4.0
2019-10-12T06:53:18.323Z
I've just found that my $X$ is different from OP's $X$. Sorry.
chan1142
CC BY-SA 4.0
2019-10-12T06:54:09.257Z
Note that $(Z'Z)^{-1}Z'X$ is a matrix of coefficients, so $\hat X = Z(Z'Z)^{-1}Z'X$ is a $N\times 2$ matrix of the predicted values of the regressions of $\boldsymbol {y_2}$ and $\boldsymbol {x_1}$ on $\boldsymbol {x_1}, \boldsymbol {x_2}$ and $\boldsymbol {x_3}$ (known as the "projection"). You can see that $\boldsymbol{\hat x_1} = \boldsymbol {x_1}$ since $\boldsymbol {x_1}, \boldsymbol {x_2}$ and $\boldsymbol {x_3}$ are independent. However, $\boldsymbol {\hat y_2}\neq\boldsymbol {y_2}$.
ahorn
CC BY-SA 4.0
2019-10-12T06:55:05.407Z
no, I didn't pick up any mistakes in your answer.
ahorn
CC BY-SA 4.0
2019-10-12T07:16:27.440Z
You re-defined $X$, which was the correct thing to do (in order to maintain conventional notation).
ahorn
CC BY-SA 4.0
2019-10-12T07:20:22.413Z
Yes, I did. I noticed it after reading your first comment. Hope your comment helps people.
chan1142
CC BY-SA 4.0
2019-10-12T01:51:38.437Z
2019-10-14T12:21:08.933Z
2019-10-14T12:21:08.933Z
chan1142
<p>For a system of equations with <span class="math-container">$M=2$</span> endogenous variables, <span class="math-container">$ Y=\begin{bmatrix}  y_1 &amp; y_2 \end{bmatrix}$</span> and <span class="math-container">$K=3$</span> exogenous variables, <span class="math-container">$X=\begin{bmatrix}  x_1 &amp; x_2 &amp; x_3 \end{bmatrix}$</span>. 
The first equation of the system is given by: 
<span class="math-container">$y_{1i}=\gamma_{12}y_{2i}+\beta_{11}x_{1i}+\epsilon_{1i}$</span>.
The data matrices yield, <span class="math-container">$X'X$</span>, <span class="math-container">$X'Y$</span> and <span class="math-container">$Y'Y$</span>, which are of dimension, <span class="math-container">$3*3$</span>, <span class="math-container">$3*2$</span> and <span class="math-container">$2*2$</span> respectively</p>

<p>I need to </p>

<ul>
<li>Write the OLS and 2SLS normal equations in terms of cross products of the data matrices.</li>
</ul>

<p>I tried using the standard procedure for deriving normal equations, that is, by minimizing sum of squared residuals, but I fail to understand how the equations can be modelled in form of the cross products of the given data matrices. I think I'm missing the point of the exercise. Please help me in understanding what the question demands. Thanks! </p>

CC BY-SA 4.0
2019-10-12T06:20:07.303Z
Note that $x_1, x_2, x_3, y_1$ and $y_2$ are all $n\times 1$ vectors.
ahorn
<econometrics><self-study>
OLS and 2SLS normal equations
S.Rana
# OLS and 2SLS normal equations

(Asked by: username_0 on 2019-10-11)

For a system of equations with $M=2$ endogenous variables, $ Y=\begin{bmatrix} y\_1 & y\_2 \end{bmatrix}$ and $K=3$ exogenous variables, $X=\begin{bmatrix} x\_1 & x\_2 & x\_3 \end{bmatrix}$.
The first equation of the system is given by:
$y\_{1i}=\gamma\_{12}y\_{2i}+\beta\_{11}x\_{1i}+\epsilon\_{1i}$.
The data matrices yield, $X'X$, $X'Y$ and $Y'Y$, which are of dimension, $3\*3$, $3\*2$ and $2\*2$ respectively

I need to

* Write the OLS and 2SLS normal equations in terms of cross products of the data matrices.

I tried using the standard procedure for deriving normal equations, that is, by minimizing sum of squared residuals, but I fail to understand how the equations can be modelled in form of the cross products of the given data matrices. I think I'm missing the point of the exercise. Please help me in understanding what the question demands. Thanks!

**Tags:** econometrics, self-study

> **Comments:**
> **username_1**: Note that $x_1, x_2, x_3, y_1$ and $y_2$ are all $n\times 1$ vectors.  

---

## 1 answers

### Answer by username_2 on 2019-10-12. Score: 1

**OLS**

The OLS part is clear: $\sum\_{i=1}^n y\_{2i} (y\_{1i} - \hat\gamma\_{12} y\_{2i} - \hat\beta\_{11} x\_{1i}) = 0$ and $\sum\_{i=1}^n x\_{1i} (y\_{1i} - \hat\gamma\_{12} y\_{2i} - \hat\beta\_{11} x\_{1i}) = 0$, where $\hat\gamma\_{12}$ and $\hat\beta\_{11}$ are the OLS estimators.

**2SLS**

For 2SLS, I will assume that $(x\_{1i}, x\_{2i}, x\_{3i})$ is used as IV. The corresponding 2SLS is identical to the IV estimator using $\hat{y}\_{2i}$ and $x\_{1i}$ as IV. Thus, (I think) the "normal equations" are
\begin{align}
\sum\_{i=1}^n \hat{y}\_{2i} (y\_{1i} - \tilde\gamma\_{12} y\_{2i} - \tilde\beta\_{11} x\_{1i}) &= 0,\\
\sum\_{i=1}^n x\_{1i} (y\_{1i} - \tilde\gamma\_{12} y\_{2i} - \tilde\beta\_{11} x\_{1i}) &= 0,
\end{align}
where $\tilde\gamma\_{12}$ and $\tilde\beta\_{11}$ are the 2SLS estimators using $x\_{1i}, x\_{2i}, x\_{3i}$ as IV, and $\hat{y}\_{2i}$ is the fitted value obtained by regressing $y\_{2i}$ on the instruments $x\_{1i}$, $x\_{2i}$ and $x\_{3i}$.

In the above, I understood 2SLS as the IV estimator using $(\hat{y}\_{2i}, x\_{1i})$ as IV. We can also understand 2SLS as the OLS estimator of $y\_{1i}$ on $(\hat{y}\_{2i}, x\_{1i})$. In that case, the normal equations can be written as
\begin{align}
\sum\_{i=1}^n \hat{y}\_{2i} (y\_{1i} - \tilde\gamma\_{12} \hat{y}\_{2i} - \tilde\beta\_{11} x\_{1i}) &= 0,\\
\sum\_{i=1}^n x\_{1i} (y\_{1i} - \tilde\gamma\_{12} \hat{y}\_{2i} - \tilde\beta\_{11} x\_{1i}) &= 0,
\end{align}
where $y\_{2i}$ is replaced with $\hat{y}\_{2i}$. The two sets of normal equations for 2SLS (one in terms of $y\_{2i}$ and the other in terms of $\hat{y}\_{2i}$) are identical.

**Intercept**

I have noticed that the intercept is excluded in the question. If the model is $y\_{1i} = \beta\_{10} + \gamma\_{12} y\_{2i} + \beta\_{11} x\_{1i} + \epsilon\_i$ instead, you need equations for the intercept as well.

**Matrix notations**

Using matrix notations, let $y$ be the $n\times 1$ matrix of the LHS variable, $X$ the $n\times 2$ matrix of $(y\_{2i}, x\_{1i})$ and $Z$ the $n\times 3$ matrix of $(x\_{i1}, x\_{i2}, x\_{i3})$. Let $\beta = (\gamma\_{12}, \beta\_{11})'$. (I am considering the model without the intercept. For the model with the intercept, $X$ and $Z$ contain a column of ones.) Then the OLS estimator is $\hat\beta = (X'X)^{-1} X'y$, which solves the normal equations $X'(y-X\hat\beta)=\boldsymbol 0$. To see this, note that $$X'(y-X\hat\beta)=\boldsymbol 0 \\
X'y - X'X\hat\beta = \boldsymbol 0\\
X'X\hat\beta = X'y\\
\hat\beta = (X'X)^{-1}X'y$$

The 2SLS (using $Z$ as instruments) estimator is $$\tilde\beta = [X'Z(Z'Z)^{-1} Z'X]^{-1} X'Z(Z'Z)^{-1} Z'y = (\hat{X}'X)^{-1} \hat{X}'y$$ where $\hat{X} = Z(Z'Z)^{-1} Z'X$. This 2SLS solves $\hat{X}' (y-X\tilde\beta)=\boldsymbol 0$, the normal equations for this 2SLS. Again, to see this, note that
$$\hat{X}' (y-X\tilde\beta)=\boldsymbol 0\\
\hat X'y - \hat X'X\tilde\beta = \boldsymbol 0\\
\hat X'X\tilde\beta = \hat X' y\\
\tilde\beta = (\hat X'X)^{-1} \hat X'y $$
You will see that $\hat{X}$ is the $n\times 2$ matrix of $(\hat{y}\_{2i}, x\_{1i})$. It is also true that $\tilde\beta = (\hat{X}'\hat{X})^{-1} \hat{X}'y$, and the normal equations are also written as $\hat{X}' (y-\hat{X}\tilde\beta)=0$. The two different expressions are identical because $\hat{X}'X = \hat{X}'\hat{X}$.

> **Comments:**
> **username_1**: Note that $\boldsymbol {X'}(\boldsymbol y-\boldsymbol{X\hat\beta})=\boldsymbol 0$ is a $K\times 1$ vector. This corresponds to the equations at the beginning of the answer.  
> **username_2**: I've just found that my $X$ is different from OP's $X$. Sorry.  
> **username_1**: Note that $(Z'Z)^{-1}Z'X$ is a matrix of coefficients, so $\hat X = Z(Z'Z)^{-1}Z'X$ is a $N\times 2$ matrix of the predicted values of the regressions of $\boldsymbol {y_2}$ and $\boldsymbol {x_1}$ on $\boldsymbol {x_1}, \boldsymbol {x_2}$ and $\boldsymbol {x_3}$ (known as the "projection"). You can see that $\boldsymbol{\hat x_1} = \boldsymbol {x_1}$ since $\boldsymbol {x_1}, \boldsymbol {x_2}$ and $\boldsymbol {x_3}$ are independent. However, $\boldsymbol {\hat y_2}\neq\boldsymbol {y_2}$.  
> **username_1**: no, I didn't pick up any mistakes in your answer.  
> **username_1**: You re-defined $X$, which was the correct thing to do (in order to maintain conventional notation).  
> **username_2**: Yes, I did. I noticed it after reading your first comment. Hope your comment helps people.  

---

