<p>The answer is trivially zero. The estimate from regressing residuals <span class="math-container">$e$</span> on regressor is
<span class="math-container">$$
(X'X)^{-1} X'e.
$$</span>
But <span class="math-container">$X'e = 0 \in \mathbb{R}^p$</span>, where <span class="math-container">$p$</span> is the number of regressors (this is just, e.g. the FOC's defining the OLS <span class="math-container">$\hat{\beta}$</span>). So <span class="math-container">$(X'X)^{-1} X'e = 0$</span>.</p>
<p>This has <em>nothing</em> to do with the exogeneity assumption <span class="math-container">$E[\epsilon_i X_i] = 0$</span> (or <span class="math-container">$E[\epsilon_i|X_i] = 0$</span>). You can always run the regression, and the OLS residuals will always have property, regardless of whether exogeneity is economically plausible.</p>

CC BY-SA 4.0
2021-10-09T18:11:09.207Z
2021-10-09T23:56:36.560Z
2021-10-09T23:56:36.560Z
Michael
<p>To fix notation, <span class="math-container">$y_i = x_i^T\beta + \epsilon_i$</span>. If the standard assumption on non-multicollinearity is made, then the residuals <span class="math-container">$\hat\epsilon$</span> regressed on <span class="math-container">$x$</span> is zero.</p>
<p>In matrix notation, a model with <span class="math-container">$k$</span> regressors, <span class="math-container">$x=(x_1,\dots,x_k)$</span>, and <span class="math-container">$n$</span> observations, we have,
<span class="math-container">\begin{eqnarray}
Y = X \beta + \epsilon,&amp;\text{ where }&amp; Y=\begin{bmatrix}
y_1\\
\vdots\\
y_n
\end{bmatrix} 
&amp; X=\begin{bmatrix}
x_{11}&amp;x_{21}&amp;\cdots &amp;x_{k1} \\
x_{12}&amp;x_{22}&amp;\cdots &amp;x_{k2} \\
\vdots &amp; \vdots &amp; \ddots &amp; \vdots\\
x_{1n}&amp;x_{2n}&amp;\cdots &amp;x_{kn}
\end{bmatrix},
&amp;
\beta = 
\begin{bmatrix}
\beta_1\\
\vdots\\
\beta_k
\end{bmatrix}
\end{eqnarray}</span>
(take the first column to be a column of 1's if there is constant term) and that the least squares estimator of <span class="math-container">$\beta$</span>, is
<span class="math-container">\begin{equation}
\hat\beta = (X^T X)^{-1}X^T Y.\label{beta-hat}\tag{1}
\end{equation}</span> (Non-multicolinearity guarantees that <span class="math-container">$X^T X$</span> is invertible.)</p>
<p>To use computational brute force without any insight, regress the residuals <span class="math-container">$\hat\epsilon:=Y-X\hat\beta$</span> on <span class="math-container">$x$</span> assuming a linear model
<span class="math-container">\begin{equation}
\hat\epsilon = x^T\gamma + \delta.\label{epsilon-model}\tag{2}
\end{equation}</span>
Then apply \ref{beta-hat},
<span class="math-container">\begin{align}
\hat\gamma&amp;= (X^T X)^{-1}X^T\hat\epsilon\\ 
&amp;= (X^T X)^{-1}X^T(Y-X\hat\beta)\label{X^T e=0}\tag{3}\\
&amp;= (X^T X)^{-1}X^T(Y-X(X^T X)^{-1}X^TY)\\
&amp;= 0.
\end{align}</span></p>
<p>**corrected with the comments below, noting that, assuming <span class="math-container">$X^TX$</span> is invertible, (\ref{X^T e=0})<span class="math-container">$\iff X^T\hat\epsilon=0\iff(\ref{beta-hat})$</span>.</p>

CC BY-SA 4.0
2021-10-08T15:03:13.700Z
Thank you so much. I was getting the OLS coefficient as zero but I thought I must be doing something wrong. The intuition really helps. Thank you!
Jacob
CC BY-SA 4.0
2021-10-08T16:10:33.020Z
@Jacob The $\hat \epsilon$ should probably be a $\hat \gamma$: the estimate of the coefficient of $X$ for the equation $(2)$.  The intuition is that the coefficient is zero as $X$ is orthogonal to $\epsilon = Y - X \beta$.
tdm
CC BY-SA 4.0
2021-10-09T06:40:27.437Z
Do you mean *exogenous* rather than *endogenous*?
Richard Hardy
CC BY-SA 4.0
2021-10-09T16:36:38.860Z
The question is about residuals ("e hat" as the question puts it), not the (unobserved) error term $\epsilon$. Regressing the residuals on regressors always gives trivially 0, just by property of the OLS procedure.  It has nothing to do with the exogeneity assumption, which is a condition on the population/DGP.
Michael
CC BY-SA 4.0
2021-10-10T14:22:20.720Z
@Michael. You're absolutely right! I was trying to give some insight  w/o resorting to geometry, but as turns out, wrong. Hope I have not confused the OP. Thanks for the clarification.
p.co
CC BY-SA 4.0
2021-10-10T14:25:01.680Z
@RichardHardy. Yes, you're right. No way to correct a comment so I deleted it. Thanks for pointing that out!
p.co
CC BY-SA 4.0
2021-10-08T13:37:39.717Z
2021-10-10T14:18:37.977Z
2021-10-10T14:18:37.977Z
p.co
<p>Suppose e hat is the OLS residual of a regression of Y on X. If not we regress e hat on X, what would the OLS coefficient be?</p>

CC BY-SA 4.0
2021-10-08T06:43:21.437Z
Can you please clarify the question? What do you mean by what happens? You will get regression of residuals on your independent variable… nothing else will happen your PC won’t explode
1muflon1
CC BY-SA 4.0
2021-10-08T06:47:31.957Z
I have elaborated on the main question. Bruce Hansen's Econometrics has this exercise question that asks what will the OLS coefficient be when the residual is regressed on the regressor. I was myself wondering what will be special about it.
Jacob
CC BY-SA 4.0
2021-10-08T07:40:15.567Z
@Jacob I recommend actually running these two regressions in a program. That should give you a hint. Rereading the book chapter will probably give you the answer.
Giskard
<econometrics>
What happens if you regress the residuals on the regressor itself?
Jacob
# What happens if you regress the residuals on the regressor itself?

(Asked by: username_0 on 2021-10-08)

Suppose e hat is the OLS residual of a regression of Y on X. If not we regress e hat on X, what would the OLS coefficient be?

**Tags:** econometrics

> **Comments:**
> **username_1**: Can you please clarify the question? What do you mean by what happens? You will get regression of residuals on your independent variable… nothing else will happen your PC won’t explode  
> **username_0**: I have elaborated on the main question. Bruce Hansen's Econometrics has this exercise question that asks what will the OLS coefficient be when the residual is regressed on the regressor. I was myself wondering what will be special about it.  
> **username_2**: @username_0 I recommend actually running these two regressions in a program. That should give you a hint. Rereading the book chapter will probably give you the answer.  

---

## 2 answers

### Answer by username_3 on 2021-10-09. Score: 1

The answer is trivially zero. The estimate from regressing residuals $e$ on regressor is
$$
(X'X)^{-1} X'e.
$$
But $X'e = 0 \in \mathbb{R}^p$, where $p$ is the number of regressors (this is just, e.g. the FOC's defining the OLS $\hat{\beta}$). So $(X'X)^{-1} X'e = 0$.

This has *nothing* to do with the exogeneity assumption $E[\epsilon\_i X\_i] = 0$ (or $E[\epsilon\_i|X\_i] = 0$). You can always run the regression, and the OLS residuals will always have property, regardless of whether exogeneity is economically plausible.


---

### Answer by username_4 on 2021-10-08. Score: 0

To fix notation, $y\_i = x\_i^T\beta + \epsilon\_i$. If the standard assumption on non-multicollinearity is made, then the residuals $\hat\epsilon$ regressed on $x$ is zero.

In matrix notation, a model with $k$ regressors, $x=(x\_1,\dots,x\_k)$, and $n$ observations, we have,
\begin{eqnarray}
Y = X \beta + \epsilon,&\text{ where }& Y=\begin{bmatrix}
y\_1\\
\vdots\\
y\_n
\end{bmatrix}
& X=\begin{bmatrix}
x\_{11}&x\_{21}&\cdots &x\_{k1} \\
x\_{12}&x\_{22}&\cdots &x\_{k2} \\
\vdots & \vdots & \ddots & \vdots\\
x\_{1n}&x\_{2n}&\cdots &x\_{kn}
\end{bmatrix},
&
\beta =
\begin{bmatrix}
\beta\_1\\
\vdots\\
\beta\_k
\end{bmatrix}
\end{eqnarray}
(take the first column to be a column of 1's if there is constant term) and that the least squares estimator of $\beta$, is
\begin{equation}
\hat\beta = (X^T X)^{-1}X^T Y.\label{beta-hat}\tag{1}
\end{equation} (Non-multicolinearity guarantees that $X^T X$ is invertible.)

To use computational brute force without any insight, regress the residuals $\hat\epsilon:=Y-X\hat\beta$ on $x$ assuming a linear model
\begin{equation}
\hat\epsilon = x^T\gamma + \delta.\label{epsilon-model}\tag{2}
\end{equation}
Then apply \ref{beta-hat},
\begin{align}
\hat\gamma&= (X^T X)^{-1}X^T\hat\epsilon\\
&= (X^T X)^{-1}X^T(Y-X\hat\beta)\label{X^T e=0}\tag{3}\\
&= (X^T X)^{-1}X^T(Y-X(X^T X)^{-1}X^TY)\\
&= 0.
\end{align}

\*\*corrected with the comments below, noting that, assuming $X^TX$ is invertible, (\ref{X^T e=0})$\iff X^T\hat\epsilon=0\iff(\ref{beta-hat})$.

> **Comments:**
> **username_0**: Thank you so much. I was getting the OLS coefficient as zero but I thought I must be doing something wrong. The intuition really helps. Thank you!  
> **username_5**: @username_0 The $\hat \epsilon$ should probably be a $\hat \gamma$: the estimate of the coefficient of $X$ for the equation $(2)$.  The intuition is that the coefficient is zero as $X$ is orthogonal to $\epsilon = Y - X \beta$.  
> **username_6**: Do you mean *exogenous* rather than *endogenous*?  
> **username_3**: The question is about residuals ("e hat" as the question puts it), not the (unobserved) error term $\epsilon$. Regressing the residuals on regressors always gives trivially 0, just by property of the OLS procedure.  It has nothing to do with the exogeneity assumption, which is a condition on the population/DGP.  
> **username_4**: @username_3. You're absolutely right! I was trying to give some insight  w/o resorting to geometry, but as turns out, wrong. Hope I have not confused the OP. Thanks for the clarification.  
> **username_4**: @username_6. Yes, you're right. No way to correct a comment so I deleted it. Thanks for pointing that out!  

---

