Regressing on a variable, and then regressing the residuals on another variable Consider observations on three variables $X1, X2,$ and $X3$. Suppose that $X1$ is regressed on $X2$. When the residual of the above regression is regressed on $X3$, the regression coefficient of $X3$ is $β_3$. When $X1$ is regressed on $X2$ and $X3$ simultaneously, the regression coefficient of $X3$ is $β_3^*$.
We need to show that $|β_3| ≤ |β_3^*|$ and determine when equality holds.
A huge part of my confusion here is that I'm unsure how to perform the regression of the residuals on $X3$. If I could have any pointers on how to go about that, I'd be very grateful.

Given the equation for the regression coefficients
$$
\hat\beta = (X'X)^{-1}X'Y
$$
and the residuals
$$
R = Y-\hat Y =Y-X\hat\beta
$$
We get
$$
R=(I-X(X'X)^{-1}X')Y
$$
The matrix $M= I-X(X'X)^{-1}X'$ is sometimes called the "residual maker" because it turns $Y$ into $R$.
So, to perform a regression of the residuals on $X_3$, first get the residuals:
$$
R = (I-X_2(X_2'X_2)X_2')X1
$$
Then use this as the outcome in a regression on $X_3$ to get $\beta_3$:
\begin{align}
\hat\beta_3 &= (X_3'X_3)^{-1}X_3'R \\
&=(X_3'X_3)^{-1}X_3'(I-X_2(X_2'X_2)X_2')X_1
\end{align}
The real versions of these expressions would involve intercepts, but I omitted them for clarity (assuming all variables are mean centered).
