Proof that combined matrices are idempotent. Suppose that $X_{nxp} (n>p)$ is a matrix such that $X'X$  is invertible. 


*

*Prove that both $X(X'X)^{-1}X'$ and $I_n-X(X^TX)^{-1}X^T$  are idempotent. 

*Prove that the tr $X(X'X)^{-1}X' =p$  and tr $(I_n - X(X'X)^{-1}X')=n-p$.


Let us put
$$ A \colon= X \left(X^\prime X\right)^{-1} X^\prime. $$
Then we note that
$$
\begin{align}
A^2 &= AA \\
&= \left[ X \left(X^\prime X\right)^{-1} X^\prime \right] \left[ X \left(X^\prime X\right)^{-1} X^\prime  \right] \\
&= X \left[ \left(X^\prime X\right)^{-1}  \left( X^\prime X \right) \right] \left(X^\prime X\right)^{-1} X^\prime  \\
&= X I_{p\times p} \left(X^\prime X\right)^{-1} X^\prime \\
&= X \left(X^\prime X\right)^{-1} X^\prime \\
&= A.
\end{align}
$$
Now let us put 
$$ B \colon= I_n-X(X^TX)^{-1}X^T. $$
Then we find that 
$$
\begin{align}
B^2 &= BB \\
&=  \left[ I_n-X(X^TX)^{-1}X^T \right] \left[ I_n-X(X^TX)^{-1}X^T \right] \\
&= I_n \left[ I_n- X(X^TX)^{-1}X^T \right] - X(X^TX)^{-1}X^T \left[ I_n-X(X^TX)^{-1}X^T \right] \\
&= I_n I_n - I_n X(X^TX)^{-1}X^T - X(X^TX)^{-1}X^T I_n + \left[ X(X^TX)^{-1}X^T \right] \left[ X(X^TX)^{-1}X^T \right] \\
&= I_n -  X(X^TX)^{-1}X^T - X(X^TX)^{-1}X^T + X\left[ (X^TX)^{-1} \left( X^T X\right) \right] \left[ (X^TX)^{-1}X^T \right] \\
&= I_n - 2 X(X^TX)^{-1}X^T   + X I_n \left[ (X^TX)^{-1} X^T \right] \\
&= I_n - 2 X(X^TX)^{-1}X^T   + X \left[ (X^TX)^{-1} X^T  \right] \\
&= I_n - 2 X(X^TX)^{-1}X^T   + X (X^TX)^{-1} X^T   \\
&= I_n -  X(X^TX)^{-1}X^T \\
&= B. 
\end{align}
$$
