Linear regression as a projection I'm trying to refresh my memory as to the relationship between ordinary least squares and the linear algebra procedure of projection. (possibly highly related to Linear algebra attributes of a typical data set)
Starting with a model matrix, $\mathbf{X} = [\mathbf{1}, \vec{x_1}, \ldots, \vec{x_p}]$ (with the usual $n$ rows) and a response $Y$, we can define the hat matrix as a projection $\mathbf{H} = \mathbf{X}(\mathbf{X}^t \mathbf{X})^{-1}\mathbf{X}^t $. Then $\hat{Y} = \mathbf{H}Y$. In linear algebra, we define the column space, the row space, and the null space for a matrix.
Considering OLS as a projection:

*

*Is the row/column space defined for $\mathbf{X}$? Does the column space correspond to what you can call a "fitted space" or similar, i.e. on $\mathbb{R}^{p}$?

*Is the "null space" a basis defined by the vector of residuals, i.e. on $\mathbb{R}^{n-p}$?

*If the model matrix $\mathbf{X}$ contains a duplicate, do we consider the row space to actually be $\mathbb{R}^{n-1}$?


Unless there are other collinearities, the column vector space now has dimension $p$  (i.e., $(p+1) -1$). Again, it is not $\mathbb{R}^{n}$ or $\mathbb{R}^{n-1}$; rather, it is a subset of $\mathbb{R}^{n}$.
