Normal equation question Let $C = \begin{bmatrix}
1 & -1 & 1 \\
1 & 1 & 3 \\
2 & 0 & 4
\end{bmatrix}$ and let $\mathbf{b} = \begin{bmatrix}
3 \\
5 \\
5
\end{bmatrix}$. Find the set of all solutions $\mathbf{x}$ to the normal equation}
\begin{equation*}
C^TC\mathbf{x} = C^T\mathbf{b}.
\end{equation*}
Hence, or otherwise, find the best approximation to $\mathbf{b}$ in $\text{col}(C)$.
 Lets first calculate $C^TC$ which is
\begin{equation*}
\begin{bmatrix}
1 & 1 & 2 \\
-1 & 1 & 0\\
1 & 3 & 4
\end{bmatrix}
\begin{bmatrix}
1 & -1 & 1 \\
1 & 1 & 3 \\
2 & 0 & 4
\end{bmatrix} = \begin{bmatrix}
6 & 0 & 12 \\
0 & 2 & 2 \\
12 & 2 & 26
\end{bmatrix}.
\end{equation*}
Also
\begin{equation*}
C^T\mathbf{b} =\begin{bmatrix}
1 & 1 & 2 \\
-1 & 1 & 0\\
1 & 3 & 4
\end{bmatrix}
\begin{bmatrix}
3 \\
5 \\
5
\end{bmatrix} = \begin{bmatrix}
18 \\
2 \\
38
\end{bmatrix}.
\end{equation*}
Thus, to find $\mathbf{x}$ we solve
\begin{equation*}
\begin{bmatrix}
6 & 0 & 12 \\
0 & 2 & 2 \\
12 & 2 & 26
\end{bmatrix} \mathbf{x}= \begin{bmatrix}
18 \\
2 \\
38
\end{bmatrix}.
\end{equation*}
To do this we just use row reduction to get
\begin{equation*}
\begin{bmatrix}
6 & 0 & 12 & 18 \\
0 & 2 & 2 & 2 \\
12 & 2 & 26 & 38
\end{bmatrix} \sim \begin{bmatrix}
1 & 0 & 2 & 3 \\
0 & 1 & 1 & 1 \\
0 & 0 & 0 & 0
\end{bmatrix}.
\end{equation*}
Using $R$ we must have then $x_1 = 3-2x_3$, $v_2 = 1-x_3$. So
\begin{equation*}
\begin{bmatrix}
x_1 \\
x_2 \\
x_3
\end{bmatrix} = \begin{bmatrix}
3 \\
1 \\
0
\end{bmatrix}+x_3\begin{bmatrix}
-2 \\
-1 \\
1
\end{bmatrix}.
\end{equation*}
Hence, we see that
\begin{equation*}
\begin{bmatrix}
x_1 \\
x_2 \\
x_3
\end{bmatrix}\in  \left\{\begin{bmatrix}
3 \\
1 \\
0
\end{bmatrix}+c\begin{bmatrix}
-2 \\
-1 \\
1
\end{bmatrix}:c\in \mathbb{R}\right\} = \begin{bmatrix}
3 \\
1 \\
0
\end{bmatrix}+\text{span}\left\{\begin{bmatrix}
-2 \\
-1 \\
1
\end{bmatrix}\right\}.
\end{equation*}

*

*Solve the system with $C^T C x = C^T b$ with the Gaussian elimination (or the row echelon form) --- i.e. do not compute the inverse (which does not exist here)

*If $C^T C$ is singular, it means that you will get either no solution or an infinite number of solutions. The row echelon form should help you determine whether or not there is a solution.

