Calculate the matrix for the projection of $R^3$ onto the plane $x+y+z=0$. Calculate the matrix for the projection of $R^3$ onto the plane $x+y+z=0$.
If $b=\begin{bmatrix} x \\ y \\ z \\ \end{bmatrix}$ and $A=\begin{bmatrix} 1 & 0 \\ 0 & 1 \\ -1 & -1 \\ \end{bmatrix}$ I get stuck when trying to solve using $$A^TA\hat x=A^Tb$$ and $$proj_vb=A\hat x$$
$A^T=\begin{bmatrix} 1 & 0 & -1 \\ 0 & 1 & -1 \\ \end{bmatrix}$   
therefore $A^TA=\begin{bmatrix} 2 & 1 \\ 1 & 2 \\ \end{bmatrix}$  
and $A^Tb=\begin{bmatrix} x-z \\ y-z \\ \end{bmatrix}$  
So $$\begin{bmatrix}2 & 1 \\ 1 & 2 \\ \end{bmatrix} \hat x = \begin{bmatrix} x-z \\ y-z \\ \end{bmatrix}$$
Let's say $\hat x=\begin{bmatrix} a \\ b \\ \end{bmatrix}$,
then $2a+b=x-z$ and $a+2b=y-z$  
Here is where I get confused, what is $\hat x$ so that I can plug it into $proj_vb=A\hat x$ and get the projection vector?

Orthogonal Projection from a basis
The matrix for a Projection satisfies
$$
A^2=A
$$
However, for an Orthogonal Projection, we must also have
$$
A=A^T
$$
Since $\frac1{\sqrt2}(1,0,-1)$ and $\frac1{\sqrt2}(1,0,-1)\times\frac1{\sqrt3}(1,1,1)=\frac1{\sqrt6}(1,-2,1)$ form an orthonormal basis for the space so that $x+y+z=0$, we get that
$$
\begin{bmatrix}
\frac1{\sqrt2}&\frac1{\sqrt6}\\
0&-\frac2{\sqrt6}\\
-\frac1{\sqrt2}&\frac1{\sqrt6}
\end{bmatrix}
\begin{bmatrix}
\frac1{\sqrt2}&0&-\frac1{\sqrt2}\\
\frac1{\sqrt6}&-\frac2{\sqrt6}&\frac1{\sqrt6}
\end{bmatrix}
=
\begin{bmatrix}
\frac23&-\frac13&-\frac13\\
-\frac13&\frac23&-\frac13\\
-\frac13&-\frac13&\frac23
\end{bmatrix}
$$
is the projection onto the space so that $x+y+z=0$.

Orthogonal Projection from a unit normal
We can also use Jyrki Lahtonen's approach and use the unit normal $\frac1{\sqrt3}(1,1,1)$ to get
$$
\begin{bmatrix}
1&0&0\\0&1&0\\0&0&1
\end{bmatrix}
-
\begin{bmatrix}
\frac1{\sqrt3}\\\frac1{\sqrt3}\\\frac1{\sqrt3}
\end{bmatrix}
\begin{bmatrix}
\frac1{\sqrt3}&\frac1{\sqrt3}&\frac1{\sqrt3}
\end{bmatrix}
=
\begin{bmatrix}
\frac23&-\frac13&-\frac13\\
-\frac13&\frac23&-\frac13\\
-\frac13&-\frac13&\frac23
\end{bmatrix}
$$

General Projection
For any vector $v$ so that $v\cdot(1,1,1)\ne0$, we have that
$$
Ax=x-\frac{x\cdot(1,1,1)}{v\cdot(1,1,1)}v
$$
is a projection onto the space where $x\cdot(1,1,1)=0$. This projection only depends on the direction of $v$, not the length. Thus, there is a two dimensional family of projections, parameterized by a vector in $\mathbb{P}^2$. The matrix for this projection is
$$
\begin{align}
A
&=\begin{bmatrix}
1&0&0\\0&1&0\\0&0&1
\end{bmatrix}
-
\frac
{v
\begin{bmatrix}
1&1&1
\end{bmatrix}}
{v\cdot
\begin{bmatrix}
1&1&1
\end{bmatrix}}\\[6pt]
&=
\begin{bmatrix}
1-\frac{v_1}{v_1+v_2+v_3}&-\frac{v_1}{v_1+v_2+v_3}&-\frac{v_1}{v_1+v_2+v_3}\\
-\frac{v_2}{v_1+v_2+v_3}&1-\frac{v_2}{v_1+v_2+v_3}&-\frac{v_2}{v_1+v_2+v_3}\\
-\frac{v_3}{v_1+v_2+v_3}&-\frac{v_3}{v_1+v_2+v_3}&1-\frac{v_3}{v_1+v_2+v_3}\\
\end{bmatrix}
\end{align}
$$
As matrices, vectors are represented here by column vectors. Note that when $v=(1,1,1)$, we get the orthogonal projection mentioned above in the first two sections.
