How exactly do we do Gauss elimination? 
This is a matrix:
  $$\begin{bmatrix}
1 & 1 & 1\\
1 & 2 & 3\\
1 & 3 & k
\end{bmatrix}\begin{bmatrix}
x\\
y\\
z
\end{bmatrix}=
\begin{bmatrix}
3\\
6\\
4+k
\end{bmatrix}$$
  Find $k$ so that it has no unique solution. Solve the equations for this value of $k$.

I found $k = 5$ by using $\text{determinant}=0$.
Then I tried Gauss elimination, from what I understand from my lecturer, I use the 3 rules of Gauss elimination to solve, I just randomly use whatever I need to achieve echelon form. I have come to this: 
$$\begin{bmatrix}
1 & 0 & 0 & 0\\
0 & 1 & 0 & 3\\
0 & 0 & 1 & 0\\
\end{bmatrix}$$
The answer is 
$$\begin{bmatrix}x\\y\\z\end{bmatrix}=\begin{bmatrix}0\\3\\0\end{bmatrix}+t\cdot\begin{bmatrix}1\\-2\\1\end{bmatrix}$$
Where does the $t$ come from? Does it involve eigenvalues? 

By Gauss-Jordan elimination I get the following:
$$\left(\begin{array}{ccc|c}
  1 & 1 & 1 & 3 \\
  1 & 2 & 3 & 6 \\
  1 & 3 & k & 4+k
\end{array}\right)\sim
\left(\begin{array}{ccc|c}
  1 & 1 & 1 & 3 \\
  0 & 1 & 2 & 3 \\
  0 & 2 &k-1& 1+k
\end{array}\right)\sim
\left(\begin{array}{ccc|c}
  1 & 1 & 1 & 3 \\
  0 & 1 & 2 & 3 \\
  0 & 0 &k-5& k-5
\end{array}\right)$$
In the next step I would like to divide the last row by $k-5$. Notice that this is valid only for $k-5\ne0$, i.e., $k\ne 5$. 

If $\boxed{k\ne5}$ I get
$$\left(\begin{array}{ccc|c}
  1 & 1 & 1 & 3 \\
  0 & 1 & 2 & 3 \\
  0 & 0 &k-5& k-5
\end{array}\right)\sim
\left(\begin{array}{ccc|c}
  1 & 1 & 1 & 3 \\
  0 & 1 & 2 & 3 \\
  0 & 0 & 1 & 1
\end{array}\right)\sim
\left(\begin{array}{ccc|c}
  1 & 1 & 0 & 2 \\
  0 & 1 & 0 & 1 \\
  0 & 0 & 1 & 1
\end{array}\right)\sim
\left(\begin{array}{ccc|c}
  1 & 0 & 0 & 1 \\
  0 & 1 & 0 & 1 \\
  0 & 0 & 1 & 1
\end{array}\right)$$
I see that for $k\ne5$ I have the unique solution $(1,1,1)$. 
I can check that this is indeed a solution by pluggin these values into the original system:
\begin{align*}
1+1+1&=3\\
1+2+3&=6\\
1+3+k&=4+k
\end{align*}

For $\boxed{k=5}$ I get
$$
\left(\begin{array}{ccc|c}
  1 & 1 & 1 & 3 \\
  0 & 1 & 2 & 3 \\
  0 & 0 & 0 & 0
\end{array}\right)\sim
\left(\begin{array}{ccc|c}
  1 & 0 &-1 & 0 \\
  0 & 1 & 2 & 3 \\
  0 & 0 & 0 & 0
\end{array}\right)
$$
This equation has infinitely many solutions $(t,3-2t,t)$, $t\in\mathbb R$, as explained in Adriano's answer.
