2nd degree matrix equation Let $X$ be a matrix with 2 rows and 2 columns.
Solve the following equation:
$$ X^2 = \begin{pmatrix}
3 & 5\\ 
-5 & 8
\end{pmatrix} $$ 
Here is what I did:
Let $ X = \begin{pmatrix}
a & b\\ 
c & d
\end{pmatrix} $. After multiplying I got the following system:
$$ \left\{\begin{matrix}
a^2 + bc = 3\\ 
ab + bd = 5\\ 
ac + cd = -5\\ 
d^2 + bc = 8
\end{matrix}\right. $$
At this point I got stucked. 
If you know how to solve this please help me! Thank you!

We have the following criteria which you already stated correctly, but you missed one more information $(5)$ - still, you can solve this root problem without this additional knowledge by plugging in recursively - which comes from the determinant, we get then
\begin{align}
a^2 + bc &= 3 \tag1\\ 
ab + bd &= 5\tag2 \\ 
ac + cd &= -5\tag3 \\ 
d^2 + bc &= 8\tag4 \\
\det(X)=ad-bc&=7=\sqrt{\det(M)} \tag{5a}
\end{align}
Remark remember that we have $\det(AB)=\det(A)\det(B)$
This gives us  
\begin{align} (1)-(4)&=a^2-d^2=(a-d)(a+d)=-5\\  (2)&=b(a+d)=5 \\ 
(3)&=c(a+d)=-5 \\ (1)+(5a)&=a(a+d)=10 \end{align} 
so we get from $(2)\wedge(3)$ $b=-c$ and further $a-d=c$ and $a=-2(a-d)\iff\frac32a=d$ 
therefore
\begin{align} a(a+d)=10=a(a+\frac32a)=\frac52a^2\iff4=a^2 \end{align}
and thus we get for $a=2$ 
\begin{align}
a=2,d=3,c=-1,b=1
\end{align}
so 
$$
X_1=\begin{pmatrix}2 &1\\ -1&3\end{pmatrix}
$$
and for $a=-2$ 
\begin{align}
a=-2,d=-3,c=1,b=-1
\end{align}
so 
$$
X_2=\begin{pmatrix}-2 &-1\\ 1&-3\end{pmatrix}=-X_1
$$
Remark due to Robert Israel:
Indeed we have to investigate the other possible determinant solution 
\begin{align}
\det(X)=ad-bc&=-7 \tag{5b}
\end{align}
then we get 
\begin{align}
(1)+(5b)&=a(a+d)=-4 \\
(1)-(4)&=a^2-d^2=(a-d)(a+d)=-5
\end{align}
which gives us $\frac54a=(a-d)\iff-\frac14a=d$ and therefore
\begin{align}
a(a+d)=-4=a(a-\frac14a)\iff-4=\frac34a^2
\end{align}
which leads, if we stay in the field of the real numbers, to a contradiction. However, one might find for example other complex solutions. For a more detailed discussion please check out the comment section.
