Question: Does there exist a matrix $A$ in $\Bbb{R}^{3\times 3}$ that has only $0$ as eigenvalue? 

As far as I think, an upper diagonal matrix with only zeros on the diagonal is such an example. 

And if I'm correct, is such a matrix diagonalizable?

Answer: 
Anyway, in your case, $y\ge 0$, so you should transform all this to
$$|x|<3\;\textit{ or }\; |x|\ge 5\iff -3<x<3\;\textit{ or }\; x\ge 5\;\textit{ or }\; x\le -5$$
or, as a set, the union of three intervals:
$$(-\infty,-5]\cup(-3,3)\cup[5,+\infty). $$
The equivalences $$\lvert A\rvert< B\iff\begin{cases}B\ge 0\\ A> -B\\ A< B\end{cases}\\ \lvert A\rvert\ge B\iff B\le 0\lor \begin{cases}B> 0\\ A\le -B\end{cases}\lor\begin{cases}B>0\\ A\ge B\end{cases}$$

are a good place to start from.