What does it mean to represent a number in term of a $2\times2$ matrix? Today my friend showed me that the imaginary number can be represented in term of a matrix
$$i = \pmatrix{0&-1\\1&0}$$
This was very very confusing for me because I have never thought of it as a matrix. But it is apparent that the properties of imaginary number holds even in this representation, namely $i\cdot i = -1$
Even more confusing is that a bunch of quantities can be represented by matrices
$$e^{i\theta} = \pmatrix{\cos\theta&-\sin\theta\\
\sin\theta&\cos\theta}$$
Naturally I wonder if we can perform this for any number. What is the big picture here? What is this operation called turning a number into a matrix. What is the deeper implication - how does knowing this help?
100 points to anyone who can answer this in a comprehensive way.

So far so good:
$$ i^2 =
\left[ \begin{array}{cc} 0 & -1 \\ 1 & 0 \end{array} \right]
\left[ \begin{array}{cc} 0 & -1 \\ 1 & 0 \end{array} \right]
= \left[ \begin{array}{cc} -1 & 0 \\ 0 & -1 \end{array} \right] =
- \left[ \begin{array}{cc} 1 & 0 \\ 0 & 1 \end{array} \right] = -1
$$
But what I'm missing in the current answers is this:
$$ e^{i\theta} =
e^{\left[ \begin{array}{cc} 0 & -1 \\ 1 & 0 \end{array} \right]\theta} = \\
\left(\left[ \begin{array}{cc} 1 & 0 \\ 0 & 1 \end{array} \right]\theta\right)^0
+ \left(\left[ \begin{array}{cc} 0 & -1 \\ 1 & 0 \end{array} \right]\theta\right)^1
+ \frac{1}{2!}\left(
\left[ \begin{array}{cc} 0 & -1 \\ 1 & 0 \end{array} \right]\theta
\right)^2 + \frac{1}{3!}\left(
\left[ \begin{array}{cc} 0 & -1 \\ 1 & 0 \end{array} \right]\theta
\right)^3 + \\ \frac{1}{4!}\left(
\left[ \begin{array}{cc} 0 & -1 \\ 1 & 0 \end{array} \right]\theta
\right)^4 + \frac{1}{5!}\left(
\left[ \begin{array}{cc} 0 & -1 \\ 1 & 0 \end{array} \right]\theta
\right)^5 + \frac{1}{6!}\left(
\left[ \begin{array}{cc} 0 & -1 \\ 1 & 0 \end{array} \right]\theta
\right)^6 + \frac{1}{7!}\left(
\left[ \begin{array}{cc} 0 & -1 \\ 1 & 0 \end{array} \right]\theta
\right)^7 +\; \cdots \; = \\
\left[ \begin{array}{cc} 1 & 0 \\ 0 & 1 \end{array} \right] +
\left[ \begin{array}{cc} 0 & -1 \\ 1 & 0 \end{array} \right]\theta
+ \frac{1}{2!}
\left[ \begin{array}{cc} -1 & 0 \\ 0 & -1 \end{array} \right]\theta^2
+ \frac{1}{3!}
\left[ \begin{array}{cc} 0 & 1 \\ -1 & 0 \end{array} \right]\theta^3
+ \\ \frac{1}{4!}
\left[ \begin{array}{cc} 1 & 0 \\ 0 & 1 \end{array} \right]\theta^4
+ \frac{1}{5!}
\left[ \begin{array}{cc} 0 & -1 \\ 1 & 0 \end{array} \right]\theta^5
+ \frac{1}{6!} 
\left[ \begin{array}{cc} -1 & 0 \\ 0 & -1 \end{array} \right]\theta^6
+ \frac{1}{7!} 
\left[ \begin{array}{cc} 0 & 1 \\ -1 & 0 \end{array} \right]\theta^7
+ \; \cdots \; = \\
\left[ \; \begin{array}{cc}
1 - \theta^2/2! + \theta^4/4! - \theta^6/6! + \; \cdots \; & \;
- \, \theta + \theta^3/3! - \theta^5/5! + \theta^7/7! + \; \cdots \\
+ \, \theta - \theta^3/3! + \theta^5/5! - \theta^7/7! + \; \cdots \; & \;
1 - \theta^2/2 + \theta^4/4! - \theta^6/6! + \; \cdots
\end{array} \; \right] = \\
\left[ \begin{array}{cc} \cos(\theta) & -\sin(\theta) \\
\sin(\theta) & \cos(\theta) \end{array} \right]
$$
