Solving recursive sequence using generating functions $$a_{0} = 0$$
$$a_{1} = 1$$
$$a_{n} = a_{n-1} - a_{n-2}$$
I have to find the solution of this equation ($a_{n} = ...$, non-recursive, you know what I mean...). So let's pretend that:
$$ A(x) = \sum_{n=0}a_{n}x^{n}$$
Using this formula and the recursive equation I'm getting:
$$A(x) = xA(x) - x^{2}A(x)$$
Substituting $t = A(x)$, solving simple quadratic equation, and I'm getting two solutions:
$t = A(x) = \frac{1 - i\sqrt{3}}{2}$ or $t = A(x) = \frac{1 + i\sqrt{3}}{2}$
So actually this should be the right side of the generating function $A(x)$, it also has no variable so it already is a coefficient - the job is done.

However, the book shows different results, and they differ a lot. Let me write it:
$a_{n} = -\frac{i\sqrt{3}}{3}(\frac{1+i\sqrt{3}}{2})^{n mod6}$ or $a_{n} = \frac{i\sqrt{3}}{3}(\frac{1-i\sqrt{3}}{2})^{n mod6}$
What did I do wrong?

The trick is to forget about convergence. To encourage self-study, I will feature a slightly different series (the Fibonacci numbers), but the technique is the same.
$$a_n=a_{n-1}+a_{n-2} \implies a_nx^n=xa_{n-1}x^{n-1}+x^2a_{n-2}x^{n-2}\quad \text{for n}=2,3,\dots$$
Let $A(x)$ be the generating function of the sequence, $A(x):=\sum_{n=0}^{\infty}a_nx^n$.
$$A(x)-a_1x-a_0=x(A(x)-a_0)+x^2A(x)$$
Note that we subtract some terms from the sum. Next, plug in the initial conditions $a_0=0$ and $a_1=1$.
$$A(x)-1\cdot x-0=x(A(x)-0)+x^2A(x) \implies A(x)=\frac{x}{1-x-x^2}$$
Up to now, the only difference between the sequences was a sign. You should have arrived at $A(x)=\frac{x}{1-x+x^2}$.
Unfortunately, the following part is specific to the Fibonacci series. Now, we use the geometric series to obtain a closed formula.
$$\begin{align*}A(x)&=\frac{1}{\sqrt{5}\left(1-\frac{2}{\left(\sqrt{5}-1\right)}x\right)}-\frac{1}{\sqrt{5}\left(1+\frac{2}{\left(\sqrt{5}+1\right)}x\right)}=\\
&=\frac{1}{\sqrt{5}}\sum_{n=0}^{\infty}\left(\frac{2}{\sqrt{5}-1}x\right)^n-\frac{1}{\sqrt{5}}\sum_{n=0}^{\infty}\left(-\frac{2}{\sqrt{5}+1}x\right)^n=\\
&=\frac{1}{\sqrt{5}}\sum_{n=0}^{\infty}\left(\left(\frac{2\left(\sqrt{5}+1\right)}{4}\right)^n-\left(-\frac{2\left(\sqrt{5}-1\right)}{4}\right)^n\right)x^n\end{align*}$$
Finally, let us identify the coefficients of both expansions.
$$a_n=\frac{1}{\sqrt{5}}\left(\left(\frac{\sqrt{5}+1}{2}\right)^n-\left(\frac{1-\sqrt{5}}{2}\right)^n\right) \implies a_n=0,1,1,2,3,5,8,13,\dots$$
