Diagonalising quadratic form Given the quadratic form $$Q(x) = \alpha\alpha_1\alpha_2 + 2\alpha^2\alpha_1\alpha_3$$ on $\mathbb{R}^2$ where $x = (\alpha_1,\alpha_2,\alpha_3)$ in some basis I want to find the signature of $Q$ dependent on $\alpha$ and the diagonal basis of $Q$.
I was trying to rewrite $Q$ into a sum of squares like this:
$$ Q(x) = (\alpha\alpha_1 + \alpha\alpha_3 + \frac{1}{2}\alpha_2)^2 - \alpha^2\alpha_1^2 - \alpha^2\alpha_3^2 - \frac{1}{4}\alpha_2^2 - \alpha\alpha_3\alpha_2 $$
which yields $$  (\alpha\alpha_1 + \alpha\alpha_3 + \frac{1}{2}\alpha_2)^2 - (\alpha\alpha_1)^2 - (\alpha\alpha_3 + \frac{1}{2}\alpha_2)^2 = Q(x)$$
Since now I have $Q$ represented as a sum of squares, I see the signature being $(0,1,2)$ if $\alpha \neq 0$ and $(1,1,1)$ if $\alpha = 0$. Also the terms in parentheses must be the coordinates of $x$ in the diagonal basis of $Q$:
$$ \beta_1 = \alpha\alpha_1 + \alpha\alpha_3 + \frac{1}{2}\alpha_2, \beta_2 = \alpha\alpha_1, \beta_3 = \alpha\alpha_3 + \frac{1}{2}\alpha_2$$
which corresponds to a matrix
\begin{pmatrix}\alpha & \frac{1}{2} & \alpha\\
\alpha & 0 & 0\\
0 & \frac{1}{2} & \alpha
\end{pmatrix}
To get the diagonal basis, I need to invert this matrix (get alphas represented by betas), but it is clearly singular. What am I doing wrong?

As in your question, let
$$Q(x) = \alpha \alpha_1 \alpha_2 + 2 \alpha^2 \alpha_1 \alpha_3$$
be a quadratic form on $\mathbb{R}^3$ in some basis.  We know what the signature is $(3,0,0)$ if $\alpha = 0$, so let us assume that $\alpha \neq 0$.
Something went wrong in your diagonalization process because you managed to diagonalize to a non-singular quadratic form.  The quadratic form
$$ (\alpha\alpha_1 + \alpha\alpha_3 + \frac{1}{2}\alpha_2)^2 - (\alpha\alpha_1)^2 - (\alpha\alpha_3 + \frac{1}{2}\alpha_2)^2 $$
you got has signature $(0,1,2)$.  You may want to try expanding and then comparing the result with $Q(x)$. 
The Gram matrix
$$\begin{pmatrix} 0 & \frac{\alpha}{2} & \alpha^2 \\ \frac{\alpha}{2} & 0 & 0 \\ \alpha^2 & 0 & 0 \end{pmatrix}$$
of $Q(x)$ is singular because its second and third rows are linearly dependent.  Let us first consider the upper left $2 \times 2$ submatrix which corresponds to the form $\alpha \alpha_1 \alpha_2$.  This form represents all real numbers, so it must have signature $(0,1,1)$.  Thus the first two entries of the diagonalized matrix must be $1$ and $-1$.  Since the original Gram matrix is singular, the final element on the diagonal must be $0$.  Putting everything together, we see that the signature of $Q(x)$ is $(1,1,1)$.
If we carry out the diagonalization by hand, we will find that indeed
$$ \begin{pmatrix} 1 & 1 & 0 \\ \frac{1}{a} & -\frac{1}{a} & -2 \\ 0 & 0 & \frac{1}{a} \end{pmatrix}^{\top} \begin{pmatrix} 0 & \frac{\alpha}{2} & \alpha^2 \\ \frac{\alpha}{2} & 0 & 0 \\ \alpha^2 & 0 & 0 \end{pmatrix} \begin{pmatrix} 1 & 1 & 0 \\ \frac{1}{a} & -\frac{1}{a} & -2 \\ 0 & 0 & \frac{1}{a} \end{pmatrix} = \begin{pmatrix} 1 & 0 & 0 \\ 0 & -1 & 0 \\ 0 & 0 & 0 \end{pmatrix} $$
The change-of-basis matrix is non-singular.
