Express a quadratic form in three variable in the format $x^tAx$ using a substitution $x=Py$ I was asked to determine if a quadratic form is positive definite. To do so I must convert in the format $x^tAx$ using a substitution x=Py. So that "it can be written in diagonal form".
$$Q(x,y,z) = 3x^2 + 8xz+2y^2+z^2$$
My idea is:
$$(x,y,z) \begin{pmatrix} 3x+4z \\ 2y \\ 4x+z^2 \end{pmatrix} $$ 
$$\begin{pmatrix} 3&0&4 \\0&2&0 \\ 4&0&1 \end{pmatrix} $$ 
Is this correct? And then it would be positive definite if all eigen values of this are equal to or larger than 0?

Ummm, $$ (4x+z)^2 = 16 x^2 + 8 x z + z^2, $$ so
$$ -13 x^2 +  (4x+z)^2 = 3 x^2 + 8 x z + z^2,  $$
$$ -13 x^2 +  (4x+z)^2 + 2 y^2 = 3 x^2 + 8 x z + z^2 + 2 y^2 = Q(x,y,z).  $$
Note $Q(1,0,-1) = -4$
If
$$
D = 
\left(
\begin{array}{ccc}
-13 & 0 & 0 \\
0 & 1 & 0 \\
0 & 0 & 2
\end{array}
\right)
$$
and
$$
R = 
\left(
\begin{array}{ccc}
1 & 0 & 0 \\
4 & 0 & 1 \\
0 & 1 & 0
\end{array}
\right),
$$
what is
$$ R^T D R? $$
