Prove the given determinant Prove the given determinant:
$$\left|
\begin{matrix}
a&b&ax+by \\
b&c&bx+cy \\
ax+by&bx+cy&0 \\
\end{matrix}\right|=(b^2-ac)(ax^2+2bxy+cy^2)$$
I didn't get any idea. Please help me solve this.

A bit late in the day, here is the symmetric matrix version of this, writing $P^TAP = B,$ where $B$ is also symmetric with obvious determinant. We say that $A$ and $B$ are "congruent"
$$
\left(
\begin{array}{ccc}
1&0&0 \\
0&1&0 \\
-x&-y&0 \\
\end{array}
\right)
\left(
\begin{array}{ccc}
a&b&ax+by \\
b&c&bx+cy \\
ax+by&bx+cy&0 \\
\end{array}
\right)
\left(
\begin{array}{ccc}
1&0&-x \\
0&1&-y \\
0&0&1 \\
\end{array}
\right) =
\left(
\begin{array}{ccc}
a&b&0 \\
b&c&0 \\
0&0& - \left(a x^2 + 2bxy+cy^2 \right) \\
\end{array}
\right)
$$
