How to factor the quadratic polynomial $2x^2-5xy-y^2$? How do I factor this polynomial:  $2x^2-5xy-y^2$ ?

EDIT: Your example cannot be factored over the integers, because $25 + 8 = 33$ is not a perfect square. That is, taking $$ a x^2 + b x y + c y^2 = 2 x^2 - 5 x y - y^2,  $$ we get
$$ a = 2, \; b = -5, \; c = -1, \; \Delta = b^2 - 4 a c = (-5)^2 - 4 \cdot 2 \cdot (-1)= 25 + 8 = 33,  $$
and $33$ is nonnegative but is not a square.
ORIGINAL: Either the one-variable function $$ a x^2 + b x + c  $$ or the quadratic form
$$ a x^2 + b x y + c y^2,    $$ with integers $a,b,c,$ factor over the rational numbers if and only if the discriminant 
$$ \Delta = b^2 - 4 a c  $$ is a square.
You are familiar with this because of the quadratic formula
$$ \frac{-b \pm \sqrt \Delta}{2a}    $$
which gives the roots $x$ for $ a x^2 + b x + c =0. $
Only a little changes when inserting the letter $y,$ giving  $ a x^2 + b x y + c y^2.    $ First, if both $a,c$ are $0,$ then we have $bxy$ which is already factored. So, let me show the traditional case, when $a \neq 0.$ Also let
$$  \Delta = \delta^2,   $$ say with integer $\delta \geq 0.$
$$ a x^2 + b x y + c y^2 = \frac{1}{4a} \left( 4 a^2 x^2 + 4 a b x y + 4 a c y^2  \right)   $$
$$   = \frac{1}{4a} \left( 4 a^2 x^2 + 4 a b x y + b^2 y^2 - b^2 y^2 + 4 a c y^2  \right)    $$
$$   = \frac{1}{4a} \left( 4 a^2 x^2 + 4 a b x y + b^2 y^2 - (b^2 y^2 - 4 a c y^2)  \right)    $$
$$   = \frac{1}{4a} \left( 4 a^2 x^2 + 4 a b x y + b^2 y^2 - (b^2  - 4 a c ) y^2  \right)    $$
$$   = \frac{1}{4a} \left( 4 a^2 x^2 + 4 a b x y + b^2 y^2 - \Delta y^2  \right)    $$
$$   = \frac{1}{4a} \left( (2ax+by)^2 - \Delta y^2  \right)    $$
$$   = \frac{1}{4a} \left( (2ax+by)^2 - \delta^2 y^2  \right)    $$
$$   = \frac{1}{4a} \left( \; (2ax+by + \delta y) \; (2ax+by - \delta y)  \; \right)    $$
$$   = \frac{1}{4a} \left( \; (2ax+ (b + \delta) y) \; (2ax+ (b - \delta) y) \; \right)    $$
Now, either $b,\delta$ are both even or both odd. Either way, we may absorb a factor of $4$ into
$$   = \frac{1}{a} \left( \; (ax+ \frac{(b + \delta)}{2} y) \; (ax+ \frac{(b - \delta)}{2} y) \; \right)    $$
Finally, since
$$  \frac{(b + \delta)}{2} \frac{(b - \delta)}{2} = \frac{b^2 - \Delta}{4} = ac   $$
is divisible by $a,$ by unique factorization we may write 
$$a = a_1 a_2$$ with $a_1$ dividing the first fraction and $a_2$ the second, thus finally getting
$$  a x^2 + b x y + c y^2 =  \; \left(a_2x+ \left( \frac{b + \delta}{2a_1} \right) y \right) \; \; \left(a_1x+ \left( \frac{b - \delta}{2a_2} \right) y \right) \;    $$ in integers.
NOTE, October 3, 2013: we may simply take $$ a_1 = \gcd \left( a, 
  \frac{(b + \delta)}{2}  \right) ; \; \; \; \; \; a_2 = \frac{a}{a_1}  $$ without paying attention to any prime factorizations.
