Generalization of $1988$ IMO #6 $1988$ IMO Problem 6 states:

Let $a$ and $b$ be positive integers such that $ab + 1$ divides $a^2 + b^2$. Show that
$$\frac{a^2 + b^2}{ab + 1}$$
is the square of an integer.

My question is: what if $a$ and $b$ don't need to be positive? What other values of $\frac{a^2 + b^2}{ab + 1}$ are possible? I found a way to get $-5$ by setting $a = -1$ and $b = 2$. Is there a simple classification of such reachable integers?

I prefer to graph everything. The thing that Vieta Jumping does is to guarantee a point (if there are any integer points)  on a short arc of the hyperbola. Then  inequalities based on the size of $n$ tell us whether there can be any such integer points.
Here $$  \frac{x^2 + y^2}{xy-1}  = n $$
with $x,y > 0.$

If there are any integer points at all,  there are jumped integer points  that lie on the arc between the endpoints
$$  \left( \; \sqrt{\frac{4n}{n^2-4}} \; , \; \;  \sqrt{\frac{n^3}{n^2-4}} \; \; \right) $$
and
$$  \left( \; \sqrt{\frac{n^3}{n^2-4}} \; , \; \;  \sqrt{\frac{4n}{n^2-4}} \; \; \right) $$
One must consider $n=3,4$ separately. For $n= 5$  there are indeed integer points  at $(1,2) , \; \;  (2,1).$
For $n \geq 6$   we see the picture below (done with $n=16.$  The axis of symmetry $y=x$  meets the hyperbola at $(t,t)$  where $t = \sqrt{   \frac{n}{n-2}},$  between $1$ and $2$
For $x \geq \sqrt{   \frac{n}{n-2}},$  but $x \leq  \sqrt{\frac{n^3}{n^2-4}} , $  we see that $y$ is decreasing.  Furthermore, for $x = 2$  we find
$$ y_2 =  \frac{n+4}{n+\sqrt{n^2 - n - 4}}  .$$
Thus, for
$$ 2 \leq x  \leq   \sqrt{\frac{n^3}{n^2-4}}$$
we see $0 < y \leq \frac{n+4}{n+\sqrt{n^2 - n - 4}} .$  For $n \geq 6$ this is smaller than $1$
So that's it:  we can jump an integer point to an integer point  along a certain bounded arc of the hyperbola. Then, for $n \geq 6$  we see there are no integer points within that arc. So, no integer points at all.
