Extremal problem with positive integer numbers Let $a,b$ be two positive integer numbers such that $a\sqrt{3}-b\sqrt{7}>0$. Find the minimum value of
$$
S=(a\sqrt{3}-b\sqrt{7})(a+b).
$$
Attempt I have tried and guess that the minimum value of $S$ is $(55+36)(55\sqrt{3}-36\sqrt{7})$, where $(55,36)$ is the integer solution of the Pell equation $3a^2-7b^2=3$.

You can always find solutions $(a,b)$ that give smaller values of $S$. 
Consider solutions to the equation $x^2-21y^2=1$, $(x_1,y_1)=(55,12),(x_2,y_2)=(6049,1320)$ solutions from $(x_3,y_3)$ onward can be generated using solutions to the recursive solution $$x_{k+1}=x_1x_k+21y_1y_k\\y_{k+1}=x_1y_k+y_1x_k$$
Solutions that satisfy $3a^2-7b^2=3$ are then given by $(a_i,b_i)=(x_i,3y_i)$
We can also write $(a,b)$ as $(\sec t, \sqrt\frac{3}
{7}\tan t$. Subtitute this in the expression for $S$.
$$S=(\sqrt{3}\sec t-\sqrt{7}\sqrt\frac{3}
{7}\tan t)(\sec t+\sqrt\frac{3}
{7}\tan t)\\ =\sqrt3(\sec t-\tan t)(\sec t+\sqrt\frac{3}
{7}\tan t)$$
Now at the global minimum we must have $dS/dt = 0$. We compute the derivative and we get $$\frac{dS}{dt}=\frac{\sqrt3}7(\sqrt{21}-1)\sec t(\sec t - \tan t)^2=0 \\ \implies \sec t - \tan t=0.$$
The solutions to this lie at $t=\frac{\pi}2+2n\pi$. At these points $(a,b)=(\infty, \infty)$, hence there is no upper bound on integer solutions. 
