Problem with Lagrange multipliers I am asked to find local extrema of $f(x,y,z)=ax+by$ ($a,b$ non-zero and fixed) defined on $\{(x,y,z)\colon (x,y)\neq 0\}$ subject to
$$\left (R-\sqrt{x^2+y^2}\right)^2 + z^2 - r^2 = 0.$$
(here $0<r<R$ are fixed). Okay, let us define the auxiliary function by:
$$F(x,y,z) = ax+by - \lambda \left( \left(R-\sqrt{x^2+y^2}\right)^2 + z^2 - r^2 \right).$$
How to get rid of $\lambda$ from $F^\prime_x, F^\prime_y, F^\prime_z$? Can one please help me to find stationary points of $F$?

First we calculate $F_x'$, $F_y'$, and $F_z'$:
$$F_x'=a-\lambda\left(2\left(R-\sqrt{x^2+y^2}\right)\times\frac{-2x}{2\sqrt{x^2+y^2}}\right)=a+\frac{2\lambda x\left(R-\sqrt{x^2+y^2}\right)}{\sqrt{x^2+y^2}}$$
$$F_y'=b-\lambda\left(2\left(R-\sqrt{x^2+y^2}\right)\times\frac{-2y}{2\sqrt{x^2+y^2}}\right)=b+\frac{2\lambda y\left(R-\sqrt{x^2+y^2}\right)}{\sqrt{x^2+y^2}}$$
$$F_z'=-2\lambda z$$
To make these equations nicer, let's define $u:=\frac{2\left(R-\sqrt{x^2+y^2}\right)}{\sqrt{x^2+y^2}}$. Now, setting the derivatives to $0$, we have the following system of equations:
$$0=a+\lambda x u$$
$$0=b+\lambda y u$$
$$0=-2\lambda z$$
From the final equation, we have either $\lambda=0$ (in which case we have a solution if and only if we also have $a=b=0$ and so $f(x,y,z)=0$), or $z=0$. Since we have dealt with the former case, we'll assume from here that $z=0$.
Now multiplying the first equation by $y$, the second by $x$, and taking the difference, we get
$$0=ay+\lambda xyu-bx-\lambda xyu=ay-bx$$
so we have $y=\dfrac{b}{a}x$.
Finally, we can plug this into our constraint equation:
$$\begin{align}&0=\left (R-\sqrt{x^2+y^2}\right)^2 + z^2 - r^2
\\\implies&0=\left (R-\sqrt{x^2+\left(\frac{b}{a}\right)^2x^2}\right)^2 - r^2
\\\implies&r^2=\left (R-|x|\sqrt{1+\left(\frac{b}{a}\right)^2}\right)^2
\\\implies&\pm r=R-|x|\sqrt{1+\left(\frac{b}{a}\right)^2}
\\\implies&|x|=\frac{R\pm r}{\sqrt{1+\left(\frac{b}{a}\right)^2}}
\\\implies&x=\frac{\pm R\pm r}{\sqrt{1+\left(\frac{b}{a}\right)^2}}
\end{align}$$
Since we have $y=\frac{b}{a}x$,
$$y=\frac{\pm R\pm r}{\sqrt{1+\left(\frac{a}{b}\right)^2}}$$
Hence the critcal points are
$$\frac{a(\pm R\pm r)}{\sqrt{1+\left(\frac{b}{a}\right)^2}}+\frac{b(\pm R\pm r)}{\sqrt{1+\left(\frac{a}{b}\right)^2}}$$
Note in this final expression if we choose signs for $R$ and $r$ in the $ax$ term, we need to choose the same signs in the $by$ term.
