$A,B$ coefficients for maximal convergence 
In order to approximate the solution to the equation $f(x) = x^2-2$ we use the following iteration:
  $$x_{n+1} = x_n + A\frac{x_n^2 -2}{x^n} + B \frac{x_n^2-2}{x_n^3}$$ 
  Find $A,B$ for maximal rate of convergence.

So first we can easily see that it is a proper function since if we apply $\sqrt 2$ for $x_n$ we indeed get $x_{n+1} = \sqrt 2$.
Now, I've understood that in order to get a maximal rate of convergence one should check when the derivatives are vanished. Is that the case here? What exactly the criteria? 
I'd be glad for guidance.
Thanks!


By varying the coefficients $A$ and $B$, we can achieve some mathematical convenience, this is called method of undetermined coefficients.

Let $x_{n}=\sqrt{2}+e_{n}$, then
\begin{align*}
  x_{n+1} &= x_{n}+A\frac{x_{n}^{2}-2}{x_{n}}+B\frac{x_{n}^{2}-2}{x_{n}^{3}} \\
  &= \sqrt{2}+e_{n}+A\frac{(\sqrt{2}+e_{n})^{2}-2}{\sqrt{2}+e_{n}}+
      B\frac{(\sqrt{2}+e_{n})^2-2}{(\sqrt{2}+e_{n})^3} \\
  &= \sqrt{2}+e_{n}+A(\sqrt{2}+e_{n})+\frac{B-2A}{\sqrt{2}+e_{n}}-
     \frac{2B}{(\sqrt{2}+e_{n})^{3}} \\
  &= \sqrt{2}+e_{n}+A(\sqrt{2}+e_{n})+(B-2A)
     \left(
       \frac{1}{\sqrt{2}}-\frac{e_{n}}{2}+
       \frac{e_{n}^{2}}{2\sqrt{2}}-\frac{e_{n}^{3}}{4}+\ldots
     \right) \\
  & \quad  -2B
    \left(
       \frac{1}{2\sqrt{2}}-\frac{3e_{n}}{4}+
       \frac{3e_{n}^{2}}{2\sqrt{2}}-\frac{5e_{n}^{3}}{4}+\ldots
     \right) \\
  &= \sqrt{2}+(1+2A+B)e_{n}+\frac{2A+5B}{2\sqrt{2}}e_{n}^{2}+
     \frac{2A+9B}{4}e_{n}^{3}+\ldots
\end{align*}
To optimize the rate of convergence, we make the linear and quadratic terms in $e_{n}$ vanished:
$$
\left \{
  \begin{align*}
  1+2A+B &= 0 \\
  2A+5B &= 0
\end{align*}
\right.$$
On solving, we have
$$(A,B)=\left( -\frac{5}{8},\frac{1}{4} \right)$$
Substitute back into $x_{n+1}$, we obtain cubic convergence:
$$e_{n+1} \approx \frac{e_{n}^3}{4}$$
