求函数 $f(x, y)=\left(y-x^{2}\right)\left(y-x^{3}\right)$ 的极值.
$\left\{\begin{array}{l}f_{x}^{\prime}=-x\left(2 y+3 x y-5 x^{3}\right)=0 \\ f_{y}^{\prime}=2 y-x^{2}-x^{3}=0\end{array}\right.$, 得驻点为 $(0,0),(1,1),\left(\frac{2}{3}, \frac{10}{27}\right)$.
$f_{x x}^{\prime \prime}=-\left(2 y+3 x y-5 x^{3}\right)-x\left(3 y-15 x^{2}\right), f_{x y}^{\prime \prime}=-x(2+3 x), \quad f_{y y}^{\prime \prime}=2$.
代入 $(0,0),\left\{\begin{array}{l}A=f_{x x}^{\prime \prime}=0 \\ B=f_{x y}^{\prime \prime}=0, \\ C=f_{y y}^{\prime \prime}=2\end{array}\right.$ 则 $A C-B^{2}=0$, 故充分条件失效, 当 $x \rightarrow 0$ 时, 取
$y=x^{2}+k x^{3}(k>0), \quad f(x, y)=\left(y-x^{2}\right)\left(y-x^{3}\right)=k x^{3}\left[x^{2}+(k-1) x^{3}\right]=k x^{5}+o\left(x^{5}\right)$,
则 $\lim _{x \rightarrow 0} \frac{f(x, y)}{x^{5}}=\lim _{x \rightarrow 0} \frac{k x^{5}+o\left(x^{5}\right)}{x^{5}}=k>0$, 由极限的局部保号性: 存在 $\delta>0$, 当 $x \in(-\delta, 0)$ 时, $\frac{f(x, y)}{x^{5}}>0, f(x, y)<0=f(0,0)$, 当 $x \in(0, \delta)$ 时, $\frac{f(x, y)}{x^{5}}>0$, $f(x, y)>0=f(0,0)$, 故 $(0,0)$ 不是极值点;
代入 $(1,1),\left\{\begin{array}{l}A=f_{x x}^{\prime \prime}=12 \\ B=f_{x y}^{\prime \prime}=-5, \\ C=f_{y y}^{\prime \prime}=2\end{array}\right.$ 则 $A C-B^{2}<0$, 故 $(1,1)$ 不是极值点;
代入 $\left(\frac{2}{3}, \frac{10}{27}\right)$ 的 $\left\{\begin{array}{l}A=f_{x x}^{\prime \prime}=\frac{100}{27} \\ B=f_{x y}^{\prime \prime}=-\frac{8}{3}, \\ C=f_{y y}^{\prime \prime}=2\end{array}\right.$ 则 $A C-B^{2}>0$ 且 $A>0$, 故 $\left(\frac{2}{3}, \frac{10}{27}\right)$ 是极小值点;故 $f\left(\frac{2}{3}, \frac{10}{27}\right)=-\frac{4}{729}$ 为极小值.