11. (20 points) In the Cartesian coordinate system, points $A$, $B$, and $C$ lie on the hyperbola $x y=1$, satisfying that $\triangle ABC$ is an isosceles right triangle. Find the minimum value of the area of $\triangle ABC$.
11. As shown in Figure 2, let's assume the vertices $A, B, C$ of the isosceles right triangle $\triangle ABC$ are arranged counterclockwise, with $A$ being the right-angle vertex.
Let $\overrightarrow{A B}=(s, t)$.
Then $\overrightarrow{A C}=(-t, s)$,

and the area of $\triangle ABC$ is
$$
\begin{array}{l}
S_{\triangle A B C}=\frac{1}{2}|\overrightarrow{A B}|^{2} \\
=\frac{s^{2}+t^{2}}{2} .
\end{array}
$$

Note that, point $A$ lies on the hyperbola $x y=1$.
Let $A\left(a, \frac{1}{a}\right)$. Then
$$
B\left(a+s, \frac{1}{a}+t\right), C\left(a-t, \frac{1}{a}+s\right) \text {. }
$$

Since points $B$ and $C$ lie on the hyperbola $x y=1$, we have
$$
(a+s)\left(\frac{1}{a}+t\right)=(a-t)\left(\frac{1}{a}+s\right)=1 \text {, }
$$

which is equivalent to
$$
\begin{array}{c}
\frac{s}{a}+a t=-s t, \\
-\frac{t}{a}+a s=s t .
\end{array}
$$
Adding (1) and (2) gives
$\frac{s-t}{a}+a(t+s)=0$
$\Rightarrow a^{2}=\frac{t-s}{t+s}$.
Multiplying (1) and (2) and using equation (3) gives
$$
\begin{array}{l}
-s^{2} t^{2}=\left(\frac{s}{a}+a t\right)\left(-\frac{t}{a}+a s\right) \\
=\left(a^{2}-\frac{1}{a^{2}}\right) s t+s^{2}-t^{2} \\
=\left(\frac{t-s}{t+s}-\frac{t+s}{t-s}\right) s t+s^{2}-t^{2} \\
=\frac{4 s t}{s^{2}-t^{2}} \cdot s t+s^{2}-t^{2}=\frac{\left(s^{2}+t^{2}\right)^{2}}{s^{2}-t^{2}} .
\end{array}
$$

Thus, by the AM-GM inequality, we have
$$
\begin{array}{l}
\left(s^{2}+t^{2}\right)^{4}=\left(-s^{2} t^{2}\left(s^{2}-t^{2}\right)\right)^{2} \\
=\frac{1}{4} \cdot 2 s^{2} t^{2} \cdot 2 s^{2} t^{2}\left(s^{2}-t^{2}\right)^{2} \\
\leqslant \frac{1}{4}\left(\frac{2 s^{2} t^{2}+2 s^{2} t^{2}+\left(s^{2}-t^{2}\right)^{2}}{3}\right)^{3} \\
=\frac{\left(s^{2}+t^{2}\right)^{6}}{108} \\
\Rightarrow s^{2}+t^{2} \geqslant \sqrt{108}=6 \sqrt{3} .
\end{array}
$$

Now, let's find a set of real numbers $(s, t, a)$ that satisfy the conditions, such that $s^{2}+t^{2}=6 \sqrt{3}$ (and thus by $s, t, a$ we can determine a $\triangle ABC$ that satisfies the conditions, with $S_{\triangle A B C}=\frac{s^{2}+t^{2}}{2}=3 \sqrt{3}$).
Considering the equality condition of equation (4), we have
$$
2 s^{2} t^{2}=\left(s^{2}-t^{2}\right)^{2} \Rightarrow \frac{s^{2}}{t^{2}}=2 \pm \sqrt{3} \text {. }
$$

Assuming $0<s<t$, and combining with $s^{2}+t^{2}=6 \sqrt{3}$, we get $s=\sqrt{3(\sqrt{3}-1)}, t=\sqrt{3(\sqrt{3}+1)}$.
From equation (1), we know $a<0$. Hence, from equation (3), we get
$a=-\sqrt{\frac{t-s}{t+s}}\left(t=s \sqrt{\frac{\sqrt{3}+1}{\sqrt{3}-1}}=\frac{\sqrt{3}+1}{\sqrt{2}} s\right)$.
Therefore, $a=-\sqrt{\frac{\sqrt{3}+1-\sqrt{2}}{\sqrt{3}+1+\sqrt{2}}}$.
In summary, the minimum area of $\triangle ABC$ is $3 \sqrt{3}$.