B. As shown in Figure 2, in the square $A B C D$ with side length 1, $E$ and $F$ are points on $B C$ and $C D$ respectively, and $\triangle A E F$ is an equilateral triangle. Then the area of $\triangle A E F$ is
B. $2 \sqrt{3}-3$.

From the given information, we have
$\triangle A B E \cong \triangle A D F \Rightarrow B E=D F, E C=F C$.
Let $E C=F C=a$. Then
$$
A E=E F=\sqrt{2} a, B E=1-a \text {. }
$$

Thus, $A E=\sqrt{A B^{2}+B E^{2}}$
$$
\begin{array}{l}
=\sqrt{1+(1-a)^{2}}=\sqrt{2} a \\
\Rightarrow a^{2}+2 a-2=0 .
\end{array}
$$

Since $a>0$, we have $a=\sqrt{3}-1$.
Therefore, $S_{\triangle A E F}=\frac{\sqrt{3}}{4} E F^{2}=\frac{\sqrt{3}}{4}(\sqrt{2} a)^{2}$
$$
=\frac{\sqrt{3}}{2}(\sqrt{3}-1)^{2}=2 \sqrt{3}-3 \text {. }
$$