5. As shown in Figure 3, a circular paper piece with radius $r$ moves arbitrarily within an equilateral triangle with side length $a(a \geqslant 2 \sqrt{3} r)$. Then, within this equilateral triangle, the area of the part that the circular paper piece "cannot touch" is ( ).
(A) $\frac{\pi}{3} r^{2}$
(B) $\frac{(3 \sqrt{3}-\pi)}{3} r^{2}$
(C) $(3 \sqrt{3}-\pi) r^{2}$
(D) $\pi r^{2}$
5. C.

As shown in Figure 11, when the circular paper piece moves to a position where it is tangent to the two sides of $\angle A$, draw perpendiculars from the center $O_{1}$ of the circular paper piece to the two sides, with the feet of the perpendiculars being $D$ and $E$, respectively, and connect $A O_{1}$. Then, in the right triangle $\triangle A D O_{1}$,
$$
\begin{array}{l}
\angle O_{1} A D=30^{\circ}, \\
O_{1} D=r, \\
A D=\sqrt{3} r .
\end{array}
$$

Thus, $S_{\triangle A D O_{1}}=\frac{1}{2} O_{1} D \cdot A D=\frac{\sqrt{3}}{2} r^{2}$.
Therefore, $S_{\text {quadrilateral } A D O_{1} E}=2 S_{\triangle A D O_{1}}=\sqrt{3} r^{2}$.
Given that $\angle D O_{1} E=120^{\circ}$, we have
$S_{\text {sector } O_{1} D E}=\frac{\pi}{3} r^{2}$.
Hence, the area of the part that the circular paper piece cannot touch is
$$
3\left(\sqrt{3} r^{2}-\frac{\pi}{3} r^{2}\right)=(3 \sqrt{3}-\pi) r^{2} .
$$