11. (20 points) In the Cartesian coordinate system $x O y$, $A(-a, 0), B(a, 0)$, where the constant $a>0$. Moving points $P$ and $Q$ move above the $x$-axis, satisfying $A P \perp P Q, B Q \perp P Q$, and the quadrilateral $A B Q P$ is a convex quadrilateral with a constant area $S$. Find the equation of the boundary of the set of all points on the moving line $P Q$.

11. According to the problem, the intercept of line $P Q$ on the $y$-axis exists and is greater than 0. Therefore, we can set $l_{P Q}: y=k x+m (m>0)$, and the inclination angle of line $P Q$ is $\theta$.
Draw $O R \perp P Q$ at point $R$.
Then $S=S_{\text {quadrilateral } A B Q P}=|O R||P Q|$
$=\frac{m}{\sqrt{k^{2}+1}}|A B||\cos \theta|=\frac{2 a m}{k^{2}+1}$
$\Rightarrow m=\frac{\left(k^{2}+1\right) S}{2 a}$.
Thus, $l_{P Q}: y=k x+\frac{\left(k^{2}+1\right) S}{2 a}$.
Considering the equation of line $P Q$ as a quadratic equation in $k$, we rearrange it to get
$$
\frac{S}{2 a} \cdot k^{2}+x k+\left(\frac{S}{2 a}-y\right)=0 .
$$

The discriminant $\Delta$ is
$$
\begin{array}{l}
=x^{2}-4 \times \frac{S}{2 a}\left(\frac{S}{2 a}-y\right) \geqslant 0 \\
\Rightarrow y \geqslant-\frac{a}{2 S} x^{2}+\frac{S}{2 a},
\end{array}
$$

which means that any point $(x, y)$ on line $P Q$ must satisfy
$$
y \geqslant-\frac{a}{2 S} x^{2}+\frac{S}{2 a} .
$$

Therefore, the required equation is $y=-\frac{a}{2 S} x^{2}+\frac{S}{2 a}$.