If $\cos2\theta=0$, then $\Delta^2=$? I'll state the question from my textbook here:

If $\cos2\theta=0$, then $\begin{vmatrix} 0 & \cos \theta & \sin \theta \\ \cos \theta & \sin \theta & 0 \\ \sin \theta & 0 & \cos \theta \end{vmatrix}^2=$?

This is how I solved the problem:
$\begin{vmatrix} 0 & \cos \theta & \sin \theta \\ \cos \theta & \sin \theta & 0 \\ \sin \theta & 0 & \cos \theta \end{vmatrix}^2$
$= (\cos^3 \theta + \sin^3 \theta)^2$
$= (\cos \theta + \sin \theta)^2(\cos^2 \theta - \cos \theta \sin \theta + \sin^2 \theta)^2$
$= (1+ \sin2\theta)(1-\sin2\theta + \sin^2 \theta \cos^2 \theta)$
$= (1+ \sin2\theta)(1-\sin2\theta) + (1 + \sin2\theta)\sin^2\theta \cos^2 \theta$
$= \cos^2 2\theta + \frac 14 (1 + \sin2\theta)\sin^2 2\theta$
$= \frac 14 (1 + \sin2\theta)\sin^2 2\theta$
Now since $\cos2\theta=0$, $\sin2\theta = \pm 1$.
Therefore the above expression can take the values 0 and $\frac12$.
My textbook gives the answer as $\frac12$. I don't see any grounds on rejecting the other answer of 0. Have I made a mistake somewhere? Or am I forgetting something?

When in doubt, use the relations in the original problem. Let $\theta=\frac{3\pi}4$. Then $\cos2\theta=0$, while $\sin\theta=\sqrt2/2=a$ and $\cos\theta=-\sqrt2/2=-a$.
The expression in the question is now
$$\begin{vmatrix}0&-a&a\\-a&a&0\\a&0&-a\end{vmatrix}^2$$
Clearly, adding up all three rows of the matrix produces the zero vector, so the whole expression evaluates to zero. Your working is entirely correct: the book is wrong to omit 0 as an answer.
The result of $\frac12$ is obtained with the other principal solution to $\cos2\theta=0$, $\theta=\frac\pi4$.
