Finding ordered pairs $(a,b)$ which are positive integers for $\sqrt{8+\sqrt{32+\sqrt{768}}}=a\cos\frac {\pi}b$ 
Finding ordered pairs $(a,b)$ for $\sqrt{8+\sqrt{32+\sqrt{768}}}=a\cos\frac {\pi}b$

I did:
$768=2^8\cdot 3$
So:
$$\sqrt{32+\sqrt{768}}=\sqrt{32+2\sqrt{2^6\cdot3}}=\sqrt{8}+\sqrt{24}$$
Now:
$$\sqrt{8+\sqrt{32+\sqrt{768}}}=\sqrt{8+2\sqrt2+2\sqrt6}=?$$

Notice 
$$a\cos\frac{\pi}{b} = \sqrt{8 + \sqrt{32 + \sqrt{768}}}
\quad\implies\quad a^2\cos^2\frac{\pi}{b} - 8 = \sqrt{32 + \sqrt{768}}$$
In order for the equation on the left to have an answer of reasonable size, we should be able to simplify the expression $a^2\cos^2\frac{\pi}{b} - 8$ somehow. This strongly suggest we pick $a = 4$
as an ansatz. Assume $a = 4$, we get
$$\begin{align}
& 8\cos\frac{2\pi}{b} = 16\cos^2\frac{\pi}{b} - 8 =
\sqrt{32 + \sqrt{768}}\\
\implies &
\cos\frac{2\pi}{b} = \sqrt{\frac12 + \frac{\sqrt{3}}{4}}\\
\implies &\cos\frac{4\pi}{b} = 2\cos^2\frac{2\pi}{b} - 1 = \frac{\sqrt{3}}{2} = \cos\frac{\pi}{6}
\end{align}
$$
This suggest $(a,b) = (4,24)$ can be a solution of the original problem. This can be verified by comparing the numerical values of $\;4\cos\frac{\pi}{24}\;$ and $\;\sqrt{8+\sqrt{32+\sqrt{768}}}\;$ and using the fact both of them are roots of the same algebraic equation whose roots are simple.
$$((u^2-8)^2-32)^2-768 = 0$$
If we restrict $a$ to $4$, there are no other solutions for $b$. 
Otherwise, we have
$$\cos\frac{4\pi}{b} = \cos\frac{\pi}{6}
\quad\leadsto\quad \frac{4\pi}{b} \ge \frac{11\pi}{6}
\quad\leadsto\quad b \le \frac{24}{11}
\quad\leadsto\quad b = 1
$$
and $b = 1$ is clearly not a solution!
