Show that $\int_0^{\pi/3} \big((\sqrt{3}\cos x-\sin x)\sin x\big)^{1/2}\cos x \,dx =\frac{\pi\sqrt{3}}{8\sqrt{2}}. $ I have run a FORTRAN code and I have obtained strong evidence that
$$\int_0^{\pi/3} \!\!
\big((\sqrt{3}\cos\vartheta-\sin\vartheta)\sin\vartheta\big)^{\!1/2}\!\cos\vartheta \,d\vartheta
=\frac{\pi\sqrt{3}}{8\sqrt{2}}. 
$$
In fact, it looks like my numerical integration method (the trapezoid rule) converges to the value $\dfrac{\pi\sqrt{3}}{8\sqrt{2}}$, with at least $12$ significant digits.
Any ideas how to prove this result analytically?

OK, you can start by noting that
$$\sqrt{3} \cos{x} - \sin{x} = 2 \sin{\left ( \frac{\pi}{3}-x\right)}$$
which means that the integral is
$$ \sqrt{2} \int_0^{\pi/3} dx \, \cos{x} \sqrt{\sin{x} \sin{\left ( \frac{\pi}{3}-x\right)}} = \sqrt{2} I$$
You can make a substitution $x \mapsto \frac{\pi}{3}-x$ and see that
$$I = \sqrt{3} \int_0^{\pi/3} dx \, \sin{x} \sqrt{\sin{x} \sin{\left ( \frac{\pi}{3}-x\right)}}$$
EDIT
Integrate by parts:
$$\begin{align}I &= \left [\sin{x} \sqrt{\sin{x} \sin{\left ( \frac{\pi}{3}-x\right)}} \right ]_0^{\pi/3} - \int_0^{\pi/3} dx \, \sin{x} \frac{d}{dx} \sqrt{\sin{x} \sin{\left ( \frac{\pi}{3}-x\right)}}\\ &= -\frac12 \int_0^{\pi/3} dx \, \sin{x} \frac{\cos{x} \sin{\left ( \frac{\pi}{3}-x\right)} - \sin{x} \cos{\left ( \frac{\pi}{3}-x\right)}}{\sqrt{\sin{x} \sin{\left ( \frac{\pi}{3}-x\right)}}}\\ &= -\frac12 I + \frac12  \int_0^{\pi/3} dx \, \sin{x} \sqrt{\frac{\sin{x}}{\sin{\left ( \frac{\pi}{3}-x\right)}} } \cos{\left ( \frac{\pi}{3}-x\right)}  \end{align}$$
This means that
$$\begin{align}3 I &= \int_0^{\pi/3} dx \, \frac{\cos{x}}{\sin{x}} \sqrt{\sin{x} \sin{\left ( \frac{\pi}{3}-x\right)}} \cos{\left ( \frac{\pi}{3}-x\right)}\\ &= \frac{\sqrt{3}}{2}  \int_0^{\pi/3} dx \, \frac{\cos^2{x}}{\sin{x}} \sqrt{\sin{x} \sin{\left ( \frac{\pi}{3}-x\right)}} - \frac12 I  \end{align}$$
Combining again...
$$7 I = \sqrt{3} \int_0^{\pi/3} dx \sqrt{\frac{\sin{\left ( \frac{\pi}{3}-x\right)}}{\sin{x}}} - \underbrace{\sqrt{3} \int_0^{\pi/3} dx \, \sin{x} \sqrt{\sin{x} \sin{\left ( \frac{\pi}{3}-x\right)}}}_{\text{We know from above this equals } I}$$
Thus
END EDIT
$$I = \frac{\sqrt{3}}{8} \int_0^{\pi/3} dx \sqrt{\frac{\sin{x}}{\sin{\left ( \frac{\pi}{3}-x\right)}}}$$
Now, in a similar manipulation as in the evaluation of this integral, sub $u = \sin{x}/\sin{(\pi/3-x)}$ and find that the integral becomes
$$I = \frac{3}{16} \int_0^{\infty} du \frac{\sqrt{u}}{1+u+u^2}$$
This integral is very straightforward to evaluate via residues using, e.g., a keyhole contour about the positive real axis. By the residue theorem, the original integral is then
$$\sqrt{2} I = \frac{3 \sqrt{2}}{16} \frac12 i 2 \pi \left (\frac{e^{i \pi/3}}{i \sqrt{3}} - \frac{e^{i 2 \pi/3}}{i \sqrt{3}} \right ) = \frac{\pi}{8} \sqrt{\frac{3}{2}}$$
