$5.12 \cos ^{2} 3 x+\cos ^{2} 4 x+\cos ^{2} 5 x=1.5$
5.12 Let's use the formula for reducing the power (4.17):

$2\left(\cos ^{2} 3 x+\cos ^{2} 4 x+\cos ^{2} 5 x\right)=3$

$1+\cos 6 x+1+\cos 8 x+1+\cos 10 x=3$

$\cos 6 x+\cos 8 x+\cos 10 x=0 ; \cos 8 x+2 \cos 8 x \cos 2 x=0 ;$

1) $\cos 8 x=0 ; 8 x=\frac{\pi}{2}+\pi k ; x_{1}=\frac{\pi}{16}(2 k+1)$;
2) $\cos 2 x=-\frac{1}{2} ; 2 x= \pm \frac{2 \pi}{3}+2 \pi k ; x_{2}=\frac{\pi}{3}(3 k \pm 1)$.

Answer: $\quad x_{1}=\frac{\pi}{16}(2 k+1), x_{2}=\frac{\pi}{3}(3 k \pm 1), k \in Z$.