Triangle ABC with $\angle{ACB} = 3\angle{ABC }$ and $AB = \frac{10}{3}BC$. Find $\cos{A}\cos{B}\cos{C}$ assume that $\angle{ABC}=y, \angle{ACB}=3y, BC=3x,$ and $AB=10x$ then by sine rule, i obtain following

$ \frac{10x}{\sin{3y}}=\frac{AC}{\sin{y}}=\frac{3x}{\sin{4y}}$

by cosine rule in try to figure out AC

$AC=\sqrt{109x^2-60x^2\cos{y}}$

i have no idea how to combine these two informations in order to solve the problem

Set $\cos y=p$, then you can get this equation in sine rule you wrote:
$$
10\sin4y=3\sin 3y
\\
40\sin y\cos y\cos2y=9\sin y-12\sin^3 y
\\
40p(2p^2-1)=9-12(1-p^2)=12p^2-3
\\
80p^3-12p^2-40p+3=(4p-3)(20p^2+12p-1)=0
$$
Then, you can see $\cos B=\frac34$, and the only thing left is the calculation.
