If $\cos3A + \cos3B + \cos3C = 1$ in a triangle, find one of its length I would like to solve the following problem.

In $\triangle ABC, AC = 10, BC = 13$. If $\cos3A + \cos3B + \cos3C = 1$, compute the length of $AB$.

I thought that I could apply the Law of Cosines. Using the fact that $A+B+C=\pi$, I attempted to build the equation up from there.
What I got was that $$\cos3A+\cos3B-\cos(3A+3B)=1$$
Expanding, I got $$\cos3A+\cos3B-\cos3A\cos3B-\sin3A\sin3B=1$$
Now, it's possible that I'd be able to factor it somehow by rewriting it all in terms of cosine and arrive at the answer, but is there a better way to solve the problem? Thanks!

Using the identity
\begin{align} 
\cos3\alpha+\cos3\beta+\cos3\gamma
&=
1-\frac{r\,(3\,\rho^2-(r+3\,R)^2)}{R^3}
\tag{1}\label{1}
,
\end{align}
where $r$ is the inradius, $R$ is the circumradius
and $\rho$ is the semiperimeter 
of the triangle, 
condition 
\begin{align} 
\cos3\alpha+\cos3\beta+\cos3\gamma&=1
\end{align}
results in
\begin{align} 
3\,\rho^2-(r+3\,R)^2&=0
,\\
3\,\rho^2-\left(\frac{S}{\rho}+3\,\frac{abc}{4S}\right)^2&=0
,\\
3\,\rho^2
-\frac{S^2}{\rho^2}
-\frac{3abc}{2\rho}
-\tfrac{9}{16}\frac{(abc)^2}{S^2}&=0
\tag{2}\label{2}
,\\
\end{align}
where $S$ is the area of triangle.
With the help of substitution
\begin{align}
\rho&=\tfrac12(a+b+c)
,\\
S^2&=\tfrac1{16}(a+b+c)(-a+b+c)(a-b+c)(a+b-c)
,\\
a&=13
,\quad
b=10
\end{align}  
equation \eqref{2} becomes
\begin{align}
\frac{(c^2-399)(c^2+13c+69)(c^2+10c-69)}{(c^2-9)(c^2-23^2)}
&=0
,
\end{align}
with just two suitable roots,
\begin{align}
c_1&=\sqrt{399}
,\\
c_2&=\sqrt{94}-5
.
\end{align}

