In $\triangle ABC$, if $AC=4$ and $BC=5$, and $\cos(A-B)=\frac{7}{8}$, find $\cos(C)$ The problem is as the title suggests, in the given figure below, the goal is to find the Cosine of $\angle C$. I tried multiple ways of approaching this, such as with the Law of Sines, area formula, etc but none of them seemed to lead anywhere. My actual approach, which I will post as an answer below, uses the law of Cosines. Please share your own approaches especially if they use a different method!


A novel way to solve the problem is to change the question, with equvalent answer.
$\cos(A-B) \;=\; \sin(A)\sin(B) \;+\; \cos(A)\cos(B)$
$\cos(\quad C \quad) \;=\; \sin(A)\sin(B) \;-\; \cos(A)\cos(B)$
Only difference between the two are the sign of cosine product term.
We change the question! $\; a=5,\; b=4,\;\cos(C)=7/8,\;\text{find}\;\cos(A-B)$
$c^2 = a^2 + b^2 - 2\,a\,b\,\cos(C) = 5^2+4^2-2×5×4×\frac{7}{8} = 6$
$\displaystyle 2R = \frac{a}{\sin A} = \frac{b}{\sin B} = \frac{c}{\sin C} 
= \sqrt{\frac{6}{1-(7/8)^2}} = \sqrt{\frac{128}{5}}$
Compare with original question, we have the same $R$
$\displaystyle \cos(A-B) + \cos(C) = 2\,\sin(A)\sin(B)$
$\displaystyle \cos(A-B) = -\cos(C) + \frac{2\,a\,b}{4R^2} 
= -\frac{7}{8} \;+\; \frac{2×5×4}{128/5}
= \frac{11}{16}$
--> Original question, $\displaystyle\;\cos(C) = \frac{11}{16}$
