5. As shown in Figure 1, in $\triangle A B C$, it is known that $O$ is the midpoint of $B C$, and points $M$ and $N$ are on sides $A B$ and $A C$ respectively, and
$$
\begin{array}{l}
A M=6, M B=4, A N=4, \\
N C=3, \angle M O N=90^{\circ} .
\end{array}
$$

Then $\cos A=$
5. $\frac{3}{8}$.

Let $\overrightarrow{A B}=a, \overrightarrow{A C}=b$. Then $|a|=10,|b|=7$.
Since $O$ is the midpoint of $B C$, we have
$$
\overrightarrow{A O}=\frac{1}{2} a+\frac{1}{2} b \text {. }
$$

From the problem, we know $\overrightarrow{A M}=\frac{3}{5} a, \overrightarrow{A N}=\frac{4}{7} b$.
$$
\begin{array}{l}
\text { Therefore, } \overrightarrow{O M}=\overrightarrow{A M}-\overrightarrow{A O} \\
=\frac{3}{5} a-\left(\frac{1}{2} a+\frac{1}{2} b\right) \\
=\frac{1}{10} a-\frac{1}{2} b, \\
\overrightarrow{O N}=\overrightarrow{A N}-\overrightarrow{A O} \\
=\frac{4}{7} b-\left(\frac{1}{2} a+\frac{1}{2} b\right) \\
=-\frac{1}{2} a+\frac{1}{14} b .
\end{array}
$$

Since $\angle M O N=90^{\circ}$, we know
$$
\begin{array}{l}
\overrightarrow{O M} \cdot \overrightarrow{O N}=\left(\frac{1}{10} a-\frac{1}{2} b\right) \cdot\left(-\frac{1}{2} a+\frac{1}{14} b\right)=0 \\
\Rightarrow-\frac{1}{20}|a|^{2}+\frac{9}{35} a \cdot b-\frac{1}{28}|b|^{2}=0 \\
\Rightarrow-\frac{1}{20} \times 100+\frac{9}{35} \times 10 \times 7 \cos A-\frac{1}{28} \times 49=0 \\
\Rightarrow \cos A=\frac{3}{8} .
\end{array}
$$