Example 5 Find all prime numbers $p$, such that
$$
\left.p^{3}\right]\left[\left(\mathrm{C}_{p}^{1}\right)^{2}+\left(\mathrm{C}_{p}^{2}\right)^{2}+\cdots+\left(\mathrm{C}_{p}^{p-1}\right)^{2}\right]
$$
Solution: Clearly, $p \neq 2,3$.
$$
\begin{array}{l}
\text { Since } p-1 \equiv-1(\bmod p), \\
p-2 \equiv-2(\bmod p), \\
\cdots \cdots \\
k=-(p-k)(\bmod p),
\end{array}
$$

Therefore, multiplying all the equations, we get
$$
\frac{(p-1)!}{(k-1)!} \equiv \pm(p-k)!(\bmod p),
$$

which means $\mathrm{C}_{p-1}^{k-1} \equiv \pm 1(\bmod p)$.
Also, $\mathrm{C}_{p}^{k}=\frac{p}{k} \mathrm{C}_{p-1}^{k-1}$, so
$$
p^{3}\left|\sum_{k=1}^{s-1}\left(\mathrm{C}_{p}^{k}\right)^{2} \Leftrightarrow p\right| \sum_{k=1}^{p-1} \frac{\left(\mathrm{C}_{p-1}^{k-1}\right)^{2}}{k^{2}} \text {. }
$$

Since for any $k(1 \leqslant k \leqslant p-1)$, $k, 2 k$, $\cdots, k(p-1)$ form a reduced residue system modulo $p$, there exists $l_{k}\left(1 \leqslant l_{k} \leqslant p-1\right)$ such that $l_{k} k \equiv 1(\bmod p)$.
Thus, $\sum_{k=1}^{k=1} \frac{\left(\mathrm{C}_{p-1}^{k-1}\right)^{2}}{k^{2}} \equiv \sum_{k=1}^{p-1} \frac{\left(\mathrm{C}_{p-1}^{k-1}\right)^{2}}{k^{2}}\left(k l_{k}\right)^{2}$
$$
=\sum_{k=1}^{-1}\left(\mathrm{C}_{p-1}^{k-1} l_{k}\right)^{2} \equiv \sum_{k=1}^{p-1} l_{k}^{2}=\sum_{k=1}^{p-1} k^{2}(\bmod p) \text {. }
$$

And $\sum_{k=1}^{-1} k^{2}=\frac{p(p-1)(2 p-1)}{6}$, and when the prime $p \geqslant 5$,
$$
(p-1)(2 p-1) \equiv 1-p^{2} \equiv 0(\bmod 3) \text {. }
$$

Therefore, $61(p-1)(2 p-1)$.
Thus, $p \left\lvert\, \sum_{k=1}^{p-1} k^{2}=\frac{p(p-1)(2 p-1)}{6}\right.$.
Hence, all primes greater than 3 are the solutions.