Example 4 Find all odd prime numbers $p$ such that
$$
p \mid \sum_{k=1}^{2011} k^{p-1} .
$$
If $p>2011$, then for $k(1 \leqslant k \leqslant 2011)$ by Fermat's Little Theorem we have
$k^{p-1} \equiv 1(\bmod p)$.
Thus $\sum_{k=1}^{2011} k^{p-1} \equiv 2011 \not\equiv 0(\bmod p)$, a contradiction.
Therefore, $p \leqslant 2011$.
Let $2011=p q+r(0 \leqslant r < q)$, then if $r>q$, we have $q=r$, so
$$
2011=p q+r=(p+1) r \text {. }
$$

This implies $p$ is even, which contradicts $p$ being a prime.
If $p \leqslant q$, then $2011=p q+r \geqslant p^{2}$.
Thus, the odd primes satisfying $p^{2} \leqslant 2011$ are
$$
3,5,7,11,13, 17, 19,23,29,31,37,41,43 \text {. }
$$

Checking, we find that only $p=3$ satisfies the condition.
In summary, $p=3$.