Example 1 Given that $p$ is a prime number and $k$ is a positive integer. Try to find the remainder $S=1^{k}+2^{k}+\cdots+(p-1)^{k}(\bmod p)$.
If $(p-1) \mid k$, then when $(i, p)=1$,
$$
i^{k} \equiv 1(\bmod p) \text {. }
$$

Thus, $S \equiv p-1 \equiv-1(\bmod p)$.
If $(p-1) \nmid k$, let $a$ be a primitive root modulo $p$. Then
$$
p \nmid\left(a^{k}-1\right) \text {, }
$$

and $a i(1 \leqslant i \leqslant p-1)$ is also a reduced residue system modulo $p$.
Hence $S \equiv \sum_{i=1}^{-1}(a i)^{k}=a^{k} S(\bmod p)$,
$\left(a^{k}-1\right) S \equiv 0(\bmod p)$.
Therefore, $S \equiv 0(\bmod p)$.