Example 4 Find $n$ such that there exists a permutation $\left(a_{1}, a_{2}, \cdots, a_{n}\right)$ of $0,1,2, \cdots, n-1$ for which $a_{1}, a_{1} a_{2}, a_{1} a_{2} a_{3}, \cdots, a_{1} a_{2} \cdots a_{n}$ form a complete residue system modulo $n$.
First, experiment with smaller $n$,
when $n=1$, it obviously satisfies the condition,
when $n=2$, $(1,0)$ satisfies the condition,
when $n=3$, $(1,2,0)$ satisfies the condition,
when $n=4$, $(1,3,2,0)$ satisfies the condition,
when $n=5$, $(1,2,4,3,0)$ satisfies the condition,
when $n=6$, after testing, there is no $\left(a_{1}, a_{2}, \cdots, a_{6}\right)$ that satisfies the condition,
when $n=7$, $(1,6,2,4,5,3,0)$ satisfies the condition,
when $n=8$, after testing, there is no $\left(a_{1}, a_{2}, \cdots, a_{8}\right)$ that satisfies the condition.
From the above experiments, we conjecture that when $n$ is a prime number, it always satisfies the condition, although this may be incomplete.
Below, we first prove this conclusion.
In fact, for any prime $p$, by the Chinese Remainder Theorem, for each $k=2,3, \cdots, p$, there exists $b_{k}$ such that $b_{k} \equiv 0$ $(\bmod k-1), b_{k} \equiv k(\bmod p)$, let $a_{k}$ represent the remainder when $c_{k}=\frac{b_{k}}{k-1}$ is divided by $n$, for $k=2,3, \cdots, p$, then $b_{k} \equiv a_{k}(k-1) \equiv k(\bmod p)$.

Let $a_{1}=1$, we now prove that $a_{1}, a_{2}, \cdots, a_{p}$ are all distinct (thus $\left(a_{1}, a_{2}, \cdots, a_{p}\right)$ is a permutation of $0,1,2, \cdots, p-1$).

In fact, by $a_{p}(p-1) \equiv p \equiv 0(\bmod p)$, we have $a_{p}=0$, so $a_{1} \neq a_{p}$. Also, when $k=2,3, \cdots, p-1$, $a_{k}(k-1) \equiv k(\bmod p)$, so $p \nmid a_{k}(k-1)$, hence $p \nmid a_{k}$, which means $a_{k} \neq a_{p}$.
If there exist $k, l, 2 \leqslant k4$ and composite, $n$ does not satisfy the condition.
Therefore, the $n$ that satisfy the condition should be 1, 4, and all prime numbers.