Example 4 Find all pairs of positive real numbers $(a, b)$ such that for any $n \in \mathbf{N}^{*}$, we have
$$[a[b n]]=n-1$$
From (5), we know that for any $n \in \mathbf{N}^{*}$, we have
$$\begin{aligned}
n-1 & =[a[b n]] \leqslant a[b n] \leqslant a b n1$, then for the irrational number $b$, there exists $n \in \mathbf{N}^{*}$ such that $\{n b\}>\frac{1}{a}$, which contradicts (6). Therefore, $a<1$.

In summary, the positive real numbers $(a, b)$ should satisfy: $a b=1,0<a<1$, and both $a, b$ are irrational numbers.
Finally, let $(a, b)$ be a pair of positive real numbers satisfying the above conditions. Then, since $b$ is irrational, for any $n \in \mathbf{N}^{*}$, we have $0<\{b n\}<1$, thus,
$$-1<-a<-a\{b n\}<0$$

This shows that (6) holds. Therefore, $(a, b)$ satisfies (5).
Hence, the set of $(a, b)$ that satisfy the conditions is $\{(a, b) \mid a b=1,0<a<1, a, b$ are irrational numbers\}.