Law of Large Numbers for whole distributions I'm aware of the Law(s) of Large Numbers, concerning the means. However, intuitively, I'd expect not just the mean, but also the observed relative frequencies (or the histogram, if we have a continuous distribution) to approach the theoretical PMF/PDF as the number of trials goes into infinity.

*

*Is my intuition wrong? Always or only for some degenerate cases (e.g. Cauchy)?

*If not, is there a special name for that law?


As an alternative to Glivenko–Cantelli, you can look at Sanov's theorem, which uses the Kullback–Leibler divergence as the distance measure. For any set $A$ of frequency distributions, this theorem upper-bounds the probability that the observed frequency distribution $f$ (for $n$ IID instances of a random variable with distribution $q$ on an alphabet $X$) lies in $A$, as follows:
$$P[f\in A] \leq (n+1)^{\left|X\right|} 2^{-d_\mathrm{min} n},$$
where $d_\mathrm{min} = \min_{f\in A} D_\mathrm{KL}(f||q)$ is the minimum KL divergence between frequencies in $A$ and the true distribution $q$. To apply this to directly address the question asked, this result implies that if $\mathcal{B}^{D_\mathrm{KL}}_d(q)$ denotes the "ball" of frequency distributions with KL divergence $<d$ from the true distribution $q$, we can upper-bound the probability of getting a frequency distribution outside this ball as
$$P[f\notin \mathcal{B}^{D_\mathrm{KL}}_d(q)] \leq (n+1)^{\left|X\right|} 2^{-d n}.$$
This expression has the advantage of being an explicit bound for finite sample size, in case you need something more than an asymptotic convergence result.  (A weaker option is provided by the asymptotic equipartition property, but the set considered in that theorem potentially includes some number of frequency distributions that are "very far" from the true distribution.)
