Law of Large Numbers for whole distributions I'm aware of the Law(s) of Large Numbers, concerning the means. However, intuitively, I'd expect not just the mean, but also the observed relative frequencies (or the histogram, if we have a continuous distribution) to approach the theoretical PMF/PDF as the number of trials goes into infinity.

*

*Is my intuition wrong? Always or only for some degenerate cases (e.g. Cauchy)?

*If not, is there a special name for that law?


We can define an indicator function that gives whether an observation is inside or outside of an interval,  which will have a Bernoulli distribution, and for a sample of multiple observations we have a binomial distribution. We can then apply the CLT to show that the mean converges to the probability mass of the original PDF in the interval. The error in absolute terms converges uniformly to zero, but I don't think the error as a percentage of the expected probability does.
