Similarity between closed-form and sampling-based distribution Are there any well-known techniques that are capable of estimating the similarity between a closed-form distribution (specifically a Gaussian if that helps) and a distribution obtained by sampling? I'm not sure what the metrics are typically used for similarity estimation but one that makes sense intuitively would be to just find some measure of overlap between the two distributions.
Would I be better off using a smooth approximation of the discrete sampling-based distribution (not sure if this is something that is straightforward to do in the multi-dimensional case) available or are there methods to work directly with such a discrete distribution?

You can follow the idea of statistical tests for a special distributional shape and base the distance measure on the distance between the empirical cumulative distribution function $F_{emp}$ and the theoretical distribution function $F$. Then you can use the supremum distance
$$D_{sup}=\sup_{x}|F_{emp}(x) - F(x)|$$
or the $L_p$ distance
$$D_p=\left(\int_{-\infty}^{\infty} |F_{emp}(x) - F(x)|^p dx\right)^{1/p}$$
As $F_{emp}$ is a step function, both the supremum and the intergral can be computed from the $n$ sample values $x_1,\ldots,x_n$.
The supremum distance is used in the Kolmogorov-Smirnov Test as a test statistic, and the (square of the) $L_2$ distance by the Crame-van-Mieses test, which means that even their (approximate) distributions are known if the data are indeed generated by the comparison distribution. This allows for computing p-values for $P(D_{sup}>d)$ or $P(D_2^2>d)$, where $d$ is the measured value. One minus the p-value can also be considered a distance measure, although I am not sure whether it is zero for an exact match (if not, it would not qualify as a distance measure).
