Measuring goodness of fit of an arbitrary pdf on a given dataset There are $100$ data points over a random variable $X$, and we randomly choose $30$ of them to estimate the distribution over those $30$ observations. Let's say we use $2$ different models for estimation (for example, using MLE to fit a normal and an exponential distribution) and output two pdf, $f_1(x)$ and $f_2(x)$.
My question is, how do we know which model is better when "fitting" into the bigger dataset of $100$ data points?

First off, it would be better to use the 70 data points that were left over after you used 30 of the original 100 for fitting the models, because testing on the training data will tend to reward overfitting.
That said, your models yield full distributional, or probabilistic, predictions for the test data. The standard way of assessing which one of multiple such probabilistic predictions is better is to use proper scoring rules. This tag wiki contains more information and pointers to literature about these.
