  The Neyman-Pearson strategy for hypothesis testing can be employed for
goodness of fit if the alternative hypothesis $\rm H_1$ is generic enough not
to introduce a significant bias while at the same time avoiding overfitting. A
practical implementation of this idea (dubbed NPLM) has been developed in the
context of high energy physics, targeting the detection in collider data of new
physical effects not foreseen by the Standard Model. In this paper we initiate
a comparison of this methodology with other approaches to goodness of fit, and
in particular with classifier-based strategies that share strong similarities
with NPLM. NPLM emerges from our comparison as more sensitive to small
departures of the data from the expected distribution and not biased towards
detecting specific types of anomalies while being blind to others. These
features make it more suited for agnostic searches for new physics at collider
experiments. Its deployment in other contexts should be investigated.

High Energy Physics - Phenomenology;Machine Learning