Good methods of performing anomaly detection on high-dimensional data sets are needed, since algorithms which are trained on data are only expected to perform well on data that is similar to the training data. There are theoretical results on the ability to detect if a population of data is likely to come from a known base distribution, which is known as the goodness-of-fit problem, but those results require knowing a model of the base distribution. The ability to correctly reject anomalous data hinges on the accuracy of the model of the base distribution. For high dimensional data, learning an accurate-enough model of the base distribution such that anomaly detection works reliably is very challenging, as many researchers have noted in recent years. Existing methods for the goodness-of-fit problem do not account for the fact that a model of the base distribution is learned. To address that gap, we offer a theoretically motivated approach to account for the density learning procedure. In particular, we propose training an ensemble of density models, considering data to be anomalous if the data is anomalous with respect to any member of the ensemble. We provide a theoretical justification for this approach, proving first that a test on typicality is a valid approach to the goodness-of-fit problem, and then proving that for a correctly constructed ensemble of models, the intersection of typical sets of the models lies in the interior of the typical set of the base distribution. We present our method in the context of an example on synthetic data in which the effects we consider can easily be seen.
Summary:
I machine learning, we often have training data representative of an underlying distribution, and we want to test whether additional data come from the same distribution as the training data (e.g. for outlier/anomaly detection, or model checking). One way to do this is to learn a model of the underlying distribution, and test whether the additional data fall within the typical set of the model. This paper points out that the typical set of the model may be very different from the typical set of the underlying distribution if the model is learned by maximum likelihood, in which case a test of typicality with respect to the model would be a poor test of typicality with respect to the underlying distribution. The paper shows theoretically that the intersection of the typical sets of an ensemble of models lies within the typical set of the underlying distribution, provided that (a) each model is a good enough approximation to the underlying distribution, and (b) the models are all sufficiently different from each other. Based on that, the paper argues that a better test of typicality would be to test whether the additional data fall within the intersection of the typical sets of the ensemble of models.
Pros:
The paper addresses an interesting problem in a sound and well motivated way. There is a lot of work on outlier/anomaly detection that uses the model's probability density to determine whether a dataset is out-of-distribution or not, which is known to not be a good proxy for typicality, because atypical data can have high probability density. In contrast, this paper uses a well-founded notion of typicality based on the information-theoretic definition of a typical set.
The toy example that is used to illustrate the problem is clear and illuminating, and motivates the paper well. In particular, the example clearly illustrates the issue of local minima when training models, and the mass-covering behaviour of maximum-likelihood training.
The idea of using the intersection of the typical sets of an ensemble of models is interesting and clever, and backed by strong theoretical results.
Cons:
Even though I appreciate the paper's theoretical contribution, there are no empirical results other than the motivating example. In particular, the paper proposes an idea and theory to back it up, but it doesn't really propose a practical method, and as a result it doesn't test the theory in practice.
Theorems 2 and 3 provide a solid foundation for the proposed idea, but it's not clear how they can be used in practice. Specifically:
- How can we verify that in practice the KL between the models and the underlying distribution is small enough as required by theorem 2 when we can't usually evaluate it?
- In practice, how should we construct an ensemble such that the individual models in the ensemble are different enough from each other as required by theorem 3?
- Both theorem 2 and 3 are valid "for large enough n". However, in practice we may want to check e.g. individual datapoints for typicality (in which case n=1). Are the theorems relevant for small n?
The paper is generally well written, but some statements made are either inaccurate or subjective, and I worry that they might mislead readers. Later in my review I will point out exactly which statements I'm referring to. I strongly encourage the authors to fix or moderate these statements before the paper is published.
Decision:
I believe the paper to be an important contribution, but the work is clearly incomplete. For this reason, my recommendation is weak accept, with an encouragement to the authors to continue the good work.
Inaccuracies or subjective statements that I encourage the authors to fix/moderate:
"we are still bad at reliably predicting when those models will fail"
"we are unable to detect when the models are presented with out-of-distribution data"
These statements may come across as too strong. I suggest making the statements about our current methods, rather than about the ability of the research community, and be more specific in what ways the current methods are inadequate.
"detecting out-of-distribution data [...] is formally known as the goodness-of-fit problem"
I'm not sure that detecting our-of-distribution data and goodness-of-fit are synonymous. Goodness-of-fit testing can be used in situations other than outlier detection, e.g. for testing whether a proposed model is a good fit to a dataset.
(Second bullet-point of section 1) "distributions having low KL divergence must have non-zero intersection"
To be more precise, the typical sets must have non-zero intersection, not the distributions.
"determining which of two hypotheses are more probable"
"H0 is deemed more probable"
Classical hypothesis testing does not assign a probability to a hypothesis, which would be a Bayesian approach instead. Therefore, it's technically incorrect to talk about the probability of a hypothesis in this context.
"which accepts the null-hypothesis"
"f correctly accepting H0"
Hypothesis testing doesn't accept a hypothesis, it merely decides whether to reject the null hypothesis in favour of the alternative hypothesis. Therefore, it may "fail to reject" the null hypothesis, but it never accepts it.
"the KL-divergence is equal to zero if and only if p(x) = q(x; θ) ∀x ∈ X"
The KL is equal to zero if and only if the distributions are equal, but the densities may still differ in at most a set of measure zero. Therefore, it's not a requirement that the densities match for all x for the KL to be zero.
"For example, by looking at the form of the KL-divergence, there is no direct penalty for q(x; θ) in assigning a high density to points far away from any of the ¯xi’s"
The problem that this statement is talking about is the problem of overfitting, which is the problem of the model learning the specifics of the training data rather than the underlying distribution. However, the statement preceding the above is about the problem of local minima when optimizing then parameters of a model. These two problems are distinct and shouldn't be conflated, as they are here.
"this requires direct knowledge of p(x) to evaluate the objective"
However we can evaluate the objective up to an additive constant when p(x) is known up to a multiplicative constant, which is enough to optimize it.
"as do all divergences other than the forward KL, to the best of our knowledge"
"This makes the forward KL-divergence special in that it is the only divergence which can directly be optimized for."
I don't think this is true. For example, the Maximum Mean Discrepancy is a divergence, since it's non-negative and zero if and only if the two distributions are equal, but it only involves expectations under p(x) and can be directly optimized over the parameters of q(x; \theta). Moreover, the second statement doesn't follow from the first: it's incorrect to conclude that the forward KL is the only one that can be directly optimized for, based only on one's state of knowledge.
"Variational Auto-encoders [...] map a lower-dimensional, latent random variable"
There is no fundamental reason why the latent variable of a VAE has to be low-dimensional. We may do this often in practice, but a VAE with a high-dimensional latent variable may also be used.
"Because the image of any non-surjective function necessarily has measure zero"
This is not true; the absolute-value function is not surjective but its image doesn't have measure zero in the set of real numbers. I understand what the statement is trying to say, but it's important that it's said accurately.
"autoregressive models, such as PixelCNN"
Autoregressive models can also be used to model discrete variables in which case they can't be thought of as flows. In fact, PixelCNN as first proposed is a model of discrete variables.
"all of these models rely on optimizing the forward KL-divergence in order to learn their parameters"
Not necessarily, flow-based models don't have to be optimized by minimizing the forward KL. For example, they can be trained adversarially in the same way as GANs, and in principle can be trained with other divergences or integral probability metrics. The model and the loss are (at least in principle) orthogonal choices.
"advancements in the expressivity of the models are unlikely to fix the undesired effects"
This is a subjective assessment, and is not sufficiently backed by arguments where it first appears. I understand that the arguments are presented later in section 3, so I would at least suggest that a forward reference to the argumentation in section 3 is given here.
Figure 5 gives the impression that the model samples have less variance than the ground-truth samples. Isn't that surprising given that the problem is that minimizing the forward KL leads to mass-covering behaviour? I suspect that the problem here is that there are more ground-truth samples than model samples, and the ground-truth samples saturate the scatter plot. If that's the case, I believe that figure 5 is very misleading.
"we see that the learning procedure converged"
We know however that the learning procedure hasn't really converged, instead it is stuck at a saddle point (where the model is using a single mode to cover two modes of the underlying distribution). In other words, it appears to us that the learning procedure has converged, even though it hasn't, and possibly if we wait for long enough we will see rapid improvement when the procedure escapes the saddle point. Therefore, I would at least say "we see that the learning procedure has appeared to converge".
I would expect the bottom-right entry of table 1 to be higher than 90% like the other diagonal elements, so I suspect that it might be a typo.
In eq. (7), shouldn't each log q_k be divided by n?
"in practice we find that it is much easier to find an ensemble of models such that the multi-typical set approximates the ground-truth typical set than the bounds require"
There is no empirical evidence presented in the paper in support of this statement.
"least probable density"
"least typical density"
I understand what the intended meaning of these terms is, but these terms make little sense mathematically nevertheless. I would suggest that the statement is rewritten in a more precise and direct way.
"This measure only corresponds to measuring typicality if the bijection is volume preserving"
I'm not sure that the distance from a Gaussian mean is a valid measure of typicality. In high dimensions, the region around the mean is very atypical.
Minor errors, typos, and suggestions for improvement:
The phrase "the authors in Smith et al. (2019) propose" is a bit awkward. Better say "Smith et al. (2019) propose", as Smith et al are indeed the authors.
Missing full stop in first bullet-point of section 1.
It would be good to provide more details of the experiment in section 3. Specifically:
- What training algorithm was used to maximize the likelihood? SGD or EM?
- How many training datapoints were used?
"to index the 5 experiments ran" --> run
"refer the k-th learned density" --> refer to
interestig --> interesting
Missing closing bracket in point 1 of section 4.
Capital C in "Consider" in theorem 2.
"if every model in a density of learned distributions" --> an ensemble of learned distributions
"where as the method we propose" --> whereas
"can be found in in", double "in"