Recent work has shown that deep generative models can assign higher likelihood to out-of-distribution data sets than to their training data (Nalisnick et al., 2019;Choi et al., 2019). We posit that this phenomenon is caused by a mismatch between the model's typical set and its areas of high probability density. In-distribution inputs should reside in the former but not necessarily in the latter, as previous work has presumed (Bishop, 1994). To determine whether or not inputs reside in the typical set, we propose a statistically principled, easy-to-implement test using the empirical distribution of model likelihoods. The test is model agnostic and widely applicable, only requiring that the likelihood can be computed or closely approximated. We report experiments showing that our procedure can successfully detect the out-of-distribution sets in several of the challenging cases reported by Nalisnick et al. (2019).
Thanks for the authors for your detailed reviews.
My major concerns about the proposed method are whether "the typicality set" could be faithfully applied in the small data regime. The authors point me to the interesting Figure 4, which shows that it basically achieves converged performance when $m = 50$ or smaller numbers for some problems. I think this experiment is a strong support for the proposed method. 
However, I don't agree that the M=1 Gaussian case acts as a strong support for the method. As I said, for some other wired distribution, it is difficult to interpret what the M=1 Typicality set becomes. 
The authors also clarify the difference between different baselines. 
Overall, I will increase my score to "Weak Accept".
##########################
Recent works have shown that out-of-distribution samples can have higher likelihoods than in-distribution samples for some generative models. To explain this phenomenon and to tackle the problem for OOD detection, this paper adopts "typical sets" for identifying in-distribution samples. Specifically, a "typical set" is a set of examples whose expected log likelihood approximate the model's entropy. For a Gaussian distribution, the paper finds that a single point typical set locates exactly in the \sqrt{d} radius, which is usually favored over the high-likelihood origin. Then the paper uses the "typical set" to perform OOD for a batch of examples. Empirically they demonstrate competitive performance over MNIST and natural image tasks. 
Typical set seems natural for out-of-distribution detection. An important property is that, if one draws a large number of independent samples from the distribution, it is very likely that these samples belong to the typical set (basically Theorem 2.1).  However,  for small n, this property doesn't hold anymore, which leaves here a questionmark whether "Typical set" can be used for OOD detection in small n regime. As the author argues, for Gaussian distribution when n=1 the typical locations are those \sqrt{d} radius points. But this doesn't justify the "Typical set". If the distribution is some non-Gaussian wired distribution, the typical locations doesn't seem to make sense at all.
Following the previous argument above, the Typical set method requires to perform OOD for a batch of examples. In contrast, the Annulus method can be directly applied to one single test example. 
Empirically, the Typically set doesn't demonstrate obvious advantages compared to the baselines. For both MNIST and natural image tasks, it seems that all methods behave similarly. For comparing such big tables, I would recommend adding a column showing the average ranks among all methods. Beyond that, standard OOD tasks usually evaluate methods using AUROC and AUPR (Hendrycks and Gimpel, 2017). Is it possible to also include such metrics ?
Theorem 2.1 is confusing. It is beneficial to define what P is, and verbally state what the theorem conveys.