Recent work by Brock et al. (2018) suggests that Generative Adversarial Networks (GANs) benefit disproportionately from large mini-batch sizes. Unfortunately, using large batches is slow and expensive on conventional hardware. Thus, it would be nice if we could generate batches that were effectively large though actually small. In this work, we propose a method to do this, inspired by the use of Coreset-selection in active learning. When training a GAN, we draw a large batch of samples from the prior and then compress that batch using Coreset-selection. To create effectively large batches of 'real' images, we create a cached dataset of Inception activations of each training image, randomly project them down to a smaller dimension, and then use Coreset-selection on those projected activations at training time. We conduct experiments showing that this technique substantially reduces training time and memory usage for modern GAN variants, that it reduces the fraction of dropped modes in a synthetic dataset, and that it allows GANs to reach a new state of the art in anomaly detection.
Training with large batches provides a disproportionate improvement for GANs (e.g. FID drops from 18.65 to 12.39 by simply increasing the batch size by a factor of 8 for BigGAN). The authors point out that not everybody has access to the computing power which is required to run large batches. Therefore, this paper proposes a method to select the image of the batch and thereby obtaining the benefits of large batches while running only small batches.
The authors perform coreset selection of a large set of images (actually a greedy variant from Sener & Savarese). The selected images are then used for training with small batch size. The coreset selection is applied to the inception activations of the images (or a randomly downsampled version of them). Experiments show that the trained GANs obtain better mode coverage, improve anomaly detection,  and obtain GANs with higher FID scores on image synthesis. 
The paper is well motivated and solutions that prevent having to use large batches will have a significant impact on the field. 
Conclusion: At the moment I recommend a weak reject. The technical contribution of the paper is rather small, and also the depth of analysis could be improved. Moreover, I think experimental results could have been more complete. However, given the importance of the problem addressed in the paper (and the little existing work) I could be open to increasing my score if my concerns are addressed. 
Main points:
1. GANs aim to generate high-quality images. In addition, they aim to generate these high-quality images according to the distribution of the train dataset. The main danger of not randomly sampling from the train distribution is that the trained GAN does no longer generate images according to the train distribution. I think this issue should be more clearly discussed and evaluated in the paper (for some applications this might not be a big problem for others yes). For example, if you would introduce mixing coefficients for the Gaussian mixture and Gaussians with different variances (section 4.2), it could be possible that the GAN would generate according to these mixing coefficients with more accuracy than small-GAN (because the coreset selection would introduce a bias). The proposed metrics do not measure this. 
Maybe this could simply be solved by adding some final epochs which just sample randomly again? 
2. A naive approach to creating large batches is by only updating the network every N batches, and summing the gradients of the N batches. I would like to see this option discussed and compared to. I can see how the BN is different from a large batch but it would still be interesting to see. 
3. I think the examination of sampling factors should be explained in more detail. The importance of the sampling factor of the prior is harder to understand (and might be necessary to counter the effect I discuss in 1?) Is it not expected that both these sampling factors should be equal? 
Minor points (do not need to be addressed in rebuttal):
-Since FID several other GAN evaluation metrics have been proposed. I think the authors could also consider ‘Assessing Generative Models via Precision and Recall’ or ‘Improved precision and recall metric for assessing generative models’ for a more complete insight insight 
-I am not convinced this GAN needs a name (small-GAN), especially since it can be applied to other GAN architectures. 
-The claim in the abstract for state of the art in anomaly detection should be removed (no extensive study nor comparison is performed)
-I prefer to see the venue of publication when possible (for example Sener & Savarese is ICLR 2018)