Recent literature has demonstrated promising results on the training of Generative Adversarial Networks by employing a set of discriminators, as opposed to the traditional game involving one generator against a single adversary. Those methods perform single-objective optimization on some simple consolidation of the losses, e.g. an average. In this work, we revisit the multiple-discriminator approach by framing the simultaneous minimization of losses provided by different models as a multi-objective optimization problem. Specifically, we evaluate the performance of multiple gradient descent and the hypervolume maximization algorithm on a number of different datasets. Moreover, we argue that the previously proposed methods and hypervolume maximization can all be seen as variations of multiple gradient descent in which the update direction computation can be done efficiently. Our results indicate that hypervolume maximization presents a better compromise between sample quality and diversity, and computational cost than previous methods.
Clarity:
The work is a clear introduction/overview of this area of research. The reviewer enjoyed the connections to Multiple-Gradient Descent and clear distinctions/contrasts with previous approaches to weighting the outputs of multiple discriminators. All in all, the paper is quite clear in what its contributions are and how it differs from previous approaches. The details and motivations of the Hypervolume Maximization  (HVM) method (especially as it relates to and interacts with the slack method of picking the nadir point) were a bit harder to follow intuitively given the standalone information in the paper.
Originality:
Adapts a technique to approximate MGD called HVM (Miranda 2016) and applies it to multi-discriminator training in GANs. As far as the reviewer is aware, this is a novel application of HVM to this task and well motivated under the MGD interpretation of the problem.
Significance:
Unclear. This work in isolation appears to present an improvement over prior work in this sub-field, but it is not obvious that the findings in these experiments will continue to be robust in more competitive settings. For instance, the worst performing model on CIFAR10, WGAN-GP (according to the experiments run) WGAN-GP also holds near SOTA Inception scores on CIFAR10 when appropriately tuned. Without any experimental results extending beyond toy datasets like MNIST and CIFAR10 the reviewer is not confident whether fundamental issues with GAN training are being addressed or just artifacts of small scale setups. Closely related previous work (Neyshabur 2017) scaled to 128x128 resolution on a much more difficult dataset - Imagenet Dogs but the authors did not compare in this case.
Quality:
Some concerns about details of experiments (see cons list and significance section for further discussion).
Pros:
+ The work provides a clear overview of previous work on approaches using multiple discriminators.
+ The connections of this line of work to MGD and the re-interpretation of various other approaches in this framework is valuable.
+ The author provides direct comparisons to similar methods, which increases confidence in the results.
+ On the experiments run, the HVM method appears to be an improvement over the two previous approaches of softmax weighting and straightforward averaging for multiple discriminators.
Cons:
- Performance of GANs is highly dependent on both model size and compute expended for a given experiment (see Miyato 2018 for model size and training iterations and Brock 2018 for batch size). Training multiple discriminators (in this paper up to 24) significantly increases compute cost and effective model size. No baselines controlling for the effects of larger models and batch sizes are done.
- The paper lacks experiments beyond toy-ish tasks like MNIST and CIFAR10 and does not do a good job comparing to the broader established literature and contextualizing its results on certain tasks such as CIFAR10 (reporting ratios to a baseline instead of absolute values, for instance). The absolute inception score of the baseline DCGAN needs to be reported to allow for this. Is the Inception Score of the authors DCGAN implementation similar to the 6 to 6.5 reported in the literature?
- Figure 3 is slightly strange in that the x axis is time to best result result instead of just overall wallclock time. Without additional information I can not determine whether it is admissible. Do all models achieve their best FID scores at similar points in training? Why is this not just a visualization of FID score as a function of wallclock time? A method which has lower variance or continues to make progress for longer than methods which begin to diverge would be unfairly represented by the current Figure.
Additional comments:
In section 3.1 Eq 5 appears to be wrong. The loss of the discriminator is presented in a form to be minimized so exponentiating the negative loss in the softmax weighting term as presented will do the opposite of what is desired and assign lower weight to higher loss discriminators. 
In Fig 6 FID scores computed on a set of 10K samples are shown. The authors appear to draw the line for the FID score of real data at 0. But since it is being estimated with only 10K samples there will be sampling error resulting in non-zero FID score. The authors should update this figure to show the box-plot for FID scores computed on random draws of 10K real samples. I have only worked with FID on Imagenet where FID scores for random batches of 10K samples are much higher than 0. I admit there is some chance the value is extremely low on CIFAR10 to make this point irrelevant, however.