PA-GAN: Improving GAN Training by Progressive Augmentation
Despite recent progress, Generative Adversarial Networks (GANs) still suffer from training instability, requiring careful consideration of architecture design choices and hyper-parameter tuning. The reason for this fragile training behaviour is partially due to the discriminator performing well very quickly; its loss converges to zero, providing no reliable backpropagation signal to the generator. In this work we introduce a new technique - progressive augmentation of GANs (PA-GAN) - that helps to overcome this fundamental limitation and improve the overall stability of GAN training. The key idea is to gradually increase the task difficulty of the discriminator by progressively augmenting its input space, thus enabling continuous learning of the generator. We show that the proposed progressive augmentation preserves the original GAN objective, does not bias the optimality of the discriminator and encourages the healthy competition between the generator and discriminator, leading to a better-performing generator. We experimentally demonstrate the effectiveness of the proposed approach on multiple benchmarks (MNIST, Fashion-MNIST, CIFAR10, CELEBA) for the image generation task.
 This paper modifies the GAN objective by defining the TRUE and FAKE labels in terms of both the training sample, and a newly introduced random variable s. The intuition is that by progressively changing the definition of s, and its effect on the label, we can prevent the discriminator network from immediately learning to separate the two classes. The paper doesn*t give any strong theoretical support for this intuition. And it I found it a bit surprising that the discriminator doesn*t immediately learn the one extra bit of information introduced by every new level of augmentation. However, the results do seem to show that this augmentation has a beneficial effect on two different architectures in different data scenarios, although the increase is not uniform over all settings. The approach presented in this paper is motivated primarily as a method of increasing stability of training but this is not directly investigated. Figure 3 and Table 2 both suggest that the augmentation does nothing to reduce variance between runs. There is also no direct comparison to other methods of weakening the discriminator, although these are mentioned in the related work. I think the paper would be much improved by a thorough investigation of the method*s effect on training stability, to go along with the current set of evaluations.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
PA-GAN: Improving GAN Training by Progressive Augmentation
Despite recent progress, Generative Adversarial Networks (GANs) still suffer from training instability, requiring careful consideration of architecture design choices and hyper-parameter tuning. The reason for this fragile training behaviour is partially due to the discriminator performing well very quickly; its loss converges to zero, providing no reliable backpropagation signal to the generator. In this work we introduce a new technique - progressive augmentation of GANs (PA-GAN) - that helps to overcome this fundamental limitation and improve the overall stability of GAN training. The key idea is to gradually increase the task difficulty of the discriminator by progressively augmenting its input space, thus enabling continuous learning of the generator. We show that the proposed progressive augmentation preserves the original GAN objective, does not bias the optimality of the discriminator and encourages the healthy competition between the generator and discriminator, leading to a better-performing generator. We experimentally demonstrate the effectiveness of the proposed approach on multiple benchmarks (MNIST, Fashion-MNIST, CIFAR10, CELEBA) for the image generation task.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 This paper modifies the GAN objective by defining the TRUE and FAKE labels in terms of both the training sample, and a newly introduced random variable s. The intuition is that by progressively changing the definition of s, and its effect on the label, we can prevent the discriminator network from immediately learning to separate the two classes. The paper doesn*t give any strong theoretical support for this intuition. And it I found it a bit surprising that the discriminator doesn*t immediately learn the one extra bit of information introduced by every new level of augmentation. However, the results do seem to show that this augmentation has a beneficial effect on two different architectures in different data scenarios, although the increase is not uniform over all settings. The approach presented in this paper is motivated primarily as a method of increasing stability of training but this is not directly investigated. Figure 3 and Table 2 both suggest that the augmentation does nothing to reduce variance between runs. There is also no direct comparison to other methods of weakening the discriminator, although these are mentioned in the related work. I think the paper would be much improved by a thorough investigation of the method*s effect on training stability, to go along with the current set of evaluations.
assistant
 This paper modifies the GAN objective by defining the TRUE and FAKE labels in terms of both the training sample, and a newly introduced random variable s. The intuition is that by progressively changing the definition of s, and its effect on the label, we can prevent the discriminator network from immediately learning to separate the two classes. The paper doesn*t give any strong theoretical support for this intuition. And it I found it a bit surprising that the discriminator doesn*t immediately learn the one extra bit of information introduced by every new level of augmentation. However, the results do seem to show that this augmentation has a beneficial effect on two different architectures in different data scenarios, although the increase is not uniform over all settings. The approach presented in this paper is motivated primarily as a method of increasing stability of training but this is not directly investigated. Figure 3 and Table 2 both suggest that the augmentation does nothing to reduce variance between runs. There is also no direct comparison to other methods of weakening the discriminator, although these are mentioned in the related work. I think the paper would be much improved by a thorough investigation of the method*s effect on training stability, to go along with the current set of evaluations.