Robustness of conditional GANs to noisy labels
We study the problem of learning conditional generators from noisy labeled samples, where the labels are corrupted by random noise. A standard training of conditional GANs will not only produce samples with wrong labels, but also generate poor quality samples. We consider two scenarios, depending on whether the noise model is known or not. When the distribution of the noise is known, we introduce a novel architecture which we call Robust Conditional GAN (RCGAN). The main idea is to corrupt the label of the generated sample before feeding to the adversarial discriminator, forcing the generator to produce samples with clean labels. This approach of passing through a matching noisy channel is justified by accompanying multiplicative approximation bounds between the loss of the RCGAN and the distance between the clean real distribution and the generator distribution. This shows that the proposed approach is robust, when used with a carefully chosen discriminator architecture, known as projection discriminator. When the distribution of the noise is not known, we provide an extension of our architecture, which we call RCGAN-U, that learns the noise model simultaneously while training the generator. We show experimentally on MNIST and CIFAR-10 datasets that both the approaches consistently improve upon baseline approaches, and RCGAN-U closely matches the performance of RCGAN.
Conditional generative adversarial network (GAN) learn conditional distribution from a joint probability distribution but corrupted labels hinder this learning process. The paper highlights a unique algorithm to learn this conditional distribution with corrupted labels. The paper introduces two architectures i) RCGAN which relies on availability of matrix C which contains information regarding the errors, and ii) RCGAN-U which does not contain any information about the matrix C.   The paper claims that there is no significant loss in performance with regards to knowledge about the matrix C. Even though the problem is unique in nature the paper contains details of some of the related work and references to techniques utilized in the paper such as projection discriminator. I believe the in-depth analysis of the assumptions with theorems and proofs solidify the claims made in the paper although the math requires a more careful check. The paper provides experimentation results on two separate datasets    As far as I understand the permutation regularizer penalizes the model if generator produces samples from a class different from true one and mixing regularizer penalizes if it produces a mix of classes. The paper states that in practice we require a very weak classifier, an explanation of the intuition behind this or theoretical analysis of this statement should help concrete this claim. The paper utilizes recently proposed neural network distance which only guarantees some weak notions of generalization. Ambient GAN as stated in the related work section most closely related to the paper but the paper lacks a comparison with it under the experiments section.  To summarize the paper highlights a unique problem. It would be interesting to see its application to fix corrupted labels in the training data.