THE RELATIVISTIC DISCRIMINATOR: A KEY ELEMENT MISSING FROM STANDARD GAN
In standard generative adversarial network (SGAN), the discriminator D estimates the probability that the input data is real. The generator G is trained to increase the probability that fake data is real. We argue that it should also simultaneously decrease the probability that real data is real because 1) this would account for a priori knowledge that half of the data in the mini-batch is fake, 2) this would be observed with divergence minimization, and 3) SGAN would be more similar to integral probability metric (IPM) GANs. We show that this property can be induced by using a "relativistic discriminator" which estimate the probability that the given real data is more realistic than a randomly sampled fake data. We also present a variant in which the discriminator estimate the probability that the given real data is more realistic than fake data, on average. We generalize both approaches to non-standard GAN loss functions and we refer to them respectively as Relativistic GANs (RGANs) and Relativistic average GANs (RaGANs). We show that IPM-based GANs are a subset of RGANs which use the identity function. Empirically, we observe that 1) RGANs and RaGANs are significantly more stable and generate higher quality data samples than their non-relativistic counterparts, 2) Standard RaGAN with gradient penalty generate data of better quality than WGAN-GP while only requiring a single discriminator update per generator update (reducing the time taken for reaching the state-of-the-art by 400%), and 3) RaGANs are able to generate plausible high resolutions images (256x256) from a very small sample (N=2011), while GAN and LSGAN cannot; these images are of significantly better quality than the ones generated by WGAN-GP and SGAN with spectral normalization. The code is freely available on https://github.com/AlexiaJM/RelativisticGAN.
The paper describes an interesting tweak of the standard GAN model (inspired by IPM based GANs) where both the generator and the discriminator optimize relative realness (and fakeness) of the (real, fake) image pairs. The authors give some intuition for this tweak and ran experiments with CIFAR10 and CAT datasets. Different variants of the standard GAN and the new tweak were compared under the FID metric. The experimental setup and details are provided; and the code is made publicly available. 
The results are good and their tweak seems to help in most of the cases. The paper, however, is not very well written and is not of publication quality.  All the insights given in Section 3 are wrong, incomplete and unsatisfying. For example, in Section 3.4, the authors suggest that gradient dynamics of the tweaked model (with some unrealistic and infeasible assumptions) is same as that of an IPM-GAN and contribute to stability. This is wrong. Similar dynamics (even under the unrealistic assumption), does not imply similar performance. In fact, if one is trying to move towards IPM dynamics, then one should try to tweak an IPM model directly. Section 3.2 also seems wrong from my understanding of GAN training. Section 3.3 could also be improved. In fact, any explanations based on minimizing JS divergence is incomplete without answering as to why JS divergence minimizing is the best thing to do. 
The author should have provided more comparison images to rule out the fact that the tweak is not overfitting for the FID metric. The benchmarks are also weak and more experiments need to be done (Eg, CelebA).