title: The GAN is dead; long live the GAN! A Modern GAN Baseline
summary: There is a widely-spread claim that GANs are difficult to train, and GAN
  architectures in the literature are littered with empirical tricks. We provide
  evidence against this claim and build a modern GAN baseline in a more
  principled manner. First, we derive a well-behaved regularized relativistic GAN
  loss that addresses issues of mode dropping and non-convergence that were
  previously tackled via a bag of ad-hoc tricks. We analyze our loss
  mathematically and prove that it admits local convergence guarantees, unlike
  most existing relativistic losses. Second, our new loss allows us to discard
  all ad-hoc tricks and replace outdated backbones used in common GANs with
  modern architectures. Using StyleGAN2 as an example, we present a roadmap of
  simplification and modernization that results in a new minimalist baseline --
  R3GAN. Despite being simple, our approach surpasses StyleGAN2 on FFHQ,
  ImageNet, CIFAR, and Stacked MNIST datasets, and compares favorably against
  state-of-the-art GANs and diffusion models.
paper_authors._id: 6780827489ff720d2e028eb8
paper_authors.name: Yiwen Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-10T08:25:22.968Z
paper_authors._id: 656988c35958c68e3180961e
paper_authors.avatarUrl: /avatars/7e7916a9c9d2502774dd7727c1a03049.svg
paper_authors.fullname: Yiwen Huang
paper_authors.type: user
paper_authors.user: Eva1209
paper_authors._id: 6780827489ff720d2e028eb9
paper_authors.name: Aaron Gokaslan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-10T08:25:29.960Z
paper_authors._id: 620573d0522e40b4a18d8763
paper_authors.avatarUrl: /avatars/9353c064ef8ccac84d0397411d38fa90.svg
paper_authors.fullname: Aaron Gokaslan
paper_authors.type: user
paper_authors.user: Skylion007
paper_authors._id: 6780827489ff720d2e028eba
paper_authors.name: Volodymyr Kuleshov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-10T08:25:37.188Z
paper_authors._id: 640ea1c43c82bd463ee80e19
paper_authors.avatarUrl: /avatars/3213d2e8a433a207dfb96a35f0f52a92.svg
paper_authors.fullname: Volodymyr Kuleshov
paper_authors.type: user
paper_authors.user: kuleshov
paper_authors._id: 6780827489ff720d2e028ebb
paper_authors.name: James Tompkin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-10T08:25:45.080Z
paper_authors._id: 6581b07b4466994ea8491941
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6581b07b4466994ea8491941/dsKpoHwnyTqCaSRMV3evn.png
paper_authors.fullname: James Tompkin
paper_authors.type: user
paper_authors.user: jamestompkin
paper_title: The GAN is dead; long live the GAN! A Modern GAN Baseline
paper_summary: There is a widely-spread claim that GANs are difficult to train, and GAN
  architectures in the literature are littered with empirical tricks. We provide
  evidence against this claim and build a modern GAN baseline in a more
  principled manner. First, we derive a well-behaved regularized relativistic GAN
  loss that addresses issues of mode dropping and non-convergence that were
  previously tackled via a bag of ad-hoc tricks. We analyze our loss
  mathematically and prove that it admits local convergence guarantees, unlike
  most existing relativistic losses. Second, our new loss allows us to discard
  all ad-hoc tricks and replace outdated backbones used in common GANs with
  modern architectures. Using StyleGAN2 as an example, we present a roadmap of
  simplification and modernization that results in a new minimalist baseline --
  R3GAN. Despite being simple, our approach surpasses StyleGAN2 on FFHQ,
  ImageNet, CIFAR, and Stacked MNIST datasets, and compares favorably against
  state-of-the-art GANs and diffusion models.
paper_ai_summary: A new regularization approach and simplified architecture for GANs, named R3GAN, surpasses existing methods including StyleGAN2 and diffusion models across multiple datasets.
paper_ai_keywords: GANs relativistic GAN loss mode dropping non-convergence StyleGAN2 R3GAN FFHQ ImageNet CIFAR Stacked MNIST