title: Self-conditioned Image Generation via Generating Representations
summary: This paper presents Representation-Conditioned image
  Generation (RCG), a simple yet effective image generation framework
  which sets a new benchmark in class-unconditional image generation. RCG does
  not condition on any human annotations. Instead, it conditions on a
  self-supervised representation distribution which is mapped from the image
  distribution using a pre-trained encoder. During generation, RCG samples from
  such representation distribution using a representation diffusion model (RDM),
  and employs a pixel generator to craft image pixels conditioned on the sampled
  representation. Such a design provides substantial guidance during the
  generative process, resulting in high-quality image generation. Tested on
  ImageNet 256times256, RCG achieves a Frechet Inception Distance (FID) of
  3.31 and an Inception Score (IS) of 253.4. These results not only significantly
  improve the state-of-the-art of class-unconditional image generation but also
  rival the current leading methods in class-conditional image generation,
  bridging the long-standing performance gap between these two tasks. Code is
  available at https://github.com/LTH14/rcg.
paper_authors._id: 657132653e68d88fd61f1fb2
paper_authors.name: Tianhong Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-07T12:18:39.736Z
paper_authors._id: 65244352e0823fbfef8c6475
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65244352e0823fbfef8c6475/d6YipDIYHdhP_B6RJACNc.jpeg
paper_authors.fullname: Tianhong Li
paper_authors.type: user
paper_authors.user: LTH14
paper_authors._id: 657132653e68d88fd61f1fb3
paper_authors.name: Dina Katabi
paper_authors._id: 657132653e68d88fd61f1fb4
paper_authors.name: Kaiming He
paper_title: Self-conditioned Image Generation via Generating Representations
paper_summary: This paper presents Representation-Conditioned image
  Generation (RCG), a simple yet effective image generation framework
  which sets a new benchmark in class-unconditional image generation. RCG does
  not condition on any human annotations. Instead, it conditions on a
  self-supervised representation distribution which is mapped from the image
  distribution using a pre-trained encoder. During generation, RCG samples from
  such representation distribution using a representation diffusion model (RDM),
  and employs a pixel generator to craft image pixels conditioned on the sampled
  representation. Such a design provides substantial guidance during the
  generative process, resulting in high-quality image generation. Tested on
  ImageNet 256times256, RCG achieves a Frechet Inception Distance (FID) of
  3.31 and an Inception Score (IS) of 253.4. These results not only significantly
  improve the state-of-the-art of class-unconditional image generation but also
  rival the current leading methods in class-conditional image generation,
  bridging the long-standing performance gap between these two tasks. Code is
  available at https://github.com/LTH14/rcg.
paper_ai_keywords: representation-conditioned image generation representation distribution pre-trained encoder representation diffusion model pixel generator generative process high-quality image generation Frechet Inception Distance Inception Score