title: What You See is What You GAN: Rendering Every Pixel for High-Fidelity
    Geometry in 3D GANs
summary: 3D-aware Generative Adversarial Networks (GANs) have shown remarkable
  progress in learning to generate multi-view-consistent images and 3D geometries
  of scenes from collections of 2D images via neural volume rendering. Yet, the
  significant memory and computational costs of dense sampling in volume
  rendering have forced 3D GANs to adopt patch-based training or employ
  low-resolution rendering with post-processing 2D super resolution, which
  sacrifices multiview consistency and the quality of resolved geometry.
  Consequently, 3D GANs have not yet been able to fully resolve the rich 3D
  geometry present in 2D images. In this work, we propose techniques to scale
  neural volume rendering to the much higher resolution of native 2D images,
  thereby resolving fine-grained 3D geometry with unprecedented detail. Our
  approach employs learning-based samplers for accelerating neural rendering for
  3D GAN training using up to 5 times fewer depth samples. This enables us to
  explicitly "render every pixel" of the full-resolution image during training
  and inference without post-processing superresolution in 2D. Together with our
  strategy to learn high-quality surface geometry, our method synthesizes
  high-resolution 3D geometry and strictly view-consistent images while
  maintaining image quality on par with baselines relying on post-processing
  super resolution. We demonstrate state-of-the-art 3D gemetric quality on FFHQ
  and AFHQ, setting a new standard for unsupervised learning of 3D shapes in 3D
  GANs.
paper_authors._id: 6597784d04f13fb853d3fd35
paper_authors.name: Alex Trevithick
paper_authors._id: 6597784d04f13fb853d3fd36
paper_authors.name: Matthew Chan
paper_authors._id: 6597784d04f13fb853d3fd37
paper_authors.name: Towaki Takikawa
paper_authors._id: 6597784d04f13fb853d3fd38
paper_authors.name: Umar Iqbal
paper_authors._id: 6597784d04f13fb853d3fd39
paper_authors.name: Shalini De Mello
paper_authors._id: 6597784d04f13fb853d3fd3a
paper_authors.name: Manmohan Chandraker
paper_authors._id: 6597784d04f13fb853d3fd3b
paper_authors.name: Ravi Ramamoorthi
paper_authors._id: 6597784d04f13fb853d3fd3c
paper_authors.name: Koki Nagano
paper_title: What You See is What You GAN: Rendering Every Pixel for High-Fidelity
    Geometry in 3D GANs
paper_summary: 3D-aware Generative Adversarial Networks (GANs) have shown remarkable
  progress in learning to generate multi-view-consistent images and 3D geometries
  of scenes from collections of 2D images via neural volume rendering. Yet, the
  significant memory and computational costs of dense sampling in volume
  rendering have forced 3D GANs to adopt patch-based training or employ
  low-resolution rendering with post-processing 2D super resolution, which
  sacrifices multiview consistency and the quality of resolved geometry.
  Consequently, 3D GANs have not yet been able to fully resolve the rich 3D
  geometry present in 2D images. In this work, we propose techniques to scale
  neural volume rendering to the much higher resolution of native 2D images,
  thereby resolving fine-grained 3D geometry with unprecedented detail. Our
  approach employs learning-based samplers for accelerating neural rendering for
  3D GAN training using up to 5 times fewer depth samples. This enables us to
  explicitly "render every pixel" of the full-resolution image during training
  and inference without post-processing superresolution in 2D. Together with our
  strategy to learn high-quality surface geometry, our method synthesizes
  high-resolution 3D geometry and strictly view-consistent images while
  maintaining image quality on par with baselines relying on post-processing
  super resolution. We demonstrate state-of-the-art 3D gemetric quality on FFHQ
  and AFHQ, setting a new standard for unsupervised learning of 3D shapes in 3D
  GANs.
paper_ai_summary: The paper proposes techniques to enhance neural volume rendering in 3D GANs for high-resolution 3D geometry generation and multiview consistency without post-processing.
paper_ai_keywords: 3D-aware Generative Adversarial Networks (GANs) multi-view-consistent images 3D geometries neural volume rendering patch-based training low-resolution rendering post-processing 2D super resolution learning-based samplers depth samples full-resolution image high-quality surface geometry FFHQ AFHQ unsupervised learning of 3D shapes