title: Mesh2NeRF: Direct Mesh Supervision for Neural Radiance Field
    Representation and Generation
summary: We present Mesh2NeRF, an approach to derive ground-truth radiance fields from
  textured meshes for 3D generation tasks. Many 3D generative approaches
  represent 3D scenes as radiance fields for training. Their ground-truth
  radiance fields are usually fitted from multi-view renderings from a
  large-scale synthetic 3D dataset, which often results in artifacts due to
  occlusions or under-fitting issues. In Mesh2NeRF, we propose an analytic
  solution to directly obtain ground-truth radiance fields from 3D meshes,
  characterizing the density field with an occupancy function featuring a defined
  surface thickness, and determining view-dependent color through a reflection
  function considering both the mesh and environment lighting. Mesh2NeRF extracts
  accurate radiance fields which provides direct supervision for training
  generative NeRFs and single scene representation. We validate the effectiveness
  of Mesh2NeRF across various tasks, achieving a noteworthy 3.12dB improvement in
  PSNR for view synthesis in single scene representation on the ABO dataset, a
  0.69 PSNR enhancement in the single-view conditional generation of ShapeNet
  Cars, and notably improved mesh extraction from NeRF in the unconditional
  generation of Objaverse Mugs.
paper_authors._id: 66063cf957778f7a192ec4bf
paper_authors.name: Yujin Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-23T13:07:18.787Z
paper_authors._id: 63c307926e6561b339ddebb2
paper_authors.avatarUrl: /avatars/32548c8495494ab7682d5983fbca315e.svg
paper_authors.fullname: Yujin Chen
paper_authors.type: user
paper_authors.user: YujinChen
paper_authors._id: 66063cf957778f7a192ec4c0
paper_authors.name: Yinyu Nie
paper_authors._id: 66063cf957778f7a192ec4c1
paper_authors.name: Benjamin Ummenhofer
paper_authors._id: 66063cf957778f7a192ec4c2
paper_authors.name: Reiner Birkl
paper_authors._id: 66063cf957778f7a192ec4c3
paper_authors.name: Michael Paulitsch
paper_authors._id: 66063cf957778f7a192ec4c4
paper_authors.name: Matthias Müller
paper_authors._id: 66063cf957778f7a192ec4c5
paper_authors.name: Matthias Nießner
paper_title: Mesh2NeRF: Direct Mesh Supervision for Neural Radiance Field
    Representation and Generation
paper_summary: We present Mesh2NeRF, an approach to derive ground-truth radiance fields from
  textured meshes for 3D generation tasks. Many 3D generative approaches
  represent 3D scenes as radiance fields for training. Their ground-truth
  radiance fields are usually fitted from multi-view renderings from a
  large-scale synthetic 3D dataset, which often results in artifacts due to
  occlusions or under-fitting issues. In Mesh2NeRF, we propose an analytic
  solution to directly obtain ground-truth radiance fields from 3D meshes,
  characterizing the density field with an occupancy function featuring a defined
  surface thickness, and determining view-dependent color through a reflection
  function considering both the mesh and environment lighting. Mesh2NeRF extracts
  accurate radiance fields which provides direct supervision for training
  generative NeRFs and single scene representation. We validate the effectiveness
  of Mesh2NeRF across various tasks, achieving a noteworthy 3.12dB improvement in
  PSNR for view synthesis in single scene representation on the ABO dataset, a
  0.69 PSNR enhancement in the single-view conditional generation of ShapeNet
  Cars, and notably improved mesh extraction from NeRF in the unconditional
  generation of Objaverse Mugs.
paper_ai_keywords: Mesh2NeRF radiance fields 3D generative approaches synthetic 3D datasets artifacts occlusions under-fitting occupancy function surface thickness reflection function environment lighting direct supervision generative NeRFs single scene representation ABO dataset PSNR view synthesis ShapeNet Cars mesh extraction Objaverse Mugs unconditional generation