title: Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Models
summary: Neural Radiance Fields and 3D Gaussian Splatting have revolutionized 3D
  reconstruction and novel-view synthesis task. However, achieving photorealistic
  rendering from extreme novel viewpoints remains challenging, as artifacts
  persist across representations. In this work, we introduce Difix3D+, a novel
  pipeline designed to enhance 3D reconstruction and novel-view synthesis through
  single-step diffusion models. At the core of our approach is Difix, a
  single-step image diffusion model trained to enhance and remove artifacts in
  rendered novel views caused by underconstrained regions of the 3D
  representation. Difix serves two critical roles in our pipeline. First, it is
  used during the reconstruction phase to clean up pseudo-training views that are
  rendered from the reconstruction and then distilled back into 3D. This greatly
  enhances underconstrained regions and improves the overall 3D representation
  quality. More importantly, Difix also acts as a neural enhancer during
  inference, effectively removing residual artifacts arising from imperfect 3D
  supervision and the limited capacity of current reconstruction models. Difix3D+
  is a general solution, a single model compatible with both NeRF and 3DGS
  representations, and it achieves an average 2times improvement in FID score
  over baselines while maintaining 3D consistency.
paper_authors._id: 67c694febdab31ec59fea175
paper_authors.name: Jay Zhangjie Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-04T08:34:53.874Z
paper_authors._id: 633aaf695df91da9cea92960
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/633aaf695df91da9cea92960/9T4y1ru5wt5iKUUqf9_Tt.png
paper_authors.fullname: Jay Wu
paper_authors.type: user
paper_authors.user: jayw
paper_authors._id: 67c694febdab31ec59fea176
paper_authors.name: Yuxuan Zhang
paper_authors._id: 67c694febdab31ec59fea177
paper_authors.name: Haithem Turki
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-04T09:13:26.878Z
paper_authors._id: 656e000253703dd78fd072a9
paper_authors.avatarUrl: /avatars/6702ba8fabe3d08884aa757f90cea333.svg
paper_authors.fullname: Haithem Turki
paper_authors.type: user
paper_authors.user: hturki
paper_authors._id: 67c694febdab31ec59fea178
paper_authors.name: Xuanchi Ren
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-04T09:13:33.467Z
paper_authors._id: 658529d61c461dfe88afe8e8
paper_authors.avatarUrl: /avatars/a22c1b07d28c2662833c462c6537d835.svg
paper_authors.fullname: Xuanchi Ren
paper_authors.type: user
paper_authors.user: xrenaa
paper_authors._id: 67c694febdab31ec59fea179
paper_authors.name: Jun Gao
paper_authors._id: 67c694febdab31ec59fea17a
paper_authors.name: Mike Zheng Shou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-04T09:27:21.825Z
paper_authors._id: 661ab3da2b14565c7acccf5c
paper_authors.avatarUrl: /avatars/fa4fc03664803e02aede4d4c3d50b393.svg
paper_authors.fullname: Mike Zheng Shou
paper_authors.type: user
paper_authors.user: AnalMom
paper_authors._id: 67c694febdab31ec59fea17b
paper_authors.name: Sanja Fidler
paper_authors._id: 67c694febdab31ec59fea17c
paper_authors.name: Zan Gojcic
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-04T09:27:34.034Z
paper_authors._id: 6366cda3361a96184dc22139
paper_authors.avatarUrl: /avatars/d8a88c84cb5f69e69dd038674a29be89.svg
paper_authors.fullname: Zan Gojcic
paper_authors.type: user
paper_authors.user: zgojcic
paper_authors._id: 67c694febdab31ec59fea17d
paper_authors.name: Huan Ling
paper_title: Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Models
paper_summary: Neural Radiance Fields and 3D Gaussian Splatting have revolutionized 3D
  reconstruction and novel-view synthesis task. However, achieving photorealistic
  rendering from extreme novel viewpoints remains challenging, as artifacts
  persist across representations. In this work, we introduce Difix3D+, a novel
  pipeline designed to enhance 3D reconstruction and novel-view synthesis through
  single-step diffusion models. At the core of our approach is Difix, a
  single-step image diffusion model trained to enhance and remove artifacts in
  rendered novel views caused by underconstrained regions of the 3D
  representation. Difix serves two critical roles in our pipeline. First, it is
  used during the reconstruction phase to clean up pseudo-training views that are
  rendered from the reconstruction and then distilled back into 3D. This greatly
  enhances underconstrained regions and improves the overall 3D representation
  quality. More importantly, Difix also acts as a neural enhancer during
  inference, effectively removing residual artifacts arising from imperfect 3D
  supervision and the limited capacity of current reconstruction models. Difix3D+
  is a general solution, a single model compatible with both NeRF and 3DGS
  representations, and it achieves an average 2times improvement in FID score
  over baselines while maintaining 3D consistency.
paper_ai_keywords: Neural Radiance Fields 3D Gaussian Splatting selective single-step diffusion models Difix image diffusion model underconstrained regions pseudo-training views 3D representation neural enhancer inference residual artifacts 3D supervision reconstruction models Difix3D+ FID score 3D consistency