title: 3DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with
    View-consistent 2D Diffusion Priors
summary: Novel-view synthesis aims to generate novel views of a scene from multiple
  input images or videos, and recent advancements like 3D Gaussian splatting
  (3DGS) have achieved notable success in producing photorealistic renderings
  with efficient pipelines. However, generating high-quality novel views under
  challenging settings, such as sparse input views, remains difficult due to
  insufficient information in under-sampled areas, often resulting in noticeable
  artifacts. This paper presents 3DGS-Enhancer, a novel pipeline for enhancing
  the representation quality of 3DGS representations. We leverage 2D video
  diffusion priors to address the challenging 3D view consistency problem,
  reformulating it as achieving temporal consistency within a video generation
  process. 3DGS-Enhancer restores view-consistent latent features of rendered
  novel views and integrates them with the input views through a spatial-temporal
  decoder. The enhanced views are then used to fine-tune the initial 3DGS model,
  significantly improving its rendering performance. Extensive experiments on
  large-scale datasets of unbounded scenes demonstrate that 3DGS-Enhancer yields
  superior reconstruction performance and high-fidelity rendering results
  compared to state-of-the-art methods. The project webpage is
  https://xiliu8006.github.io/3DGS-Enhancer-project .
paper_authors._id: 67181fa03ad5d8e375bbbb21
paper_authors.name: Xi Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-23T07:33:17.542Z
paper_authors._id: 648b15305461ecec939fb9c9
paper_authors.avatarUrl: /avatars/c099e3847cc82d0b3230435feacb0319.svg
paper_authors.fullname: Liu
paper_authors.type: user
paper_authors.user: Xi8006
paper_authors._id: 67181fa03ad5d8e375bbbb22
paper_authors.name: Chaoyi Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-24T08:39:11.650Z
paper_authors._id: 65495d1008775ce78e43e77d
paper_authors.avatarUrl: /avatars/172678e8187acfec0aa8e647479cbb81.svg
paper_authors.fullname: Chaoyi Zhou
paper_authors.type: user
paper_authors.user: chaoyizh
paper_authors._id: 67181fa03ad5d8e375bbbb23
paper_authors.name: Siyu Huang
paper_title: 3DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with
    View-consistent 2D Diffusion Priors
paper_summary: Novel-view synthesis aims to generate novel views of a scene from multiple
  input images or videos, and recent advancements like 3D Gaussian splatting
  (3DGS) have achieved notable success in producing photorealistic renderings
  with efficient pipelines. However, generating high-quality novel views under
  challenging settings, such as sparse input views, remains difficult due to
  insufficient information in under-sampled areas, often resulting in noticeable
  artifacts. This paper presents 3DGS-Enhancer, a novel pipeline for enhancing
  the representation quality of 3DGS representations. We leverage 2D video
  diffusion priors to address the challenging 3D view consistency problem,
  reformulating it as achieving temporal consistency within a video generation
  process. 3DGS-Enhancer restores view-consistent latent features of rendered
  novel views and integrates them with the input views through a spatial-temporal
  decoder. The enhanced views are then used to fine-tune the initial 3DGS model,
  significantly improving its rendering performance. Extensive experiments on
  large-scale datasets of unbounded scenes demonstrate that 3DGS-Enhancer yields
  superior reconstruction performance and high-fidelity rendering results
  compared to state-of-the-art methods. The project webpage is
  https://xiliu8006.github.io/3DGS-Enhancer-project .
paper_ai_keywords: 3D Gaussian splatting (3DGS) novel-view synthesis photorealistic renderings efficient pipelines sparse input views under-sampled areas noticeable artifacts 3DGS-Enhancer L2D video diffusion priors temporal consistency view-consistent latent features spatial-temporal decoder rendering performance large-scale datasets unbounded scenes reconstruction performance high-fidelity rendering results