title: Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering
summary: Neural rendering methods have significantly advanced photo-realistic 3D scene
  rendering in various academic and industrial applications. The recent 3D
  Gaussian Splatting method has achieved the state-of-the-art rendering quality
  and speed combining the benefits of both primitive-based representations and
  volumetric representations. However, it often leads to heavily redundant
  Gaussians that try to fit every training view, neglecting the underlying scene
  geometry. Consequently, the resulting model becomes less robust to significant
  view changes, texture-less area and lighting effects. We introduce Scaffold-GS,
  which uses anchor points to distribute local 3D Gaussians, and predicts their
  attributes on-the-fly based on viewing direction and distance within the view
  frustum. Anchor growing and pruning strategies are developed based on the
  importance of neural Gaussians to reliably improve the scene coverage. We show
  that our method effectively reduces redundant Gaussians while delivering
  high-quality rendering. We also demonstrates an enhanced capability to
  accommodate scenes with varying levels-of-detail and view-dependent
  observations, without sacrificing the rendering speed.
paper_authors._id: 656d40a08a37acfa3f1f051a
paper_authors.name: Tao Lu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-05T08:40:36.885Z
paper_authors._id: 656e9db559afca5707c493c5
paper_authors.avatarUrl: /avatars/ffe72a82aaf42c80eb988344415672ba.svg
paper_authors.fullname: Tao Lu
paper_authors.type: user
paper_authors.user: Isaaclt
paper_authors._id: 656d40a08a37acfa3f1f051b
paper_authors.name: Mulin Yu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-05T08:40:33.708Z
paper_authors._id: 656e9e26bbec423d88b603e8
paper_authors.avatarUrl: /avatars/10d8cb945a60e0401bfa4f74137cb203.svg
paper_authors.fullname: MulinYu
paper_authors.type: user
paper_authors.user: UML
paper_authors._id: 656d40a08a37acfa3f1f051c
paper_authors.name: Linning Xu
paper_authors._id: 656d40a08a37acfa3f1f051d
paper_authors.name: Yuanbo Xiangli
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-04T11:04:13.845Z
paper_authors._id: 651edd0faa888068acb5d680
paper_authors.avatarUrl: /avatars/d5e03e8bc1d22bc67a3e684b52dc5598.svg
paper_authors.fullname: Yuanbo Xiangli
paper_authors.type: user
paper_authors.user: amberkk
paper_authors._id: 656d40a08a37acfa3f1f051e
paper_authors.name: Limin Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-04T11:05:17.404Z
paper_authors._id: 62c77f4352d8ae531f5511f9
paper_authors.avatarUrl: /avatars/50198ccb02ccd286975a4613fbabee28.svg
paper_authors.fullname: Limin Wang
paper_authors.type: user
paper_authors.user: lmwang
paper_authors._id: 656d40a08a37acfa3f1f051f
paper_authors.name: Dahua Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-04T11:05:26.007Z
paper_authors._id: 636317ed80c1a705a6eff396
paper_authors.avatarUrl: /avatars/3db090e101b916d9256d0d3e043db71d.svg
paper_authors.fullname: Dahua Lin
paper_authors.type: user
paper_authors.user: lindahua
paper_authors._id: 656d40a08a37acfa3f1f0520
paper_authors.name: Bo Dai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-04T11:05:34.052Z
paper_authors._id: 635f93577c05eb9f59966209
paper_authors.avatarUrl: /avatars/add48d7e3790a6b500d6c451ef8b0f75.svg
paper_authors.fullname: Intelligent Digital Creation
paper_authors.type: user
paper_authors.user: BoDai
paper_title: Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering
paper_summary: Neural rendering methods have significantly advanced photo-realistic 3D scene
  rendering in various academic and industrial applications. The recent 3D
  Gaussian Splatting method has achieved the state-of-the-art rendering quality
  and speed combining the benefits of both primitive-based representations and
  volumetric representations. However, it often leads to heavily redundant
  Gaussians that try to fit every training view, neglecting the underlying scene
  geometry. Consequently, the resulting model becomes less robust to significant
  view changes, texture-less area and lighting effects. We introduce Scaffold-GS,
  which uses anchor points to distribute local 3D Gaussians, and predicts their
  attributes on-the-fly based on viewing direction and distance within the view
  frustum. Anchor growing and pruning strategies are developed based on the
  importance of neural Gaussians to reliably improve the scene coverage. We show
  that our method effectively reduces redundant Gaussians while delivering
  high-quality rendering. We also demonstrates an enhanced capability to
  accommodate scenes with varying levels-of-detail and view-dependent
  observations, without sacrificing the rendering speed.
paper_ai_summary: Scaffold-GS improves 3D scene rendering by reducing Gaussian redundancy and enhancing robustness to varying views and lighting using anchor points and neural Gaussian importance assessment.
paper_ai_keywords: Neural rendering 3D Gaussian Splatting primitive-based representations volumetric representations Gaussians anchor points attribute prediction view frustum anchor growing anchor pruning scene geometry rendering quality view-dependent observations levels-of-detail