title: Spectrally Pruned Gaussian Fields with Neural Compensation
summary: Recently, 3D Gaussian Splatting, as a novel 3D representation, has garnered
  attention for its fast rendering speed and high rendering quality. However,
  this comes with high memory consumption, e.g., a well-trained Gaussian field
  may utilize three million Gaussian primitives and over 700 MB of memory. We
  credit this high memory footprint to the lack of consideration for the
  relationship between primitives. In this paper, we propose a memory-efficient
  Gaussian field named SUNDAE with spectral pruning and neural compensation. On
  one hand, we construct a graph on the set of Gaussian primitives to model their
  relationship and design a spectral down-sampling module to prune out primitives
  while preserving desired signals. On the other hand, to compensate for the
  quality loss of pruning Gaussians, we exploit a lightweight neural network head
  to mix splatted features, which effectively compensates for quality losses
  while capturing the relationship between primitives in its weights. We
  demonstrate the performance of SUNDAE with extensive results. For example,
  SUNDAE can achieve 26.80 PSNR at 145 FPS using 104 MB memory while the vanilla
  Gaussian splatting algorithm achieves 25.60 PSNR at 160 FPS using 523 MB
  memory, on the Mip-NeRF360 dataset. Codes are publicly available at
  https://runyiyang.github.io/projects/SUNDAE/.
paper_authors._id: 6632f7041fd3ec937a476f03
paper_authors.name: Runyi Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-02T07:29:13.791Z
paper_authors._id: 646b15873e2a7b065945679b
paper_authors.avatarUrl: /avatars/d3ffef9ea73dfa7cf37e9a06c41df2dc.svg
paper_authors.fullname: Runyi Yang
paper_authors.type: user
paper_authors.user: Runyi-Y
paper_authors._id: 6632f7041fd3ec937a476f04
paper_authors.name: Zhenxin Zhu
paper_authors._id: 6632f7041fd3ec937a476f05
paper_authors.name: Zhou Jiang
paper_authors._id: 6632f7041fd3ec937a476f06
paper_authors.name: Baijun Ye
paper_authors._id: 6632f7041fd3ec937a476f07
paper_authors.name: Xiaoxue Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-02T07:29:45.894Z
paper_authors._id: 650c1106b92f35bd37dbcdfa
paper_authors.avatarUrl: /avatars/d41088e6c5cca83a567fdf9c48160eb7.svg
paper_authors.fullname: xiaoxue chen
paper_authors.type: user
paper_authors.user: xxuechen
paper_authors._id: 6632f7041fd3ec937a476f08
paper_authors.name: Yifei Zhang
paper_authors._id: 6632f7041fd3ec937a476f09
paper_authors.name: Yuantao Chen
paper_authors._id: 6632f7041fd3ec937a476f0a
paper_authors.name: Jian Zhao
paper_authors._id: 6632f7041fd3ec937a476f0b
paper_authors.name: Hao Zhao
paper_title: Spectrally Pruned Gaussian Fields with Neural Compensation
paper_summary: Recently, 3D Gaussian Splatting, as a novel 3D representation, has garnered
  attention for its fast rendering speed and high rendering quality. However,
  this comes with high memory consumption, e.g., a well-trained Gaussian field
  may utilize three million Gaussian primitives and over 700 MB of memory. We
  credit this high memory footprint to the lack of consideration for the
  relationship between primitives. In this paper, we propose a memory-efficient
  Gaussian field named SUNDAE with spectral pruning and neural compensation. On
  one hand, we construct a graph on the set of Gaussian primitives to model their
  relationship and design a spectral down-sampling module to prune out primitives
  while preserving desired signals. On the other hand, to compensate for the
  quality loss of pruning Gaussians, we exploit a lightweight neural network head
  to mix splatted features, which effectively compensates for quality losses
  while capturing the relationship between primitives in its weights. We
  demonstrate the performance of SUNDAE with extensive results. For example,
  SUNDAE can achieve 26.80 PSNR at 145 FPS using 104 MB memory while the vanilla
  Gaussian splatting algorithm achieves 25.60 PSNR at 160 FPS using 523 MB
  memory, on the Mip-NeRF360 dataset. Codes are publicly available at
  https://runyiyang.github.io/projects/SUNDAE/.
paper_ai_summary: SUNDAE, a memory-efficient 3D Gaussian field, reduces memory usage by pruning primitives using spectral down-sampling and compensates for quality loss with a neural network head, achieving comparable or better performance than standard Gaussian splatting.
paper_ai_keywords: 3D Gaussian Splatting Gaussian field memory-efficient spectral pruning neural compensation graph spectral down-sampling module PSNR FPS Mip-NeRF360 dataset