title: TRIPS: Trilinear Point Splatting for Real-Time Radiance Field Rendering
summary: Point-based radiance field rendering has demonstrated impressive results for
  novel view synthesis, offering a compelling blend of rendering quality and
  computational efficiency. However, also latest approaches in this domain are
  not without their shortcomings. 3D Gaussian Splatting [Kerbl and Kopanas et al.
  2023] struggles when tasked with rendering highly detailed scenes, due to
  blurring and cloudy artifacts. On the other hand, ADOP [R\"uckert et al. 2022]
  can accommodate crisper images, but the neural reconstruction network decreases
  performance, it grapples with temporal instability and it is unable to
  effectively address large gaps in the point cloud.
    In this paper, we present TRIPS (Trilinear Point Splatting), an approach that
  combines ideas from both Gaussian Splatting and ADOP. The fundamental concept
  behind our novel technique involves rasterizing points into a screen-space
  image pyramid, with the selection of the pyramid layer determined by the
  projected point size. This approach allows rendering arbitrarily large points
  using a single trilinear write. A lightweight neural network is then used to
  reconstruct a hole-free image including detail beyond splat resolution.
  Importantly, our render pipeline is entirely differentiable, allowing for
  automatic optimization of both point sizes and positions.
    Our evaluation demonstrate that TRIPS surpasses existing state-of-the-art
  methods in terms of rendering quality while maintaining a real-time frame rate
  of 60 frames per second on readily available hardware. This performance extends
  to challenging scenarios, such as scenes featuring intricate geometry,
  expansive landscapes, and auto-exposed footage.
paper_authors._id: 65a0b0200c8d993b1719484a
paper_authors.name: Linus Franke
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-12T14:53:29.998Z
paper_authors._id: 65a1520eb240f62641434d00
paper_authors.avatarUrl: /avatars/30c737a333d823a139183ef3ebdb252a.svg
paper_authors.fullname: Linus Franke
paper_authors.type: user
paper_authors.user: linusfranke
paper_authors._id: 65a0b0200c8d993b1719484b
paper_authors.name: Darius Rückert
paper_authors._id: 65a0b0200c8d993b1719484c
paper_authors.name: Laura Fink
paper_authors._id: 65a0b0200c8d993b1719484d
paper_authors.name: Marc Stamminger
paper_title: TRIPS: Trilinear Point Splatting for Real-Time Radiance Field Rendering
paper_summary: Point-based radiance field rendering has demonstrated impressive results for
  novel view synthesis, offering a compelling blend of rendering quality and
  computational efficiency. However, also latest approaches in this domain are
  not without their shortcomings. 3D Gaussian Splatting [Kerbl and Kopanas et al.
  2023] struggles when tasked with rendering highly detailed scenes, due to
  blurring and cloudy artifacts. On the other hand, ADOP [R\"uckert et al. 2022]
  can accommodate crisper images, but the neural reconstruction network decreases
  performance, it grapples with temporal instability and it is unable to
  effectively address large gaps in the point cloud.
    In this paper, we present TRIPS (Trilinear Point Splatting), an approach that
  combines ideas from both Gaussian Splatting and ADOP. The fundamental concept
  behind our novel technique involves rasterizing points into a screen-space
  image pyramid, with the selection of the pyramid layer determined by the
  projected point size. This approach allows rendering arbitrarily large points
  using a single trilinear write. A lightweight neural network is then used to
  reconstruct a hole-free image including detail beyond splat resolution.
  Importantly, our render pipeline is entirely differentiable, allowing for
  automatic optimization of both point sizes and positions.
    Our evaluation demonstrate that TRIPS surpasses existing state-of-the-art
  methods in terms of rendering quality while maintaining a real-time frame rate
  of 60 frames per second on readily available hardware. This performance extends
  to challenging scenarios, such as scenes featuring intricate geometry,
  expansive landscapes, and auto-exposed footage.
paper_ai_summary: TRIPS, combining Gaussian Splatting and ADOP techniques, achieves high-quality rendering of detailed scenes with real-time performance using a screen-space image pyramid and a differentiable render pipeline.
paper_ai_keywords: point-based radiance field rendering novel view synthesis rendering quality computational efficiency 3D Gaussian Splatting ADOP neural reconstruction network temporal instability point cloud trilinear point splatting screen-space image pyramid trilinear write lightweight neural network differentiable render pipeline real-time frame rate