title: V^3: Viewing Volumetric Videos on Mobiles via Streamable 2D Dynamic
    Gaussians
summary: Experiencing high-fidelity volumetric video as seamlessly as 2D videos is a
  long-held dream. However, current dynamic 3DGS methods, despite their high
  rendering quality, face challenges in streaming on mobile devices due to
  computational and bandwidth constraints. In this paper, we introduce
  V3(Viewing Volumetric Videos), a novel approach that enables
  high-quality mobile rendering through the streaming of dynamic Gaussians. Our
  key innovation is to view dynamic 3DGS as 2D videos, facilitating the use of
  hardware video codecs. Additionally, we propose a two-stage training strategy
  to reduce storage requirements with rapid training speed. The first stage
  employs hash encoding and shallow MLP to learn motion, then reduces the number
  of Gaussians through pruning to meet the streaming requirements, while the
  second stage fine tunes other Gaussian attributes using residual entropy loss
  and temporal loss to improve temporal continuity. This strategy, which
  disentangles motion and appearance, maintains high rendering quality with
  compact storage requirements. Meanwhile, we designed a multi-platform player to
  decode and render 2D Gaussian videos. Extensive experiments demonstrate the
  effectiveness of V3, outperforming other methods by enabling
  high-quality rendering and streaming on common devices, which is unseen before.
  As the first to stream dynamic Gaussians on mobile devices, our companion
  player offers users an unprecedented volumetric video experience, including
  smooth scrolling and instant sharing. Our project page with source code is
  available at https://authoritywang.github.io/v3/.
paper_authors._id: 66f0dfa1507a27eac7ce7acd
paper_authors.name: Penghao Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-21T07:49:59.490Z
paper_authors._id: 6462120032083c28b408e754
paper_authors.avatarUrl: /avatars/cb6093ffedde49a4e0d854480b1c2d59.svg
paper_authors.fullname: Penghao Wang
paper_authors.type: user
paper_authors.user: AuWang
paper_authors._id: 66f0dfa1507a27eac7ce7ace
paper_authors.name: Zhirui Zhang
paper_authors._id: 66f0dfa1507a27eac7ce7acf
paper_authors.name: Liao Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-23T07:48:37.407Z
paper_authors._id: 64180cb3db24526c7c9b011e
paper_authors.avatarUrl: /avatars/bcbfe1e18a2c6402d5db14520279b04b.svg
paper_authors.fullname: wliao
paper_authors.type: user
paper_authors.user: liaowang11
paper_authors._id: 66f0dfa1507a27eac7ce7ad0
paper_authors.name: Kaixin Yao
paper_authors._id: 66f0dfa1507a27eac7ce7ad1
paper_authors.name: Siyuan Xie
paper_authors._id: 66f0dfa1507a27eac7ce7ad2
paper_authors.name: Jingyi Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-23T07:49:12.361Z
paper_authors._id: 63ffcec6e7767a89533b8f80
paper_authors.avatarUrl: /avatars/b9e8673943741c2e7b80b15a061bb951.svg
paper_authors.fullname: JINGYI Yu
paper_authors.type: user
paper_authors.user: Daluuu
paper_authors._id: 66f0dfa1507a27eac7ce7ad3
paper_authors.name: Minye Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-23T07:49:18.290Z
paper_authors._id: 656ef1678d218bc3ce4b7845
paper_authors.avatarUrl: /avatars/09e4091f486024c5fd97f77339096f8f.svg
paper_authors.fullname: Minye Wu
paper_authors.type: user
paper_authors.user: wuminye
paper_authors._id: 66f0dfa1507a27eac7ce7ad4
paper_authors.name: Lan Xu
paper_title: V^3: Viewing Volumetric Videos on Mobiles via Streamable 2D Dynamic
    Gaussians
paper_summary: Experiencing high-fidelity volumetric video as seamlessly as 2D videos is a
  long-held dream. However, current dynamic 3DGS methods, despite their high
  rendering quality, face challenges in streaming on mobile devices due to
  computational and bandwidth constraints. In this paper, we introduce
  V3(Viewing Volumetric Videos), a novel approach that enables
  high-quality mobile rendering through the streaming of dynamic Gaussians. Our
  key innovation is to view dynamic 3DGS as 2D videos, facilitating the use of
  hardware video codecs. Additionally, we propose a two-stage training strategy
  to reduce storage requirements with rapid training speed. The first stage
  employs hash encoding and shallow MLP to learn motion, then reduces the number
  of Gaussians through pruning to meet the streaming requirements, while the
  second stage fine tunes other Gaussian attributes using residual entropy loss
  and temporal loss to improve temporal continuity. This strategy, which
  disentangles motion and appearance, maintains high rendering quality with
  compact storage requirements. Meanwhile, we designed a multi-platform player to
  decode and render 2D Gaussian videos. Extensive experiments demonstrate the
  effectiveness of V3, outperforming other methods by enabling
  high-quality rendering and streaming on common devices, which is unseen before.
  As the first to stream dynamic Gaussians on mobile devices, our companion
  player offers users an unprecedented volumetric video experience, including
  smooth scrolling and instant sharing. Our project page with source code is
  available at https://authoritywang.github.io/v3/.
paper_ai_keywords: volumetric video 3DGS rendering quality computational constraints bandwidth constraints V\textsuperscript{3}(Viewing Volumetric Videos) dynamic Gaussians hardware video codecs two-stage training strategy storage requirements training speed hash encoding shallow MLP motion learning Gaussian pruning residual entropy temporal loss temporal continuity Gaussian attributes multi-platform player 2D Gaussian videos smooth scrolling instant sharing