title: ResFields: Residual Neural Fields for Spatiotemporal Signals
summary: Neural fields, a category of neural networks trained to represent
  high-frequency signals, have gained significant attention in recent years due
  to their impressive performance in modeling complex 3D data, especially large
  neural signed distance (SDFs) or radiance fields (NeRFs) via a single
  multi-layer perceptron (MLP). However, despite the power and simplicity of
  representing signals with an MLP, these methods still face challenges when
  modeling large and complex temporal signals due to the limited capacity of
  MLPs. In this paper, we propose an effective approach to address this
  limitation by incorporating temporal residual layers into neural fields, dubbed
  ResFields, a novel class of networks specifically designed to effectively
  represent complex temporal signals. We conduct a comprehensive analysis of the
  properties of ResFields and propose a matrix factorization technique to reduce
  the number of trainable parameters and enhance generalization capabilities.
  Importantly, our formulation seamlessly integrates with existing techniques and
  consistently improves results across various challenging tasks: 2D video
  approximation, dynamic shape modeling via temporal SDFs, and dynamic NeRF
  reconstruction. Lastly, we demonstrate the practical utility of ResFields by
  showcasing its effectiveness in capturing dynamic 3D scenes from sparse sensory
  inputs of a lightweight capture system.
paper_authors._id: 64f949728a234f114e4731ec
paper_authors.name: Marko Mihajlovic
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-12T13:31:01.950Z
paper_authors._id: 62f7eddeb9fda55613cb9f03
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1660415445288-noauth.jpeg
paper_authors.fullname: Marko Mihajlovic
paper_authors.type: user
paper_authors.user: markomih
paper_authors._id: 64f949728a234f114e4731ed
paper_authors.name: Sergey Prokudin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-12T13:31:15.323Z
paper_authors._id: 6340364b733f9eef469165ef
paper_authors.avatarUrl: /avatars/c86d79f1588bb625b97dcbe471e46def.svg
paper_authors.fullname: Sergey Prokudin
paper_authors.type: user
paper_authors.user: sprokudin
paper_authors._id: 64f949728a234f114e4731ee
paper_authors.name: Marc Pollefeys
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-25T09:43:01.582Z
paper_authors._id: 67b5fa179782a5e2fd2cb26a
paper_authors.avatarUrl: /avatars/62c38f29ec641e001eeddf840bea21a0.svg
paper_authors.fullname: Marc Pollefeys
paper_authors.type: user
paper_authors.user: mapo1
paper_authors._id: 64f949728a234f114e4731ef
paper_authors.name: Siyu Tang
paper_title: ResFields: Residual Neural Fields for Spatiotemporal Signals
paper_summary: Neural fields, a category of neural networks trained to represent
  high-frequency signals, have gained significant attention in recent years due
  to their impressive performance in modeling complex 3D data, especially large
  neural signed distance (SDFs) or radiance fields (NeRFs) via a single
  multi-layer perceptron (MLP). However, despite the power and simplicity of
  representing signals with an MLP, these methods still face challenges when
  modeling large and complex temporal signals due to the limited capacity of
  MLPs. In this paper, we propose an effective approach to address this
  limitation by incorporating temporal residual layers into neural fields, dubbed
  ResFields, a novel class of networks specifically designed to effectively
  represent complex temporal signals. We conduct a comprehensive analysis of the
  properties of ResFields and propose a matrix factorization technique to reduce
  the number of trainable parameters and enhance generalization capabilities.
  Importantly, our formulation seamlessly integrates with existing techniques and
  consistently improves results across various challenging tasks: 2D video
  approximation, dynamic shape modeling via temporal SDFs, and dynamic NeRF
  reconstruction. Lastly, we demonstrate the practical utility of ResFields by
  showcasing its effectiveness in capturing dynamic 3D scenes from sparse sensory
  inputs of a lightweight capture system.
paper_ai_summary: ResFields, a neural network variant with temporal residual layers, enhance the representation of complex temporal signals in neural fields, improving performance across 2D video approximation, dynamic shape modeling, and NeRF reconstruction.
paper_ai_keywords: neural fields neural signed distance radiance fields multi-layer perceptron MLP temporal residual layers ResFields matrix factorization generalization capabilities 2D video approximation dynamic shape modeling temporal SDFs dynamic NeRF reconstruction 3D scenes sparse sensory inputs