TFS-NeRF: Template-Free NeRF for Semantic 3D Reconstruction of Dynamic Scene
<|reference_start|>TFS-NeRF: Template-Free NeRF for Semantic 3D Reconstruction of Dynamic Scene: Despite advancements in Neural Implicit models for 3D surface reconstruction, handling dynamic environments with arbitrary rigid, non-rigid, or deformable entities remains challenging. Many template-based methods are entity-specific, focusing on humans, while generic reconstruction methods adaptable to such dynamic scenes often require additional inputs like depth or optical flow or rely on pre-trained image features for reasonable outcomes. These methods typically use latent codes to capture frame-by-frame deformations. In contrast, some template-free methods bypass these requirements and adopt traditional LBS (Linear Blend Skinning) weights for a detailed representation of deformable object motions, although they involve complex optimizations leading to lengthy training times. To this end, as a remedy, this paper introduces TFS-NeRF, a template-free 3D semantic NeRF for dynamic scenes captured from sparse or single-view RGB videos, featuring interactions among various entities and more time-efficient than other LBS-based approaches. Our framework uses an Invertible Neural Network (INN) for LBS prediction, simplifying the training process. By disentangling the motions of multiple entities and optimizing per-entity skinning weights, our method efficiently generates accurate, semantically separable geometries. Extensive experiments demonstrate that our approach produces high-quality reconstructions of both deformable and non-deformable objects in complex interactions, with improved training efficiency compared to existing methods.<|reference_end|>
@article{biswas2024tfs-nerf:,
    title={TFS-NeRF: Template-Free NeRF for Semantic 3D Reconstruction of Dynamic
  Scene},
    author={Sandika Biswas, Qianyi Wu, Biplab Banerjee, Hamid Rezatofighi},
    journal={arXiv preprint arXiv:2409.17459},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.17459},
    primaryClass={cs.CV}
}