Animal Avatars: Reconstructing Animatable 3D Animals from Casual Videos
<|reference_start|>Animal Avatars: Reconstructing Animatable 3D Animals from Casual Videos: We present a method to build animatable dog avatars from monocular videos. This is challenging as animals display a range of (unpredictable) non-rigid movements and have a variety of appearance details (e.g., fur, spots, tails). We develop an approach that links the video frames via a 4D solution that jointly solves for animal's pose variation, and its appearance (in a canonical pose). To this end, we significantly improve the quality of template-based shape fitting by endowing the SMAL parametric model with Continuous Surface Embeddings, which brings image-to-mesh reprojection constaints that are denser, and thus stronger, than the previously used sparse semantic keypoint correspondences. To model appearance, we propose an implicit duplex-mesh texture that is defined in the canonical pose, but can be deformed using SMAL pose coefficients and later rendered to enforce a photometric compatibility with the input video frames. On the challenging CoP3D and APTv2 datasets, we demonstrate superior results (both in terms of pose estimates and predicted appearance) to existing template-free (RAC) and template-based approaches (BARC, BITE).<|reference_end|>
@article{sabathier2024animal,
    title={Animal Avatars: Reconstructing Animatable 3D Animals from Casual Videos},
    author={Remy Sabathier, Niloy J. Mitra, David Novotny},
    journal={arXiv preprint arXiv:2403.17103},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.17103},
    primaryClass={cs.CV}
}