DressRecon: Freeform 4D Human Reconstruction from Monocular Video
We present a method to reconstruct time-consistent human body models from
monocular videos, focusing on extremely loose clothing or handheld object
interactions. Prior work in human reconstruction is either limited to tight
clothing with no object interactions, or requires calibrated multi-view
captures or personalized template scans which are costly to collect at scale.
Our key insight for high-quality yet flexible reconstruction is the careful
combination of generic human priors about articulated body shape (learned from
large-scale training data) with video-specific articulated "bag-of-bones"
deformation (fit to a single video via test-time optimization). We accomplish
this by learning a neural implicit model that disentangles body versus clothing
deformations as separate motion model layers. To capture subtle geometry of
clothing, we leverage image-based priors such as human body pose, surface
normals, and optical flow during optimization. The resulting neural fields can
be extracted into time-consistent meshes, or further optimized as explicit 3D
Gaussians for high-fidelity interactive rendering. On datasets with highly
challenging clothing deformations and object interactions, DressRecon yields
higher-fidelity 3D reconstructions than prior art. Project page:
https://jefftan969.github.io/dressrecon/
66fcb1ed63002e1de06055e3
Jeff Tan
claimed_verified
2024-10-02T12:44:43.969Z
6616db12f60442df33b7035c
/avatars/e0737dc8cff7ddb54ebb65447da4783b.svg
Jeff Tan
user
jefftan969
66fcb1ed63002e1de06055e4
Donglai Xiang
66fcb1ed63002e1de06055e5
Shubham Tulsiani
66fcb1ed63002e1de06055e6
Deva Ramanan
admin_assigned
2024-10-02T09:13:35.973Z
6337151b0267ebcf02640eb6
/avatars/14a723cafc5587043bdfb19304fc202d.svg
Deva Ramanan
user
devakramanan
66fcb1ed63002e1de06055e7
Gengshan Yang
admin_assigned
2024-10-02T09:13:29.334Z
62c0597543d4858f821bc87a
/avatars/ce9d28c08a5c9e8312a754714bae4634.svg
Gengshan Yang
user
gengshan-y
DressRecon: Freeform 4D Human Reconstruction from Monocular Video
We present a method to reconstruct time-consistent human body models from
monocular videos, focusing on extremely loose clothing or handheld object
interactions. Prior work in human reconstruction is either limited to tight
clothing with no object interactions, or requires calibrated multi-view
captures or personalized template scans which are costly to collect at scale.
Our key insight for high-quality yet flexible reconstruction is the careful
combination of generic human priors about articulated body shape (learned from
large-scale training data) with video-specific articulated "bag-of-bones"
deformation (fit to a single video via test-time optimization). We accomplish
this by learning a neural implicit model that disentangles body versus clothing
deformations as separate motion model layers. To capture subtle geometry of
clothing, we leverage image-based priors such as human body pose, surface
normals, and optical flow during optimization. The resulting neural fields can
be extracted into time-consistent meshes, or further optimized as explicit 3D
Gaussians for high-fidelity interactive rendering. On datasets with highly
challenging clothing deformations and object interactions, DressRecon yields
higher-fidelity 3D reconstructions than prior art. Project page:
https://jefftan969.github.io/dressrecon/
A neural implicit model disentangles body and clothing deformations for high-fidelity 3D reconstruction from monocular videos, particularly for loose clothing and object interactions.
neural implicit model articulated body shape test-time optimization bag-of-bones deformation human body pose surface normals optical flow neural fields 3D Gaussians 3D reconstructions