Single-View 3D Human Digitalization with Large Reconstruction Models
In this paper, we introduce Human-LRM, a single-stage feed-forward Large
Reconstruction Model designed to predict human Neural Radiance Fields (NeRF)
from a single image. Our approach demonstrates remarkable adaptability in
training using extensive datasets containing 3D scans and multi-view capture.
Furthermore, to enhance the model's applicability for in-the-wild scenarios
especially with occlusions, we propose a novel strategy that distills
multi-view reconstruction into single-view via a conditional triplane diffusion
model. This generative extension addresses the inherent variations in human
body shapes when observed from a single view, and makes it possible to
reconstruct the full body human from an occluded image. Through extensive
experiments, we show that Human-LRM surpasses previous methods by a significant
margin on several benchmarks.
65af54cd1f418c74494151a3
Zhenzhen Weng
admin_assigned
2024-01-23T11:15:27.845Z
630834fbf0dc38fb47c245b2
/avatars/579d083fdbfba7331b07941f0bef83a4.svg
Zhenzhen Weng
user
zzweng
65af54cd1f418c74494151a4
Jingyuan Liu
65af54cd1f418c74494151a5
Hao Tan
65af54cd1f418c74494151a6
Zhan Xu
65af54cd1f418c74494151a7
Yang Zhou
65af54cd1f418c74494151a8
Serena Yeung-Levy
65af54cd1f418c74494151a9
Jimei Yang
admin_assigned
2024-01-23T11:16:45.570Z
65ae93c53e82498d4cdaa428
/avatars/b03221ed402247f6e7a80efef834574f.svg
Yang
user
Jimei
Single-View 3D Human Digitalization with Large Reconstruction Models
In this paper, we introduce Human-LRM, a single-stage feed-forward Large
Reconstruction Model designed to predict human Neural Radiance Fields (NeRF)
from a single image. Our approach demonstrates remarkable adaptability in
training using extensive datasets containing 3D scans and multi-view capture.
Furthermore, to enhance the model's applicability for in-the-wild scenarios
especially with occlusions, we propose a novel strategy that distills
multi-view reconstruction into single-view via a conditional triplane diffusion
model. This generative extension addresses the inherent variations in human
body shapes when observed from a single view, and makes it possible to
reconstruct the full body human from an occluded image. Through extensive
experiments, we show that Human-LRM surpasses previous methods by a significant
margin on several benchmarks.
Human-LRM, a single-stage feed-forward model, predicts human Neural Radiance Fields from a single image using a conditional triplane diffusion model to handle occlusions.
Neural Radiance Fields triplane diffusion model