ZeroShape: Regression-based Zero-shot Shape Reconstruction
We study the problem of single-image zero-shot 3D shape reconstruction.
Recent works learn zero-shot shape reconstruction through generative modeling
of 3D assets, but these models are computationally expensive at train and
inference time. In contrast, the traditional approach to this problem is
regression-based, where deterministic models are trained to directly regress
the object shape. Such regression methods possess much higher computational
efficiency than generative methods. This raises a natural question: is
generative modeling necessary for high performance, or conversely, are
regression-based approaches still competitive? To answer this, we design a
strong regression-based model, called ZeroShape, based on the converging
findings in this field and a novel insight. We also curate a large real-world
evaluation benchmark, with objects from three different real-world 3D datasets.
This evaluation benchmark is more diverse and an order of magnitude larger than
what prior works use to quantitatively evaluate their models, aiming at
reducing the evaluation variance in our field. We show that ZeroShape not only
achieves superior performance over state-of-the-art methods, but also
demonstrates significantly higher computational and data efficiency.
658a60e816a6a0082058345e
Zixuan Huang
claimed_verified
2024-04-05T10:01:20.608Z
629fe0dd198a9a5a755f3ce0
/avatars/e643d43f66c10729f155edca96aef1f8.svg
Zixuan Huang
user
zxhuang1698
658a60e816a6a0082058345f
Stefan Stojanov
658a60e816a6a00820583460
Anh Thai
claimed_verified
2024-09-25T07:36:00.706Z
653f30084a52f10eaf8381b8
/avatars/ffd938c2c828c4c8cc7204eb5424b3bf.svg
Anh Thai
user
ngailapdi
658a60e816a6a00820583461
Varun Jampani
658a60e816a6a00820583462
James M. Rehg
ZeroShape: Regression-based Zero-shot Shape Reconstruction
We study the problem of single-image zero-shot 3D shape reconstruction.
Recent works learn zero-shot shape reconstruction through generative modeling
of 3D assets, but these models are computationally expensive at train and
inference time. In contrast, the traditional approach to this problem is
regression-based, where deterministic models are trained to directly regress
the object shape. Such regression methods possess much higher computational
efficiency than generative methods. This raises a natural question: is
generative modeling necessary for high performance, or conversely, are
regression-based approaches still competitive? To answer this, we design a
strong regression-based model, called ZeroShape, based on the converging
findings in this field and a novel insight. We also curate a large real-world
evaluation benchmark, with objects from three different real-world 3D datasets.
This evaluation benchmark is more diverse and an order of magnitude larger than
what prior works use to quantitatively evaluate their models, aiming at
reducing the evaluation variance in our field. We show that ZeroShape not only
achieves superior performance over state-of-the-art methods, but also
demonstrates significantly higher computational and data efficiency.
A regression-based model named ZeroShape achieves superior performance and higher computational efficiency in single-image zero-shot 3D shape reconstruction compared to generative models.
zero-shot 3D shape reconstruction generative modeling regression-based models ZeroShape 3D datasets evaluation benchmark