Magic123: One Image to High-Quality 3D Object Generation Using Both 2D
  and 3D Diffusion Priors
We present Magic123, a two-stage coarse-to-fine approach for high-quality,
textured 3D meshes generation from a single unposed image in the wild using
both2D and 3D priors. In the first stage, we optimize a neural radiance field
to produce a coarse geometry. In the second stage, we adopt a memory-efficient
differentiable mesh representation to yield a high-resolution mesh with a
visually appealing texture. In both stages, the 3D content is learned through
reference view supervision and novel views guided by a combination of 2D and 3D
diffusion priors. We introduce a single trade-off parameter between the 2D and
3D priors to control exploration (more imaginative) and exploitation (more
precise) of the generated geometry. Additionally, we employ textual inversion
and monocular depth regularization to encourage consistent appearances across
views and to prevent degenerate solutions, respectively. Magic123 demonstrates
a significant improvement over previous image-to-3D techniques, as validated
through extensive experiments on synthetic benchmarks and diverse real-world
images. Our code, models, and generated 3D assets are available at
https://github.com/guochengqian/Magic123.
64a229eb649b0d08ca028238
Guocheng Qian
claimed_verified
2023-07-04T07:24:40.936Z
645fed74335c21d19f3bf76c
https://cdn-avatars.huggingface.co/v1/production/uploads/645fed74335c21d19f3bf76c/gwVsllRWtSHbg4a1erkdF.jpeg
Guocheng Qian
user
guochengqian
64a229eb649b0d08ca028239
Jinjie Mai
64a229eb649b0d08ca02823a
Abdullah Hamdi
claimed_verified
2023-07-03T15:22:47.881Z
62fe3442e9061c0170d06e0b
https://cdn-avatars.huggingface.co/v1/production/uploads/1660827186084-62fe3442e9061c0170d06e0b.png
Abdullah Hamdi
user
ajhamdi
64a229eb649b0d08ca02823b
Jian Ren
claimed_verified
2023-07-04T19:17:17.468Z
61f19829233c91cbd2f79e70
/avatars/a0735a94542b4f7cda5aed8bc4be0538.svg
Jian Ren
user
alanspike
64a229eb649b0d08ca02823c
Aliaksandr Siarohin
64a229eb649b0d08ca02823d
Bing Li
claimed_verified
2024-06-15T21:27:46.912Z
666ddb45c0f3d5afc27e85ba
/avatars/dce98fc77bd8cf9e348f2d91bc3c0225.svg
Bing Li
user
bing-li-ai
64a229eb649b0d08ca02823e
Hsin-Ying Lee
claimed_verified
2023-07-06T09:36:19.131Z
631bbd918de8e645af71eed9
/avatars/1cb94deadc288d07c571c9289085548c.svg
Hsin-Ying Lee
user
james371507
64a229eb649b0d08ca02823f
Ivan Skorokhodov
64a229eb649b0d08ca028240
Peter Wonka
64a229eb649b0d08ca028241
Sergey Tulyakov
64a229eb649b0d08ca028242
Bernard Ghanem
Magic123: One Image to High-Quality 3D Object Generation Using Both 2D
  and 3D Diffusion Priors
We present Magic123, a two-stage coarse-to-fine approach for high-quality,
textured 3D meshes generation from a single unposed image in the wild using
both2D and 3D priors. In the first stage, we optimize a neural radiance field
to produce a coarse geometry. In the second stage, we adopt a memory-efficient
differentiable mesh representation to yield a high-resolution mesh with a
visually appealing texture. In both stages, the 3D content is learned through
reference view supervision and novel views guided by a combination of 2D and 3D
diffusion priors. We introduce a single trade-off parameter between the 2D and
3D priors to control exploration (more imaginative) and exploitation (more
precise) of the generated geometry. Additionally, we employ textual inversion
and monocular depth regularization to encourage consistent appearances across
views and to prevent degenerate solutions, respectively. Magic123 demonstrates
a significant improvement over previous image-to-3D techniques, as validated
through extensive experiments on synthetic benchmarks and diverse real-world
images. Our code, models, and generated 3D assets are available at
https://github.com/guochengqian/Magic123.
A two-stage neural approach generates high-quality textured 3D meshes from a single image using neural radiance fields, differentiable mesh representations, and combined 2D and 3D diffusion priors.
neural radiance field memory-efficient differentiable mesh representation reference view supervision 2D diffusion priors 3D diffusion priors textual inversion monocular depth regularization