title: Magic123: One Image to High-Quality 3D Object Generation Using Both 2D
    and 3D Diffusion Priors
summary: We present Magic123, a two-stage coarse-to-fine approach for high-quality,
  textured 3D meshes generation from a single unposed image in the wild using
  both2D and 3D priors. In the first stage, we optimize a neural radiance field
  to produce a coarse geometry. In the second stage, we adopt a memory-efficient
  differentiable mesh representation to yield a high-resolution mesh with a
  visually appealing texture. In both stages, the 3D content is learned through
  reference view supervision and novel views guided by a combination of 2D and 3D
  diffusion priors. We introduce a single trade-off parameter between the 2D and
  3D priors to control exploration (more imaginative) and exploitation (more
  precise) of the generated geometry. Additionally, we employ textual inversion
  and monocular depth regularization to encourage consistent appearances across
  views and to prevent degenerate solutions, respectively. Magic123 demonstrates
  a significant improvement over previous image-to-3D techniques, as validated
  through extensive experiments on synthetic benchmarks and diverse real-world
  images. Our code, models, and generated 3D assets are available at
  https://github.com/guochengqian/Magic123.
paper_authors._id: 64a229eb649b0d08ca028238
paper_authors.name: Guocheng Qian
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-04T07:24:40.936Z
paper_authors._id: 645fed74335c21d19f3bf76c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/645fed74335c21d19f3bf76c/gwVsllRWtSHbg4a1erkdF.jpeg
paper_authors.fullname: Guocheng Qian
paper_authors.type: user
paper_authors.user: guochengqian
paper_authors._id: 64a229eb649b0d08ca028239
paper_authors.name: Jinjie Mai
paper_authors._id: 64a229eb649b0d08ca02823a
paper_authors.name: Abdullah Hamdi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-03T15:22:47.881Z
paper_authors._id: 62fe3442e9061c0170d06e0b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1660827186084-62fe3442e9061c0170d06e0b.png
paper_authors.fullname: Abdullah Hamdi
paper_authors.type: user
paper_authors.user: ajhamdi
paper_authors._id: 64a229eb649b0d08ca02823b
paper_authors.name: Jian Ren
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-04T19:17:17.468Z
paper_authors._id: 61f19829233c91cbd2f79e70
paper_authors.avatarUrl: /avatars/a0735a94542b4f7cda5aed8bc4be0538.svg
paper_authors.fullname: Jian Ren
paper_authors.type: user
paper_authors.user: alanspike
paper_authors._id: 64a229eb649b0d08ca02823c
paper_authors.name: Aliaksandr Siarohin
paper_authors._id: 64a229eb649b0d08ca02823d
paper_authors.name: Bing Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-15T21:27:46.912Z
paper_authors._id: 666ddb45c0f3d5afc27e85ba
paper_authors.avatarUrl: /avatars/dce98fc77bd8cf9e348f2d91bc3c0225.svg
paper_authors.fullname: Bing Li
paper_authors.type: user
paper_authors.user: bing-li-ai
paper_authors._id: 64a229eb649b0d08ca02823e
paper_authors.name: Hsin-Ying Lee
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-06T09:36:19.131Z
paper_authors._id: 631bbd918de8e645af71eed9
paper_authors.avatarUrl: /avatars/1cb94deadc288d07c571c9289085548c.svg
paper_authors.fullname: Hsin-Ying Lee
paper_authors.type: user
paper_authors.user: james371507
paper_authors._id: 64a229eb649b0d08ca02823f
paper_authors.name: Ivan Skorokhodov
paper_authors._id: 64a229eb649b0d08ca028240
paper_authors.name: Peter Wonka
paper_authors._id: 64a229eb649b0d08ca028241
paper_authors.name: Sergey Tulyakov
paper_authors._id: 64a229eb649b0d08ca028242
paper_authors.name: Bernard Ghanem
paper_title: Magic123: One Image to High-Quality 3D Object Generation Using Both 2D
    and 3D Diffusion Priors
paper_summary: We present Magic123, a two-stage coarse-to-fine approach for high-quality,
  textured 3D meshes generation from a single unposed image in the wild using
  both2D and 3D priors. In the first stage, we optimize a neural radiance field
  to produce a coarse geometry. In the second stage, we adopt a memory-efficient
  differentiable mesh representation to yield a high-resolution mesh with a
  visually appealing texture. In both stages, the 3D content is learned through
  reference view supervision and novel views guided by a combination of 2D and 3D
  diffusion priors. We introduce a single trade-off parameter between the 2D and
  3D priors to control exploration (more imaginative) and exploitation (more
  precise) of the generated geometry. Additionally, we employ textual inversion
  and monocular depth regularization to encourage consistent appearances across
  views and to prevent degenerate solutions, respectively. Magic123 demonstrates
  a significant improvement over previous image-to-3D techniques, as validated
  through extensive experiments on synthetic benchmarks and diverse real-world
  images. Our code, models, and generated 3D assets are available at
  https://github.com/guochengqian/Magic123.
paper_ai_summary: A two-stage neural approach generates high-quality textured 3D meshes from a single image using neural radiance fields, differentiable mesh representations, and combined 2D and 3D diffusion priors.
paper_ai_keywords: neural radiance field memory-efficient differentiable mesh representation reference view supervision 2D diffusion priors 3D diffusion priors textual inversion monocular depth regularization