title: One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape
    Optimization
summary: Single image 3D reconstruction is an important but challenging task that
  requires extensive knowledge of our natural world. Many existing methods solve
  this problem by optimizing a neural radiance field under the guidance of 2D
  diffusion models but suffer from lengthy optimization time, 3D inconsistency
  results, and poor geometry. In this work, we propose a novel method that takes
  a single image of any object as input and generates a full 360-degree 3D
  textured mesh in a single feed-forward pass. Given a single image, we first use
  a view-conditioned 2D diffusion model, Zero123, to generate multi-view images
  for the input view, and then aim to lift them up to 3D space. Since traditional
  reconstruction methods struggle with inconsistent multi-view predictions, we
  build our 3D reconstruction module upon an SDF-based generalizable neural
  surface reconstruction method and propose several critical training strategies
  to enable the reconstruction of 360-degree meshes. Without costly
  optimizations, our method reconstructs 3D shapes in significantly less time
  than existing methods. Moreover, our method favors better geometry, generates
  more 3D consistent results, and adheres more closely to the input image. We
  evaluate our approach on both synthetic data and in-the-wild images and
  demonstrate its superiority in terms of both mesh quality and runtime. In
  addition, our approach can seamlessly support the text-to-3D task by
  integrating with off-the-shelf text-to-image diffusion models.
paper_authors._id: 649e46c9ab373ff26ac4d9d8
paper_authors.name: Minghua Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-30T09:40:23.246Z
paper_authors._id: 63b70a696fca9d2a1904f606
paper_authors.avatarUrl: /avatars/a11741787b884cc865c1b7e9e6a7ec24.svg
paper_authors.fullname: Minghua Liu
paper_authors.type: user
paper_authors.user: minghua
paper_authors._id: 649e46c9ab373ff26ac4d9d9
paper_authors.name: Chao Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-30T09:46:17.347Z
paper_authors._id: 6468a48cff18750165a5aad2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/vmXEWka_q2a4mBDNjbMuY.jpeg
paper_authors.fullname: Chao Xu
paper_authors.type: user
paper_authors.user: chaoxu
paper_authors._id: 649e46c9ab373ff26ac4d9da
paper_authors.name: Haian Jin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-03T07:27:28.138Z
paper_authors._id: 64a04ad150251e408af12e01
paper_authors.avatarUrl: /avatars/e4f630616a16a7fc691b5275bb62f2aa.svg
paper_authors.fullname: Haian Jin
paper_authors.type: user
paper_authors.user: coast01
paper_authors._id: 649e46c9ab373ff26ac4d9db
paper_authors.name: Linghao Chen
paper_authors._id: 649e46c9ab373ff26ac4d9dc
paper_authors.name: Mukund Varma T
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-30T09:48:30.611Z
paper_authors._id: 61b9b452d9b770fe69ff439e
paper_authors.avatarUrl: /avatars/69f49cc9b5a472393905f6b0b223a273.svg
paper_authors.fullname: Mukund Varma T
paper_authors.type: user
paper_authors.user: sneezygiraffe
paper_authors._id: 649e46c9ab373ff26ac4d9dd
paper_authors.name: Zexiang Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-21T07:54:58.221Z
paper_authors._id: 65565b1b8616d8d740abb976
paper_authors.avatarUrl: /avatars/c245b19ed05292bf1bf4a9d8cf2a409c.svg
paper_authors.fullname: Zexiang Xu
paper_authors.type: user
paper_authors.user: zexiangxu
paper_authors._id: 649e46c9ab373ff26ac4d9de
paper_authors.name: Hao Su
paper_title: One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape
    Optimization
paper_summary: Single image 3D reconstruction is an important but challenging task that
  requires extensive knowledge of our natural world. Many existing methods solve
  this problem by optimizing a neural radiance field under the guidance of 2D
  diffusion models but suffer from lengthy optimization time, 3D inconsistency
  results, and poor geometry. In this work, we propose a novel method that takes
  a single image of any object as input and generates a full 360-degree 3D
  textured mesh in a single feed-forward pass. Given a single image, we first use
  a view-conditioned 2D diffusion model, Zero123, to generate multi-view images
  for the input view, and then aim to lift them up to 3D space. Since traditional
  reconstruction methods struggle with inconsistent multi-view predictions, we
  build our 3D reconstruction module upon an SDF-based generalizable neural
  surface reconstruction method and propose several critical training strategies
  to enable the reconstruction of 360-degree meshes. Without costly
  optimizations, our method reconstructs 3D shapes in significantly less time
  than existing methods. Moreover, our method favors better geometry, generates
  more 3D consistent results, and adheres more closely to the input image. We
  evaluate our approach on both synthetic data and in-the-wild images and
  demonstrate its superiority in terms of both mesh quality and runtime. In
  addition, our approach can seamlessly support the text-to-3D task by
  integrating with off-the-shelf text-to-image diffusion models.
paper_ai_summary: A novel feed-forward method generates high-quality 360-degree 3D meshes from single images using a view-conditioned 2D diffusion model and an SDF-based neural surface reconstruction module.
paper_ai_keywords: neural radiance field 2D diffusion models Zero123 3D reconstruction feed-forward pass multi-view images SDF-based neural surface reconstruction 360-degree meshes mesh quality text-to-3D text-to-image diffusion models