title: Ouroboros3D: Image-to-3D Generation via 3D-aware Recursive Diffusion
summary: Existing single image-to-3D creation methods typically involve a two-stage
  process, first generating multi-view images, and then using these images for 3D
  reconstruction. However, training these two stages separately leads to
  significant data bias in the inference phase, thus affecting the quality of
  reconstructed results. We introduce a unified 3D generation framework, named
  Ouroboros3D, which integrates diffusion-based multi-view image generation and
  3D reconstruction into a recursive diffusion process. In our framework, these
  two modules are jointly trained through a self-conditioning mechanism, allowing
  them to adapt to each other's characteristics for robust inference. During the
  multi-view denoising process, the multi-view diffusion model uses the 3D-aware
  maps rendered by the reconstruction module at the previous timestep as
  additional conditions. The recursive diffusion framework with 3D-aware feedback
  unites the entire process and improves geometric consistency.Experiments show
  that our framework outperforms separation of these two stages and existing
  methods that combine them at the inference phase. Project page:
  https://costwen.github.io/Ouroboros3D/
paper_authors._id: 666119207427717d286d8ece
paper_authors.name: Hao Wen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-05T09:46:29.864Z
paper_authors._id: 6356194ce1e6a03a1ed3e0b3
paper_authors.avatarUrl: /avatars/92adf0c54549ac99315fa36da78b0448.svg
paper_authors.fullname: wenhao
paper_authors.type: user
paper_authors.user: costwen
paper_authors._id: 666119207427717d286d8ecf
paper_authors.name: Zehuan Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-06T07:25:33.462Z
paper_authors._id: 6375d136dee28348a9c63cbf
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6375d136dee28348a9c63cbf/gK465HBrQWIOZ-qtHb-Vh.jpeg
paper_authors.fullname: zehuan-huang
paper_authors.type: user
paper_authors.user: huanngzh
paper_authors._id: 666119207427717d286d8ed0
paper_authors.name: Yaohui Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-06T07:25:22.841Z
paper_authors._id: 63201256c6b20f03c829c4b8
paper_authors.avatarUrl: /avatars/a42092119777d65e60b12eb5ba0e45f1.svg
paper_authors.fullname: Yaohui Wang
paper_authors.type: user
paper_authors.user: YaohuiW
paper_authors._id: 666119207427717d286d8ed1
paper_authors.name: Xinyuan Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-06T07:24:50.672Z
paper_authors._id: 643e943a70c6a27621eb1c89
paper_authors.avatarUrl: /avatars/73ec521ab5ba84cc7908c52c0acef6ef.svg
paper_authors.fullname: Xinyuan Chen
paper_authors.type: user
paper_authors.user: AriaChen
paper_authors._id: 666119207427717d286d8ed2
paper_authors.name: Yu Qiao
paper_authors._id: 666119207427717d286d8ed3
paper_authors.name: Lu Sheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-06T15:58:19.372Z
paper_authors._id: 65b722dbe02a17f0f8d1cc6b
paper_authors.avatarUrl: /avatars/65f20601ef9b8ebfdddadd737f9153d6.svg
paper_authors.fullname: Lu Sheng
paper_authors.type: user
paper_authors.user: lsheng2024
paper_title: Ouroboros3D: Image-to-3D Generation via 3D-aware Recursive Diffusion
paper_summary: Existing single image-to-3D creation methods typically involve a two-stage
  process, first generating multi-view images, and then using these images for 3D
  reconstruction. However, training these two stages separately leads to
  significant data bias in the inference phase, thus affecting the quality of
  reconstructed results. We introduce a unified 3D generation framework, named
  Ouroboros3D, which integrates diffusion-based multi-view image generation and
  3D reconstruction into a recursive diffusion process. In our framework, these
  two modules are jointly trained through a self-conditioning mechanism, allowing
  them to adapt to each other's characteristics for robust inference. During the
  multi-view denoising process, the multi-view diffusion model uses the 3D-aware
  maps rendered by the reconstruction module at the previous timestep as
  additional conditions. The recursive diffusion framework with 3D-aware feedback
  unites the entire process and improves geometric consistency.Experiments show
  that our framework outperforms separation of these two stages and existing
  methods that combine them at the inference phase. Project page:
  https://costwen.github.io/Ouroboros3D/
paper_ai_keywords: diffusion-based multi-view image generation 3D reconstruction recursive diffusion process self-conditioning mechanism multi-view denoising multi-view diffusion model 3D-aware maps 3D-aware feedback geometric consistency