title: DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion
    Prior
summary: We present DreamCraft3D, a hierarchical 3D content generation method that
  produces high-fidelity and coherent 3D objects. We tackle the problem by
  leveraging a 2D reference image to guide the stages of geometry sculpting and
  texture boosting. A central focus of this work is to address the consistency
  issue that existing works encounter. To sculpt geometries that render
  coherently, we perform score distillation sampling via a view-dependent
  diffusion model. This 3D prior, alongside several training strategies,
  prioritizes the geometry consistency but compromises the texture fidelity. We
  further propose Bootstrapped Score Distillation to specifically boost the
  texture. We train a personalized diffusion model, Dreambooth, on the augmented
  renderings of the scene, imbuing it with 3D knowledge of the scene being
  optimized. The score distillation from this 3D-aware diffusion prior provides
  view-consistent guidance for the scene. Notably, through an alternating
  optimization of the diffusion prior and 3D scene representation, we achieve
  mutually reinforcing improvements: the optimized 3D scene aids in training the
  scene-specific diffusion model, which offers increasingly view-consistent
  guidance for 3D optimization. The optimization is thus bootstrapped and leads
  to substantial texture boosting. With tailored 3D priors throughout the
  hierarchical generation, DreamCraft3D generates coherent 3D objects with
  photorealistic renderings, advancing the state-of-the-art in 3D content
  generation. Code available at https://github.com/deepseek-ai/DreamCraft3D.
paper_authors._id: 6539da7f26df26ecd1350d99
paper_authors.name: Jingxiang Sun
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-26T08:16:05.029Z
paper_authors._id: 6456018a95082f722d136946
paper_authors.avatarUrl: /avatars/c9f825b9ac0af1e6f615ddcdd9a771ac.svg
paper_authors.fullname: jingxiang sun
paper_authors.type: user
paper_authors.user: starksun
paper_authors._id: 6539da7f26df26ecd1350d9a
paper_authors.name: Bo Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-12T13:33:57.287Z
paper_authors._id: 6237f3eb4cf571bb1ed4036b
paper_authors.avatarUrl: /avatars/cbfb8ddda18e65d18a818db9f176e16e.svg
paper_authors.fullname: Bo Zhang
paper_authors.type: user
paper_authors.user: zhangmozhe
paper_authors._id: 6539da7f26df26ecd1350d9b
paper_authors.name: Ruizhi Shao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-26T07:47:45.986Z
paper_authors._id: 64785ad575a9a0cfabcf5083
paper_authors.avatarUrl: /avatars/5b8526bf2d8389310ca1799655cbbcd8.svg
paper_authors.fullname: Ruizhi Shao
paper_authors.type: user
paper_authors.user: DSaurus
paper_authors._id: 6539da7f26df26ecd1350d9c
paper_authors.name: Lizhen Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-26T07:47:55.693Z
paper_authors._id: 63451d2dfeba4bdba59cf9b1
paper_authors.avatarUrl: /avatars/ff5dd2f3b502c54e7c3e2512c3e98b28.svg
paper_authors.fullname: Lizhen Wang
paper_authors.type: user
paper_authors.user: wanglz14
paper_authors._id: 6539da7f26df26ecd1350d9d
paper_authors.name: Wen Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-26T07:50:50.243Z
paper_authors._id: 63198e8802fb322037332f2d
paper_authors.avatarUrl: /avatars/d3f9c206e387df35beb0ed0ef1cdf865.svg
paper_authors.fullname: Wen Liu
paper_authors.type: user
paper_authors.user: doubility123
paper_authors._id: 6539da7f26df26ecd1350d9e
paper_authors.name: Zhenda Xie
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-26T07:48:38.640Z
paper_authors._id: 6539f6ea26df26ecd1393c37
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6539f6ea26df26ecd1393c37/9VJusLLAiLhUxfBAFggpF.jpeg
paper_authors.fullname: Zhenda Xie
paper_authors.type: user
paper_authors.user: zdaxie
paper_authors._id: 6539da7f26df26ecd1350d9f
paper_authors.name: Yebin Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-26T07:48:55.762Z
paper_authors._id: 62e14dbe4db2175cd2735a80
paper_authors.avatarUrl: /avatars/e6385dcedcb97e1b36281b49210321aa.svg
paper_authors.fullname: Yebin Liu
paper_authors.type: user
paper_authors.user: YebinLiu
paper_title: DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion
    Prior
paper_summary: We present DreamCraft3D, a hierarchical 3D content generation method that
  produces high-fidelity and coherent 3D objects. We tackle the problem by
  leveraging a 2D reference image to guide the stages of geometry sculpting and
  texture boosting. A central focus of this work is to address the consistency
  issue that existing works encounter. To sculpt geometries that render
  coherently, we perform score distillation sampling via a view-dependent
  diffusion model. This 3D prior, alongside several training strategies,
  prioritizes the geometry consistency but compromises the texture fidelity. We
  further propose Bootstrapped Score Distillation to specifically boost the
  texture. We train a personalized diffusion model, Dreambooth, on the augmented
  renderings of the scene, imbuing it with 3D knowledge of the scene being
  optimized. The score distillation from this 3D-aware diffusion prior provides
  view-consistent guidance for the scene. Notably, through an alternating
  optimization of the diffusion prior and 3D scene representation, we achieve
  mutually reinforcing improvements: the optimized 3D scene aids in training the
  scene-specific diffusion model, which offers increasingly view-consistent
  guidance for 3D optimization. The optimization is thus bootstrapped and leads
  to substantial texture boosting. With tailored 3D priors throughout the
  hierarchical generation, DreamCraft3D generates coherent 3D objects with
  photorealistic renderings, advancing the state-of-the-art in 3D content
  generation. Code available at https://github.com/deepseek-ai/DreamCraft3D.
paper_ai_summary: DreamCraft3D uses a hierarchical approach with diffusion models and tailored 3D priors to generate high-fidelity and coherent 3D objects from 2D references, addressing geometry consistency and texture fidelity.
paper_ai_keywords: diffusion model score distillation sampling Bootstrapped Score Distillation Dreambooth 3D prior alternating optimization view-dependent diffusion model 3D scene representation 3D content generation