title: CraftsMan: High-fidelity Mesh Generation with 3D Native Generation and
    Interactive Geometry Refiner
summary: We present a novel generative 3D modeling system, coined CraftsMan, which can
  generate high-fidelity 3D geometries with highly varied shapes, regular mesh
  topologies, and detailed surfaces, and, notably, allows for refining the
  geometry in an interactive manner. Despite the significant advancements in 3D
  generation, existing methods still struggle with lengthy optimization
  processes, irregular mesh topologies, noisy surfaces, and difficulties in
  accommodating user edits, consequently impeding their widespread adoption and
  implementation in 3D modeling software. Our work is inspired by the craftsman,
  who usually roughs out the holistic figure of the work first and elaborates the
  surface details subsequently. Specifically, we employ a 3D native diffusion
  model, which operates on latent space learned from latent set-based 3D
  representations, to generate coarse geometries with regular mesh topology in
  seconds. In particular, this process takes as input a text prompt or a
  reference image and leverages a powerful multi-view (MV) diffusion model to
  generate multiple views of the coarse geometry, which are fed into our
  MV-conditioned 3D diffusion model for generating the 3D geometry, significantly
  improving robustness and generalizability. Following that, a normal-based
  geometry refiner is used to significantly enhance the surface details. This
  refinement can be performed automatically, or interactively with user-supplied
  edits. Extensive experiments demonstrate that our method achieves high efficacy
  in producing superior-quality 3D assets compared to existing methods. HomePage:
  https://craftsman3d.github.io/, Code: https://github.com/wyysf-98/CraftsMan
paper_authors._id: 6653e7c995bc4a27bafe985e
paper_authors.name: Weiyu Li
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-05-27T01:54:21.710Z
paper_authors._id: 63b53090889aa6707f07afd6
paper_authors.avatarUrl: /avatars/595ecfea802c6faed50e031d442f1639.svg
paper_authors.fullname: weiyuli
paper_authors.type: user
paper_authors.user: wyysf
paper_authors._id: 6653e7c995bc4a27bafe985f
paper_authors.name: Jiarui Liu
paper_authors._id: 6653e7c995bc4a27bafe9860
paper_authors.name: Rui Chen
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-05-27T01:54:21.710Z
paper_authors._id: 650d3c195d57feb8471895f5
paper_authors.avatarUrl: /avatars/826420dc293473d6518a251e62cf1e29.svg
paper_authors.fullname: Xuelin Chen
paper_authors.type: user
paper_authors.user: XuelinChen
paper_authors._id: 6653e7c995bc4a27bafe9861
paper_authors.name: Yixun Liang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-05-27T09:10:37.180Z
paper_authors._id: 647d8f65becb41a272907e7a
paper_authors.avatarUrl: /avatars/f891a1986a9361b23b7c83c23031cf26.svg
paper_authors.fullname: yixunliang
paper_authors.type: user
paper_authors.user: lyxun
paper_authors._id: 6653e7c995bc4a27bafe9862
paper_authors.name: Xuelin Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-27T08:41:02.590Z
paper_authors._id: 650d3c195d57feb8471895f5
paper_authors.avatarUrl: /avatars/826420dc293473d6518a251e62cf1e29.svg
paper_authors.fullname: Xuelin Chen
paper_authors.type: user
paper_authors.user: XuelinChen
paper_authors._id: 6653e7c995bc4a27bafe9863
paper_authors.name: Ping Tan
paper_authors._id: 6653e7c995bc4a27bafe9864
paper_authors.name: Xiaoxiao Long
paper_title: CraftsMan: High-fidelity Mesh Generation with 3D Native Generation and
    Interactive Geometry Refiner
paper_summary: We present a novel generative 3D modeling system, coined CraftsMan, which can
  generate high-fidelity 3D geometries with highly varied shapes, regular mesh
  topologies, and detailed surfaces, and, notably, allows for refining the
  geometry in an interactive manner. Despite the significant advancements in 3D
  generation, existing methods still struggle with lengthy optimization
  processes, irregular mesh topologies, noisy surfaces, and difficulties in
  accommodating user edits, consequently impeding their widespread adoption and
  implementation in 3D modeling software. Our work is inspired by the craftsman,
  who usually roughs out the holistic figure of the work first and elaborates the
  surface details subsequently. Specifically, we employ a 3D native diffusion
  model, which operates on latent space learned from latent set-based 3D
  representations, to generate coarse geometries with regular mesh topology in
  seconds. In particular, this process takes as input a text prompt or a
  reference image and leverages a powerful multi-view (MV) diffusion model to
  generate multiple views of the coarse geometry, which are fed into our
  MV-conditioned 3D diffusion model for generating the 3D geometry, significantly
  improving robustness and generalizability. Following that, a normal-based
  geometry refiner is used to significantly enhance the surface details. This
  refinement can be performed automatically, or interactively with user-supplied
  edits. Extensive experiments demonstrate that our method achieves high efficacy
  in producing superior-quality 3D assets compared to existing methods. HomePage:
  https://craftsman3d.github.io/, Code: https://github.com/wyysf-98/CraftsMan
paper_ai_keywords: 3D generative modeling CraftsMan high-fidelity 3D geometries regular mesh topologies detailed surfaces 3D native diffusion model latent state latent set-based 3D representations coarse geometries text prompt reference image multi-view (MV) diffusion model MV-conditioned 3D diffusion model normal-based geometry refiner user-supplied edits superior-quality 3D assets