title: MeshCraft: Exploring Efficient and Controllable Mesh Generation with
    Flow-based DiTs
summary: In the domain of 3D content creation, achieving optimal mesh topology through
  AI models has long been a pursuit for 3D artists. Previous methods, such as
  MeshGPT, have explored the generation of ready-to-use 3D objects via mesh
  auto-regressive techniques. While these methods produce visually impressive
  results, their reliance on token-by-token predictions in the auto-regressive
  process leads to several significant limitations. These include extremely slow
  generation speeds and an uncontrollable number of mesh faces. In this paper, we
  introduce MeshCraft, a novel framework for efficient and controllable mesh
  generation, which leverages continuous spatial diffusion to generate discrete
  triangle faces. Specifically, MeshCraft consists of two core components: 1) a
  transformer-based VAE that encodes raw meshes into continuous face-level tokens
  and decodes them back to the original meshes, and 2) a flow-based diffusion
  transformer conditioned on the number of faces, enabling the generation of
  high-quality 3D meshes with a predefined number of faces. By utilizing the
  diffusion model for the simultaneous generation of the entire mesh topology,
  MeshCraft achieves high-fidelity mesh generation at significantly faster speeds
  compared to auto-regressive methods. Specifically, MeshCraft can generate an
  800-face mesh in just 3.2 seconds (35times faster than existing baselines).
  Extensive experiments demonstrate that MeshCraft outperforms state-of-the-art
  techniques in both qualitative and quantitative evaluations on ShapeNet dataset
  and demonstrates superior performance on Objaverse dataset. Moreover, it
  integrates seamlessly with existing conditional guidance strategies, showcasing
  its potential to relieve artists from the time-consuming manual work involved
  in mesh creation.
paper_authors._id: 67ebadecf9f9390b4cd1c6d9
paper_authors.name: Xianglong He
paper_authors._id: 67ebadecf9f9390b4cd1c6da
paper_authors.name: Junyi Chen
paper_authors._id: 67ebadecf9f9390b4cd1c6db
paper_authors.name: Di Huang
paper_authors._id: 67ebadecf9f9390b4cd1c6dc
paper_authors.name: Zexiang Liu
paper_authors._id: 67ebadecf9f9390b4cd1c6dd
paper_authors.name: Xiaoshui Huang
paper_authors._id: 67ebadecf9f9390b4cd1c6de
paper_authors.name: Wanli Ouyang
paper_authors._id: 67ebadecf9f9390b4cd1c6df
paper_authors.name: Chun Yuan
paper_authors._id: 67ebadecf9f9390b4cd1c6e0
paper_authors.name: Yangguang Li
paper_title: MeshCraft: Exploring Efficient and Controllable Mesh Generation with
    Flow-based DiTs
paper_summary: In the domain of 3D content creation, achieving optimal mesh topology through
  AI models has long been a pursuit for 3D artists. Previous methods, such as
  MeshGPT, have explored the generation of ready-to-use 3D objects via mesh
  auto-regressive techniques. While these methods produce visually impressive
  results, their reliance on token-by-token predictions in the auto-regressive
  process leads to several significant limitations. These include extremely slow
  generation speeds and an uncontrollable number of mesh faces. In this paper, we
  introduce MeshCraft, a novel framework for efficient and controllable mesh
  generation, which leverages continuous spatial diffusion to generate discrete
  triangle faces. Specifically, MeshCraft consists of two core components: 1) a
  transformer-based VAE that encodes raw meshes into continuous face-level tokens
  and decodes them back to the original meshes, and 2) a flow-based diffusion
  transformer conditioned on the number of faces, enabling the generation of
  high-quality 3D meshes with a predefined number of faces. By utilizing the
  diffusion model for the simultaneous generation of the entire mesh topology,
  MeshCraft achieves high-fidelity mesh generation at significantly faster speeds
  compared to auto-regressive methods. Specifically, MeshCraft can generate an
  800-face mesh in just 3.2 seconds (35times faster than existing baselines).
  Extensive experiments demonstrate that MeshCraft outperforms state-of-the-art
  techniques in both qualitative and quantitative evaluations on ShapeNet dataset
  and demonstrates superior performance on Objaverse dataset. Moreover, it
  integrates seamlessly with existing conditional guidance strategies, showcasing
  its potential to relieve artists from the time-consuming manual work involved
  in mesh creation.
paper_ai_summary: MeshCraft uses continuous spatial diffusion and transformer-based VAE for efficient, high-fidelity, and controllable 3D mesh generation, outperforming auto-regressive methods.
paper_ai_keywords: mesh auto-regressive techniques continuous spatial diffusion transformer-based VAE flow-based diffusion transformer mesh topology ShapeNet Objaverse conditional guidance strategies