DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement
  Learning
Triangle meshes play a crucial role in 3D applications for efficient
manipulation and rendering. While auto-regressive methods generate structured
meshes by predicting discrete vertex tokens, they are often constrained by
limited face counts and mesh incompleteness. To address these challenges, we
propose DeepMesh, a framework that optimizes mesh generation through two key
innovations: (1) an efficient pre-training strategy incorporating a novel
tokenization algorithm, along with improvements in data curation and
processing, and (2) the introduction of Reinforcement Learning (RL) into 3D
mesh generation to achieve human preference alignment via Direct Preference
Optimization (DPO). We design a scoring standard that combines human evaluation
with 3D metrics to collect preference pairs for DPO, ensuring both visual
appeal and geometric accuracy. Conditioned on point clouds and images, DeepMesh
generates meshes with intricate details and precise topology, outperforming
state-of-the-art methods in both precision and quality. Project page:
https://zhaorw02.github.io/DeepMesh/
67db8c4c9e4f93ee46411c1d
Ruowen Zhao
claimed_verified
2025-03-20T10:45:12.321Z
6522e4fbd89bc7773ddc4b58
/avatars/3e9b158af52c5f738a3eae72dcbb3824.svg
Ruowen Zhao
user
zzzrw
67db8c4c9e4f93ee46411c1e
Junliang Ye
claimed_verified
2025-03-20T10:45:14.849Z
65a420cd90e65dc39a6abe9e
/avatars/81ac5b749043e899f5017782409f9e28.svg
yejunliang
user
yejunliang23
67db8c4c9e4f93ee46411c1f
Zhengyi Wang
claimed_verified
2025-03-20T19:17:06.252Z
634e15aec1ce28f1de91c470
https://cdn-avatars.huggingface.co/v1/production/uploads/634e15aec1ce28f1de91c470/MdtJKgE_fbHydhqESr-DH.png
Zhengyi Wang
user
Zhengyi
67db8c4c9e4f93ee46411c20
Guangce Liu
admin_assigned
2025-03-20T11:06:05.371Z
674fcc541dbfdd4dee12d8e1
https://cdn-avatars.huggingface.co/v1/production/uploads/674fcc541dbfdd4dee12d8e1/Tyr8bc1PgMXTov3JeYg-R.jpeg
Guangce Liu
user
guangce
67db8c4c9e4f93ee46411c21
Yiwen Chen
admin_assigned
2025-03-20T11:06:35.743Z
6422a9669a00fbe9dfe665a1
/avatars/288bf5ea18205bcb0bbea29a304ffbbb.svg
Yiwen Chen
user
NTU-yiwen
67db8c4c9e4f93ee46411c22
Yikai Wang
admin_assigned
2025-03-20T11:06:45.552Z
63463bc4547c70e4b7d3009f
/avatars/6e5350fd998f0a7a4143d7504218164a.svg
Yikai Wang
user
yikaiwang
67db8c4c9e4f93ee46411c23
Jun Zhu
DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement
  Learning
Triangle meshes play a crucial role in 3D applications for efficient
manipulation and rendering. While auto-regressive methods generate structured
meshes by predicting discrete vertex tokens, they are often constrained by
limited face counts and mesh incompleteness. To address these challenges, we
propose DeepMesh, a framework that optimizes mesh generation through two key
innovations: (1) an efficient pre-training strategy incorporating a novel
tokenization algorithm, along with improvements in data curation and
processing, and (2) the introduction of Reinforcement Learning (RL) into 3D
mesh generation to achieve human preference alignment via Direct Preference
Optimization (DPO). We design a scoring standard that combines human evaluation
with 3D metrics to collect preference pairs for DPO, ensuring both visual
appeal and geometric accuracy. Conditioned on point clouds and images, DeepMesh
generates meshes with intricate details and precise topology, outperforming
state-of-the-art methods in both precision and quality. Project page:
https://zhaorw02.github.io/DeepMesh/
DeepMesh generates high-quality 3D meshes with intricate details and precise topology using an efficient pre-training strategy and reinforcement learning, outperforming existing methods.
auto-regressive methods tokenization algorithm data curation Reinforcement Learning Direct Preference Optimization DPO scoring standard visual appeal geometric accuracy point clouds images mesh generation topology