DiMeR: Disentangled Mesh Reconstruction Model
With the advent of large-scale 3D datasets, feed-forward 3D generative
models, such as the Large Reconstruction Model (LRM), have gained significant
attention and achieved remarkable success. However, we observe that RGB images
often lead to conflicting training objectives and lack the necessary clarity
for geometry reconstruction. In this paper, we revisit the inductive biases
associated with mesh reconstruction and introduce DiMeR, a novel disentangled
dual-stream feed-forward model for sparse-view mesh reconstruction. The key
idea is to disentangle both the input and framework into geometry and texture
parts, thereby reducing the training difficulty for each part according to the
Principle of Occam's Razor. Given that normal maps are strictly consistent with
geometry and accurately capture surface variations, we utilize normal maps as
exclusive input for the geometry branch to reduce the complexity between the
network's input and output. Moreover, we improve the mesh extraction algorithm
to introduce 3D ground truth supervision. As for texture branch, we use RGB
images as input to obtain the textured mesh. Overall, DiMeR demonstrates robust
capabilities across various tasks, including sparse-view reconstruction,
single-image-to-3D, and text-to-3D. Numerous experiments show that DiMeR
significantly outperforms previous methods, achieving over 30% improvement in
Chamfer Distance on the GSO and OmniObject3D dataset.
680af5c5e5f0b6a3068bc842
Lutao Jiang
claimed_verified
2025-04-25T08:34:32.497Z
65d61d8fd484956b5acc89fe
/avatars/47954232c90780ffe898a5a445f7fb0a.svg
Lutao Jiang
user
LutaoJiang
680af5c5e5f0b6a3068bc843
Jiantao Lin
claimed_verified
2025-04-26T08:53:36.695Z
6332e2689bf698ce68a22e8c
/avatars/c1922acfda2e6d2fe7b03194a404eb10.svg
JIANTAO LIN
user
LTT
680af5c5e5f0b6a3068bc844
Kanghao Chen
680af5c5e5f0b6a3068bc845
Wenhang Ge
680af5c5e5f0b6a3068bc846
Xin Yang
680af5c5e5f0b6a3068bc847
Yifan Jiang
680af5c5e5f0b6a3068bc848
Yuanhuiyi Lyu
680af5c5e5f0b6a3068bc849
Xu Zheng
claimed_verified
2025-06-12T12:44:11.435Z
65961a7daa677adc97d8e33f
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/FkwP-qqxjAhPeaeQ2Y5sv.jpeg
Xu Zheng
user
xz287
680af5c5e5f0b6a3068bc84a
Yingcong Chen
DiMeR: Disentangled Mesh Reconstruction Model
With the advent of large-scale 3D datasets, feed-forward 3D generative
models, such as the Large Reconstruction Model (LRM), have gained significant
attention and achieved remarkable success. However, we observe that RGB images
often lead to conflicting training objectives and lack the necessary clarity
for geometry reconstruction. In this paper, we revisit the inductive biases
associated with mesh reconstruction and introduce DiMeR, a novel disentangled
dual-stream feed-forward model for sparse-view mesh reconstruction. The key
idea is to disentangle both the input and framework into geometry and texture
parts, thereby reducing the training difficulty for each part according to the
Principle of Occam's Razor. Given that normal maps are strictly consistent with
geometry and accurately capture surface variations, we utilize normal maps as
exclusive input for the geometry branch to reduce the complexity between the
network's input and output. Moreover, we improve the mesh extraction algorithm
to introduce 3D ground truth supervision. As for texture branch, we use RGB
images as input to obtain the textured mesh. Overall, DiMeR demonstrates robust
capabilities across various tasks, including sparse-view reconstruction,
single-image-to-3D, and text-to-3D. Numerous experiments show that DiMeR
significantly outperforms previous methods, achieving over 30% improvement in
Chamfer Distance on the GSO and OmniObject3D dataset.
DiMeR, a disentangled dual-stream feed-forward model, improves sparse-view 3D mesh reconstruction using normal maps for geometry and RGB images for texture, outperforming previous methods by over 30% in Chamfer Distance.
feed-forward 3D generative models Large Reconstruction Model (LRM) mesh reconstruction disentangled dual-stream geometry branch texture branch normal maps 3D ground truth supervision Chamfer Distance GSO OmniObject3D dataset