TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified
  Flow Models
Recent advancements in diffusion techniques have propelled image and video
generation to unprece- dented levels of quality, significantly accelerating the
deployment and application of generative AI. However, 3D shape generation
technology has so far lagged behind, constrained by limitations in 3D data
scale, complexity of 3D data process- ing, and insufficient exploration of
advanced tech- niques in the 3D domain. Current approaches to 3D shape
generation face substantial challenges in terms of output quality,
generalization capa- bility, and alignment with input conditions. We present
TripoSG, a new streamlined shape diffu- sion paradigm capable of generating
high-fidelity 3D meshes with precise correspondence to input images.
Specifically, we propose: 1) A large-scale rectified flow transformer for 3D
shape generation, achieving state-of-the-art fidelity through training on
extensive, high-quality data. 2) A hybrid supervised training strategy
combining SDF, normal, and eikonal losses for 3D VAE, achieving high- quality
3D reconstruction performance. 3) A data processing pipeline to generate 2
million high- quality 3D samples, highlighting the crucial rules for data
quality and quantity in training 3D gen- erative models. Through comprehensive
experi- ments, we have validated the effectiveness of each component in our new
framework. The seamless integration of these parts has enabled TripoSG to
achieve state-of-the-art performance in 3D shape generation. The resulting 3D
shapes exhibit en- hanced detail due to high-resolution capabilities and
demonstrate exceptional fidelity to input im- ages. Moreover, TripoSG
demonstrates improved versatility in generating 3D models from diverse image
styles and contents, showcasing strong gen- eralization capabilities. To foster
progress and innovation in the field of 3D generation, we will make our model
publicly available.
67aebe57f47426f753bc3b07
Yangguang Li
claimed_verified
2025-02-14T16:20:05.446Z
64d71083a787c9bc7b9f1238
/avatars/d0b0546dec7fc5792921154bec41385a.svg
Yangguang Li
user
Lp256
67aebe57f47426f753bc3b08
Zi-Xin Zou
admin_assigned
2025-02-14T12:44:17.925Z
644dbf6453ad80c6593bf748
/avatars/0e170cf2aa8d7f0f3f83e36f06f023f8.svg
Zixin Zou
user
zouzx
67aebe57f47426f753bc3b09
Zexiang Liu
admin_assigned
2025-02-14T12:44:23.987Z
65ff33a830b872fe2ccc6c1e
/avatars/cf15da8fc5a51f0b3ae0e11e3ff685cf.svg
Zexiang Liu
user
zexiangliu
67aebe57f47426f753bc3b0a
Dehu Wang
admin_assigned
2025-02-14T12:44:35.149Z
666c4e95dc348adcab8f2836
/avatars/875545d88ba7d935340015a719e6e5f0.svg
dehuwang
user
dehu168
67aebe57f47426f753bc3b0b
Yuan Liang
67aebe57f47426f753bc3b0c
Zhipeng Yu
67aebe57f47426f753bc3b0d
Xingchao Liu
admin_assigned
2025-02-14T12:45:05.465Z
646b0bbdec9a61e871799339
https://cdn-avatars.huggingface.co/v1/production/uploads/646b0bbdec9a61e871799339/Xippv-ajkHkrGGAA7caLn.jpeg
Xingchao Liu
user
XCLiu
67aebe57f47426f753bc3b0e
Yuan-Chen Guo
admin_assigned
2025-02-14T12:45:11.634Z
6346aaa3f06b237ba4e297b0
/avatars/5acb986e993eab1461200f3e9d99d022.svg
Yuan-Chen Guo
user
bennyguo
67aebe57f47426f753bc3b0f
Ding Liang
admin_assigned
2025-02-14T12:45:32.067Z
6476b01cbb7fdd7f425dcefb
/avatars/65efe7067f47a68c204a1ab5a772b939.svg
dingliang
user
dingliang01
67aebe57f47426f753bc3b10
Wanli Ouyang
67aebe57f47426f753bc3b11
Yan-Pei Cao
TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified
  Flow Models
Recent advancements in diffusion techniques have propelled image and video
generation to unprece- dented levels of quality, significantly accelerating the
deployment and application of generative AI. However, 3D shape generation
technology has so far lagged behind, constrained by limitations in 3D data
scale, complexity of 3D data process- ing, and insufficient exploration of
advanced tech- niques in the 3D domain. Current approaches to 3D shape
generation face substantial challenges in terms of output quality,
generalization capa- bility, and alignment with input conditions. We present
TripoSG, a new streamlined shape diffu- sion paradigm capable of generating
high-fidelity 3D meshes with precise correspondence to input images.
Specifically, we propose: 1) A large-scale rectified flow transformer for 3D
shape generation, achieving state-of-the-art fidelity through training on
extensive, high-quality data. 2) A hybrid supervised training strategy
combining SDF, normal, and eikonal losses for 3D VAE, achieving high- quality
3D reconstruction performance. 3) A data processing pipeline to generate 2
million high- quality 3D samples, highlighting the crucial rules for data
quality and quantity in training 3D gen- erative models. Through comprehensive
experi- ments, we have validated the effectiveness of each component in our new
framework. The seamless integration of these parts has enabled TripoSG to
achieve state-of-the-art performance in 3D shape generation. The resulting 3D
shapes exhibit en- hanced detail due to high-resolution capabilities and
demonstrate exceptional fidelity to input im- ages. Moreover, TripoSG
demonstrates improved versatility in generating 3D models from diverse image
styles and contents, showcasing strong gen- eralization capabilities. To foster
progress and innovation in the field of 3D generation, we will make our model
publicly available.
TripoSG, a novel shape diffusion framework, generates high-fidelity 3D meshes from images using a rectified flow transformer and hybrid training strategy.
shape diffusion rectified flow transformer SDF loss normal loss eikonal loss 3D VAE 3D VAE training 3D reconstruction data processing pipeline 3D shape generation