title: TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified
    Flow Models
summary: Recent advancements in diffusion techniques have propelled image and video
  generation to unprece- dented levels of quality, significantly accelerating the
  deployment and application of generative AI. However, 3D shape generation
  technology has so far lagged behind, constrained by limitations in 3D data
  scale, complexity of 3D data process- ing, and insufficient exploration of
  advanced tech- niques in the 3D domain. Current approaches to 3D shape
  generation face substantial challenges in terms of output quality,
  generalization capa- bility, and alignment with input conditions. We present
  TripoSG, a new streamlined shape diffu- sion paradigm capable of generating
  high-fidelity 3D meshes with precise correspondence to input images.
  Specifically, we propose: 1) A large-scale rectified flow transformer for 3D
  shape generation, achieving state-of-the-art fidelity through training on
  extensive, high-quality data. 2) A hybrid supervised training strategy
  combining SDF, normal, and eikonal losses for 3D VAE, achieving high- quality
  3D reconstruction performance. 3) A data processing pipeline to generate 2
  million high- quality 3D samples, highlighting the crucial rules for data
  quality and quantity in training 3D gen- erative models. Through comprehensive
  experi- ments, we have validated the effectiveness of each component in our new
  framework. The seamless integration of these parts has enabled TripoSG to
  achieve state-of-the-art performance in 3D shape generation. The resulting 3D
  shapes exhibit en- hanced detail due to high-resolution capabilities and
  demonstrate exceptional fidelity to input im- ages. Moreover, TripoSG
  demonstrates improved versatility in generating 3D models from diverse image
  styles and contents, showcasing strong gen- eralization capabilities. To foster
  progress and innovation in the field of 3D generation, we will make our model
  publicly available.
paper_authors._id: 67aebe57f47426f753bc3b07
paper_authors.name: Yangguang Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-14T16:20:05.446Z
paper_authors._id: 64d71083a787c9bc7b9f1238
paper_authors.avatarUrl: /avatars/d0b0546dec7fc5792921154bec41385a.svg
paper_authors.fullname: Yangguang Li
paper_authors.type: user
paper_authors.user: Lp256
paper_authors._id: 67aebe57f47426f753bc3b08
paper_authors.name: Zi-Xin Zou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-14T12:44:17.925Z
paper_authors._id: 644dbf6453ad80c6593bf748
paper_authors.avatarUrl: /avatars/0e170cf2aa8d7f0f3f83e36f06f023f8.svg
paper_authors.fullname: Zixin Zou
paper_authors.type: user
paper_authors.user: zouzx
paper_authors._id: 67aebe57f47426f753bc3b09
paper_authors.name: Zexiang Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-14T12:44:23.987Z
paper_authors._id: 65ff33a830b872fe2ccc6c1e
paper_authors.avatarUrl: /avatars/cf15da8fc5a51f0b3ae0e11e3ff685cf.svg
paper_authors.fullname: Zexiang Liu
paper_authors.type: user
paper_authors.user: zexiangliu
paper_authors._id: 67aebe57f47426f753bc3b0a
paper_authors.name: Dehu Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-14T12:44:35.149Z
paper_authors._id: 666c4e95dc348adcab8f2836
paper_authors.avatarUrl: /avatars/875545d88ba7d935340015a719e6e5f0.svg
paper_authors.fullname: dehuwang
paper_authors.type: user
paper_authors.user: dehu168
paper_authors._id: 67aebe57f47426f753bc3b0b
paper_authors.name: Yuan Liang
paper_authors._id: 67aebe57f47426f753bc3b0c
paper_authors.name: Zhipeng Yu
paper_authors._id: 67aebe57f47426f753bc3b0d
paper_authors.name: Xingchao Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-14T12:45:05.465Z
paper_authors._id: 646b0bbdec9a61e871799339
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/646b0bbdec9a61e871799339/Xippv-ajkHkrGGAA7caLn.jpeg
paper_authors.fullname: Xingchao Liu
paper_authors.type: user
paper_authors.user: XCLiu
paper_authors._id: 67aebe57f47426f753bc3b0e
paper_authors.name: Yuan-Chen Guo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-14T12:45:11.634Z
paper_authors._id: 6346aaa3f06b237ba4e297b0
paper_authors.avatarUrl: /avatars/5acb986e993eab1461200f3e9d99d022.svg
paper_authors.fullname: Yuan-Chen Guo
paper_authors.type: user
paper_authors.user: bennyguo
paper_authors._id: 67aebe57f47426f753bc3b0f
paper_authors.name: Ding Liang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-14T12:45:32.067Z
paper_authors._id: 6476b01cbb7fdd7f425dcefb
paper_authors.avatarUrl: /avatars/65efe7067f47a68c204a1ab5a772b939.svg
paper_authors.fullname: dingliang
paper_authors.type: user
paper_authors.user: dingliang01
paper_authors._id: 67aebe57f47426f753bc3b10
paper_authors.name: Wanli Ouyang
paper_authors._id: 67aebe57f47426f753bc3b11
paper_authors.name: Yan-Pei Cao
paper_title: TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified
    Flow Models
paper_summary: Recent advancements in diffusion techniques have propelled image and video
  generation to unprece- dented levels of quality, significantly accelerating the
  deployment and application of generative AI. However, 3D shape generation
  technology has so far lagged behind, constrained by limitations in 3D data
  scale, complexity of 3D data process- ing, and insufficient exploration of
  advanced tech- niques in the 3D domain. Current approaches to 3D shape
  generation face substantial challenges in terms of output quality,
  generalization capa- bility, and alignment with input conditions. We present
  TripoSG, a new streamlined shape diffu- sion paradigm capable of generating
  high-fidelity 3D meshes with precise correspondence to input images.
  Specifically, we propose: 1) A large-scale rectified flow transformer for 3D
  shape generation, achieving state-of-the-art fidelity through training on
  extensive, high-quality data. 2) A hybrid supervised training strategy
  combining SDF, normal, and eikonal losses for 3D VAE, achieving high- quality
  3D reconstruction performance. 3) A data processing pipeline to generate 2
  million high- quality 3D samples, highlighting the crucial rules for data
  quality and quantity in training 3D gen- erative models. Through comprehensive
  experi- ments, we have validated the effectiveness of each component in our new
  framework. The seamless integration of these parts has enabled TripoSG to
  achieve state-of-the-art performance in 3D shape generation. The resulting 3D
  shapes exhibit en- hanced detail due to high-resolution capabilities and
  demonstrate exceptional fidelity to input im- ages. Moreover, TripoSG
  demonstrates improved versatility in generating 3D models from diverse image
  styles and contents, showcasing strong gen- eralization capabilities. To foster
  progress and innovation in the field of 3D generation, we will make our model
  publicly available.
paper_ai_summary: TripoSG, a novel shape diffusion framework, generates high-fidelity 3D meshes from images using a rectified flow transformer and hybrid training strategy.
paper_ai_keywords: shape diffusion rectified flow transformer SDF loss normal loss eikonal loss 3D VAE 3D VAE training 3D reconstruction data processing pipeline 3D shape generation