title: LRM-Zero: Training Large Reconstruction Models with Synthesized Data
summary: We present LRM-Zero, a Large Reconstruction Model (LRM) trained entirely on
  synthesized 3D data, achieving high-quality sparse-view 3D reconstruction. The
  core of LRM-Zero is our procedural 3D dataset, Zeroverse, which is
  automatically synthesized from simple primitive shapes with random texturing
  and augmentations (e.g., height fields, boolean differences, and wireframes).
  Unlike previous 3D datasets (e.g., Objaverse) which are often captured or
  crafted by humans to approximate real 3D data, Zeroverse completely ignores
  realistic global semantics but is rich in complex geometric and texture details
  that are locally similar to or even more intricate than real objects. We
  demonstrate that our LRM-Zero, trained with our fully synthesized Zeroverse,
  can achieve high visual quality in the reconstruction of real-world objects,
  competitive with models trained on Objaverse. We also analyze several critical
  design choices of Zeroverse that contribute to LRM-Zero's capability and
  training stability. Our work demonstrates that 3D reconstruction, one of the
  core tasks in 3D vision, can potentially be addressed without the semantics of
  real-world objects. The Zeroverse's procedural synthesis code and interactive
  visualization are available at: https://desaixie.github.io/lrm-zero/.
paper_authors._id: 666bd7d2f1b3867caf4b73c7
paper_authors.name: Desai Xie
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-14T11:09:38.093Z
paper_authors._id: 6279f597812ee439d9c8de67
paper_authors.avatarUrl: /avatars/49bea87294118e18df7b774f716af81a.svg
paper_authors.fullname: Desai Xie
paper_authors.type: user
paper_authors.user: desaix
paper_authors._id: 666bd7d2f1b3867caf4b73c8
paper_authors.name: Sai Bi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-14T11:09:44.136Z
paper_authors._id: 66303560597dc7c535168f07
paper_authors.avatarUrl: /avatars/a69d52a4a198c6c8451c1d137031bf07.svg
paper_authors.fullname: Sai Bi
paper_authors.type: user
paper_authors.user: saibi
paper_authors._id: 666bd7d2f1b3867caf4b73c9
paper_authors.name: Zhixin Shu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-14T11:09:51.363Z
paper_authors._id: 62a8efd508a7ea93ff18785a
paper_authors.avatarUrl: /avatars/ff259233d437833a304329bb973a5a04.svg
paper_authors.fullname: Zhixin Shu
paper_authors.type: user
paper_authors.user: zhixinshu
paper_authors._id: 666bd7d2f1b3867caf4b73ca
paper_authors.name: Kai Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-14T11:10:15.966Z
paper_authors._id: 63e0a50242591dda0b9dca5c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63e0a50242591dda0b9dca5c/c7cBPEBWQDFYimfGnO_SI.png
paper_authors.fullname: Kai Zhang
paper_authors.type: user
paper_authors.user: drogozhang
paper_authors._id: 666bd7d2f1b3867caf4b73cb
paper_authors.name: Zexiang Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-14T11:10:21.402Z
paper_authors._id: 65565b1b8616d8d740abb976
paper_authors.avatarUrl: /avatars/c245b19ed05292bf1bf4a9d8cf2a409c.svg
paper_authors.fullname: Zexiang Xu
paper_authors.type: user
paper_authors.user: zexiangxu
paper_authors._id: 666bd7d2f1b3867caf4b73cc
paper_authors.name: Yi Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-14T11:11:08.044Z
paper_authors._id: 6621dc68486de1d8db526535
paper_authors.avatarUrl: /avatars/1c8d6a62e6e61fda98748c2656e17851.svg
paper_authors.fullname: Yi Zhou
paper_authors.type: user
paper_authors.user: yizhou-seu
paper_authors._id: 666bd7d2f1b3867caf4b73cd
paper_authors.name: Sören Pirk
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-06-14T05:56:40.690Z
paper_authors._id: 658e87be43edad2169db050a
paper_authors.avatarUrl: /avatars/467d8949626539a7482dccbee4fb4d25.svg
paper_authors.fullname: Soren Pirk
paper_authors.type: user
paper_authors.user: pirk
paper_authors._id: 666bd7d2f1b3867caf4b73ce
paper_authors.name: Arie Kaufman
paper_authors._id: 666bd7d2f1b3867caf4b73cf
paper_authors.name: Xin Sun
paper_authors._id: 666bd7d2f1b3867caf4b73d0
paper_authors.name: Hao Tan
paper_title: LRM-Zero: Training Large Reconstruction Models with Synthesized Data
paper_summary: We present LRM-Zero, a Large Reconstruction Model (LRM) trained entirely on
  synthesized 3D data, achieving high-quality sparse-view 3D reconstruction. The
  core of LRM-Zero is our procedural 3D dataset, Zeroverse, which is
  automatically synthesized from simple primitive shapes with random texturing
  and augmentations (e.g., height fields, boolean differences, and wireframes).
  Unlike previous 3D datasets (e.g., Objaverse) which are often captured or
  crafted by humans to approximate real 3D data, Zeroverse completely ignores
  realistic global semantics but is rich in complex geometric and texture details
  that are locally similar to or even more intricate than real objects. We
  demonstrate that our LRM-Zero, trained with our fully synthesized Zeroverse,
  can achieve high visual quality in the reconstruction of real-world objects,
  competitive with models trained on Objaverse. We also analyze several critical
  design choices of Zeroverse that contribute to LRM-Zero's capability and
  training stability. Our work demonstrates that 3D reconstruction, one of the
  core tasks in 3D vision, can potentially be addressed without the semantics of
  real-world objects. The Zeroverse's procedural synthesis code and interactive
  visualization are available at: https://desaixie.github.io/lrm-zero/.
paper_ai_summary: LRM-Zero, a 3D reconstruction model trained on synthesized data, achieves high visual quality comparable to models using real-world datasets.
paper_ai_keywords: Large Reconstruction Model 3D reconstruction procedural 3D dataset Zeroverse primitive shapes random texturing augmentations height fields boolean differences wireframes Objaverse 3D vision semantic semantics procedural synthesis