title: AToM: Amortized Text-to-Mesh using 2D Diffusion
summary: We introduce Amortized Text-to-Mesh (AToM), a feed-forward text-to-mesh
  framework optimized across multiple text prompts simultaneously. In contrast to
  existing text-to-3D methods that often entail time-consuming per-prompt
  optimization and commonly output representations other than polygonal meshes,
  AToM directly generates high-quality textured meshes in less than 1 second with
  around 10 times reduction in the training cost, and generalizes to unseen
  prompts. Our key idea is a novel triplane-based text-to-mesh architecture with
  a two-stage amortized optimization strategy that ensures stable training and
  enables scalability. Through extensive experiments on various prompt
  benchmarks, AToM significantly outperforms state-of-the-art amortized
  approaches with over 4 times higher accuracy (in DF415 dataset) and produces
  more distinguishable and higher-quality 3D outputs. AToM demonstrates strong
  generalizability, offering finegrained 3D assets for unseen interpolated
  prompts without further optimization during inference, unlike per-prompt
  solutions.
paper_authors._id: 65bc734d4b5f8c270d7aaac8
paper_authors.name: Guocheng Qian
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T13:00:55.810Z
paper_authors._id: 645fed74335c21d19f3bf76c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/645fed74335c21d19f3bf76c/gwVsllRWtSHbg4a1erkdF.jpeg
paper_authors.fullname: Guocheng Qian
paper_authors.type: user
paper_authors.user: guochengqian
paper_authors._id: 65bc734d4b5f8c270d7aaac9
paper_authors.name: Junli Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T13:01:04.102Z
paper_authors._id: 63f54aa73aa49d8cb97b84bc
paper_authors.avatarUrl: /avatars/c73c5870039611ab9162daad46a1ba20.svg
paper_authors.fullname: junli cao
paper_authors.type: user
paper_authors.user: jlcao2
paper_authors._id: 65bc734d4b5f8c270d7aaaca
paper_authors.name: Aliaksandr Siarohin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T13:01:13.483Z
paper_authors._id: 64276311eb9a0ed86180715b
paper_authors.avatarUrl: /avatars/76f933cd549f10e5e2db379de235d304.svg
paper_authors.fullname: Aliaksandr Siarohin
paper_authors.type: user
paper_authors.user: aliaksandr-siarohin
paper_authors._id: 65bc734d4b5f8c270d7aaacb
paper_authors.name: Yash Kant
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T13:19:16.013Z
paper_authors._id: 638546d0a179f856005ae310
paper_authors.avatarUrl: /avatars/f7f2fface336c8e168a1daaf9fd4d40c.svg
paper_authors.fullname: yashkant
paper_authors.type: user
paper_authors.user: yashkant
paper_authors._id: 65bc734d4b5f8c270d7aaacc
paper_authors.name: Chaoyang Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T13:20:50.472Z
paper_authors._id: 64c166de11f3f1d23a99c6fd
paper_authors.avatarUrl: /avatars/d4730f70a6f1cb51266862c7a8d54f77.svg
paper_authors.fullname: Chaoyang Wang
paper_authors.type: user
paper_authors.user: cwang9
paper_authors._id: 65bc734d4b5f8c270d7aaacd
paper_authors.name: Michael Vasilkovsky
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T13:20:56.666Z
paper_authors._id: 64775c34dbc2a416f8b648f6
paper_authors.avatarUrl: /avatars/014b7011e0bd2e8ab6fa91a71ddb2d60.svg
paper_authors.fullname: Michael Vasilkovsky
paper_authors.type: user
paper_authors.user: waytobehigh
paper_authors._id: 65bc734d4b5f8c270d7aaace
paper_authors.name: Hsin-Ying Lee
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T13:21:03.325Z
paper_authors._id: 631bbd918de8e645af71eed9
paper_authors.avatarUrl: /avatars/1cb94deadc288d07c571c9289085548c.svg
paper_authors.fullname: Hsin-Ying Lee
paper_authors.type: user
paper_authors.user: james371507
paper_authors._id: 65bc734d4b5f8c270d7aaacf
paper_authors.name: Yuwei Fang
paper_authors._id: 65bc734d4b5f8c270d7aaad0
paper_authors.name: Ivan Skorokhodov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T13:21:31.381Z
paper_authors._id: 63610db13c7147fae7de88e3
paper_authors.avatarUrl: /avatars/d7e97a16cfee39e1e50d7a5b747876f1.svg
paper_authors.fullname: Ivan Skorokhodov
paper_authors.type: user
paper_authors.user: universome
paper_authors._id: 65bc734d4b5f8c270d7aaad1
paper_authors.name: Peiye Zhuang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T13:21:37.853Z
paper_authors._id: 63e473cf9db5da2dc1f4e9ff
paper_authors.avatarUrl: /avatars/6973b4c959ef8bb72c40c611d384a78a.svg
paper_authors.fullname: peiye zhuang
paper_authors.type: user
paper_authors.user: Kelest
paper_authors._id: 65bc734d4b5f8c270d7aaad2
paper_authors.name: Igor Gilitschenski
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T13:21:43.989Z
paper_authors._id: 64893f0b9ab28735f1a1ec0b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64893f0b9ab28735f1a1ec0b/2lPBjV16l2qLEWOKU2zwR.jpeg
paper_authors.fullname: Igor Gilitschenski
paper_authors.type: user
paper_authors.user: igilitschenski
paper_authors._id: 65bc734d4b5f8c270d7aaad3
paper_authors.name: Jian Ren
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-03T10:48:24.734Z
paper_authors._id: 61f19829233c91cbd2f79e70
paper_authors.avatarUrl: /avatars/a0735a94542b4f7cda5aed8bc4be0538.svg
paper_authors.fullname: Jian Ren
paper_authors.type: user
paper_authors.user: alanspike
paper_authors._id: 65bc734d4b5f8c270d7aaad4
paper_authors.name: Bernard Ghanem
paper_authors._id: 65bc734d4b5f8c270d7aaad5
paper_authors.name: Kfir Aberman
paper_authors._id: 65bc734d4b5f8c270d7aaad6
paper_authors.name: Sergey Tulyakov
paper_title: AToM: Amortized Text-to-Mesh using 2D Diffusion
paper_summary: We introduce Amortized Text-to-Mesh (AToM), a feed-forward text-to-mesh
  framework optimized across multiple text prompts simultaneously. In contrast to
  existing text-to-3D methods that often entail time-consuming per-prompt
  optimization and commonly output representations other than polygonal meshes,
  AToM directly generates high-quality textured meshes in less than 1 second with
  around 10 times reduction in the training cost, and generalizes to unseen
  prompts. Our key idea is a novel triplane-based text-to-mesh architecture with
  a two-stage amortized optimization strategy that ensures stable training and
  enables scalability. Through extensive experiments on various prompt
  benchmarks, AToM significantly outperforms state-of-the-art amortized
  approaches with over 4 times higher accuracy (in DF415 dataset) and produces
  more distinguishable and higher-quality 3D outputs. AToM demonstrates strong
  generalizability, offering finegrained 3D assets for unseen interpolated
  prompts without further optimization during inference, unlike per-prompt
  solutions.
paper_ai_keywords: feed-forward text-to-mesh triplane-based text-to-mesh two-stage amortized optimization polygonal meshes training cost prompt benchmarks DF415 dataset 3D outputs generalizability finegrained 3D assets inference