AToM: Amortized Text-to-Mesh using 2D Diffusion
We introduce Amortized Text-to-Mesh (AToM), a feed-forward text-to-mesh
framework optimized across multiple text prompts simultaneously. In contrast to
existing text-to-3D methods that often entail time-consuming per-prompt
optimization and commonly output representations other than polygonal meshes,
AToM directly generates high-quality textured meshes in less than 1 second with
around 10 times reduction in the training cost, and generalizes to unseen
prompts. Our key idea is a novel triplane-based text-to-mesh architecture with
a two-stage amortized optimization strategy that ensures stable training and
enables scalability. Through extensive experiments on various prompt
benchmarks, AToM significantly outperforms state-of-the-art amortized
approaches with over 4 times higher accuracy (in DF415 dataset) and produces
more distinguishable and higher-quality 3D outputs. AToM demonstrates strong
generalizability, offering finegrained 3D assets for unseen interpolated
prompts without further optimization during inference, unlike per-prompt
solutions.
65bc734d4b5f8c270d7aaac8
Guocheng Qian
admin_assigned
2024-02-02T13:00:55.810Z
645fed74335c21d19f3bf76c
https://cdn-avatars.huggingface.co/v1/production/uploads/645fed74335c21d19f3bf76c/gwVsllRWtSHbg4a1erkdF.jpeg
Guocheng Qian
user
guochengqian
65bc734d4b5f8c270d7aaac9
Junli Cao
admin_assigned
2024-02-02T13:01:04.102Z
63f54aa73aa49d8cb97b84bc
/avatars/c73c5870039611ab9162daad46a1ba20.svg
junli cao
user
jlcao2
65bc734d4b5f8c270d7aaaca
Aliaksandr Siarohin
admin_assigned
2024-02-02T13:01:13.483Z
64276311eb9a0ed86180715b
/avatars/76f933cd549f10e5e2db379de235d304.svg
Aliaksandr Siarohin
user
aliaksandr-siarohin
65bc734d4b5f8c270d7aaacb
Yash Kant
admin_assigned
2024-02-02T13:19:16.013Z
638546d0a179f856005ae310
/avatars/f7f2fface336c8e168a1daaf9fd4d40c.svg
yashkant
user
yashkant
65bc734d4b5f8c270d7aaacc
Chaoyang Wang
admin_assigned
2024-02-02T13:20:50.472Z
64c166de11f3f1d23a99c6fd
/avatars/d4730f70a6f1cb51266862c7a8d54f77.svg
Chaoyang Wang
user
cwang9
65bc734d4b5f8c270d7aaacd
Michael Vasilkovsky
admin_assigned
2024-02-02T13:20:56.666Z
64775c34dbc2a416f8b648f6
/avatars/014b7011e0bd2e8ab6fa91a71ddb2d60.svg
Michael Vasilkovsky
user
waytobehigh
65bc734d4b5f8c270d7aaace
Hsin-Ying Lee
admin_assigned
2024-02-02T13:21:03.325Z
631bbd918de8e645af71eed9
/avatars/1cb94deadc288d07c571c9289085548c.svg
Hsin-Ying Lee
user
james371507
65bc734d4b5f8c270d7aaacf
Yuwei Fang
65bc734d4b5f8c270d7aaad0
Ivan Skorokhodov
admin_assigned
2024-02-02T13:21:31.381Z
63610db13c7147fae7de88e3
/avatars/d7e97a16cfee39e1e50d7a5b747876f1.svg
Ivan Skorokhodov
user
universome
65bc734d4b5f8c270d7aaad1
Peiye Zhuang
admin_assigned
2024-02-02T13:21:37.853Z
63e473cf9db5da2dc1f4e9ff
/avatars/6973b4c959ef8bb72c40c611d384a78a.svg
peiye zhuang
user
Kelest
65bc734d4b5f8c270d7aaad2
Igor Gilitschenski
admin_assigned
2024-02-02T13:21:43.989Z
64893f0b9ab28735f1a1ec0b
https://cdn-avatars.huggingface.co/v1/production/uploads/64893f0b9ab28735f1a1ec0b/2lPBjV16l2qLEWOKU2zwR.jpeg
Igor Gilitschenski
user
igilitschenski
65bc734d4b5f8c270d7aaad3
Jian Ren
claimed_verified
2024-02-03T10:48:24.734Z
61f19829233c91cbd2f79e70
/avatars/a0735a94542b4f7cda5aed8bc4be0538.svg
Jian Ren
user
alanspike
65bc734d4b5f8c270d7aaad4
Bernard Ghanem
65bc734d4b5f8c270d7aaad5
Kfir Aberman
65bc734d4b5f8c270d7aaad6
Sergey Tulyakov
AToM: Amortized Text-to-Mesh using 2D Diffusion
We introduce Amortized Text-to-Mesh (AToM), a feed-forward text-to-mesh
framework optimized across multiple text prompts simultaneously. In contrast to
existing text-to-3D methods that often entail time-consuming per-prompt
optimization and commonly output representations other than polygonal meshes,
AToM directly generates high-quality textured meshes in less than 1 second with
around 10 times reduction in the training cost, and generalizes to unseen
prompts. Our key idea is a novel triplane-based text-to-mesh architecture with
a two-stage amortized optimization strategy that ensures stable training and
enables scalability. Through extensive experiments on various prompt
benchmarks, AToM significantly outperforms state-of-the-art amortized
approaches with over 4 times higher accuracy (in DF415 dataset) and produces
more distinguishable and higher-quality 3D outputs. AToM demonstrates strong
generalizability, offering finegrained 3D assets for unseen interpolated
prompts without further optimization during inference, unlike per-prompt
solutions.
Amortized Text-to-Mesh (AToM) generates high-quality textured meshes efficiently from text prompts using a novel triplane-based architecture and achieves superior accuracy and generalization compared to existing methods.
feed-forward text-to-mesh triplane-based architecture amortized optimization polygonal meshes DF415 dataset