Uni-3DAR: Unified 3D Generation and Understanding via Autoregression on
  Compressed Spatial Tokens
Recent advancements in large language models and their multi-modal extensions
have demonstrated the effectiveness of unifying generation and understanding
through autoregressive next-token prediction. However, despite the critical
role of 3D structural generation and understanding ({3D GU}) in AI for science,
these tasks have largely evolved independently, with autoregressive methods
remaining underexplored. To bridge this gap, we introduce Uni-3DAR, a unified
framework that seamlessly integrates {3D GU} tasks via autoregressive
prediction. At its core, Uni-3DAR employs a novel hierarchical tokenization
that compresses 3D space using an octree, leveraging the inherent sparsity of
3D structures. It then applies an additional tokenization for fine-grained
structural details, capturing key attributes such as atom types and precise
spatial coordinates in microscopic 3D structures. We further propose two
optimizations to enhance efficiency and effectiveness. The first is a two-level
subtree compression strategy, which reduces the octree token sequence by up to
8x. The second is a masked next-token prediction mechanism tailored for
dynamically varying token positions, significantly boosting model performance.
By combining these strategies, Uni-3DAR successfully unifies diverse {3D GU}
tasks within a single autoregressive framework. Extensive experiments across
multiple microscopic {3D GU} tasks, including molecules, proteins, polymers,
and crystals, validate its effectiveness and versatility. Notably, Uni-3DAR
surpasses previous state-of-the-art diffusion models by a substantial margin,
achieving up to 256\% relative improvement while delivering inference speeds up
to 21.8x faster. The code is publicly available at
https://github.com/dptech-corp/Uni-3DAR.
67dcc98b54dcfdc1fd17d9b6
Shuqi Lu
67dcc98b54dcfdc1fd17d9b7
Haowei Lin
claimed_verified
2025-04-08T06:57:39.558Z
63453f02a05b51f7ded3c579
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/u7p-wTj8zyeWpSFRI8Go0.png
Andy Lin
user
pkuHaowei
67dcc98b54dcfdc1fd17d9b8
Lin Yao
67dcc98b54dcfdc1fd17d9b9
Zhifeng Gao
67dcc98b54dcfdc1fd17d9ba
Xiaohong Ji
67dcc98b54dcfdc1fd17d9bb
Weinan E
67dcc98b54dcfdc1fd17d9bc
Linfeng Zhang
67dcc98b54dcfdc1fd17d9bd
Guolin Ke
claimed_verified
2025-03-21T11:41:08.637Z
6348de0c62c668c7b48d83c9
/avatars/7296ea9bb301e19c10926022959b2023.svg
Guolin Ke
user
guolinke
Uni-3DAR: Unified 3D Generation and Understanding via Autoregression on
  Compressed Spatial Tokens
Recent advancements in large language models and their multi-modal extensions
have demonstrated the effectiveness of unifying generation and understanding
through autoregressive next-token prediction. However, despite the critical
role of 3D structural generation and understanding ({3D GU}) in AI for science,
these tasks have largely evolved independently, with autoregressive methods
remaining underexplored. To bridge this gap, we introduce Uni-3DAR, a unified
framework that seamlessly integrates {3D GU} tasks via autoregressive
prediction. At its core, Uni-3DAR employs a novel hierarchical tokenization
that compresses 3D space using an octree, leveraging the inherent sparsity of
3D structures. It then applies an additional tokenization for fine-grained
structural details, capturing key attributes such as atom types and precise
spatial coordinates in microscopic 3D structures. We further propose two
optimizations to enhance efficiency and effectiveness. The first is a two-level
subtree compression strategy, which reduces the octree token sequence by up to
8x. The second is a masked next-token prediction mechanism tailored for
dynamically varying token positions, significantly boosting model performance.
By combining these strategies, Uni-3DAR successfully unifies diverse {3D GU}
tasks within a single autoregressive framework. Extensive experiments across
multiple microscopic {3D GU} tasks, including molecules, proteins, polymers,
and crystals, validate its effectiveness and versatility. Notably, Uni-3DAR
surpasses previous state-of-the-art diffusion models by a substantial margin,
achieving up to 256\% relative improvement while delivering inference speeds up
to 21.8x faster. The code is publicly available at
https://github.com/dptech-corp/Uni-3DAR.
Uni-3DAR is a unified autoregressive framework for 3D structural generation and understanding, using hierarchical tokenization and optimizations to outperform diffusion models.
unified framework 3D GU autoregressive next-token prediction hierarchical tokenization octree subtree compression masked next-token prediction diffusion models