title: Uni-3DAR: Unified 3D Generation and Understanding via Autoregression on
    Compressed Spatial Tokens
summary: Recent advancements in large language models and their multi-modal extensions
  have demonstrated the effectiveness of unifying generation and understanding
  through autoregressive next-token prediction. However, despite the critical
  role of 3D structural generation and understanding ({3D GU}) in AI for science,
  these tasks have largely evolved independently, with autoregressive methods
  remaining underexplored. To bridge this gap, we introduce Uni-3DAR, a unified
  framework that seamlessly integrates {3D GU} tasks via autoregressive
  prediction. At its core, Uni-3DAR employs a novel hierarchical tokenization
  that compresses 3D space using an octree, leveraging the inherent sparsity of
  3D structures. It then applies an additional tokenization for fine-grained
  structural details, capturing key attributes such as atom types and precise
  spatial coordinates in microscopic 3D structures. We further propose two
  optimizations to enhance efficiency and effectiveness. The first is a two-level
  subtree compression strategy, which reduces the octree token sequence by up to
  8x. The second is a masked next-token prediction mechanism tailored for
  dynamically varying token positions, significantly boosting model performance.
  By combining these strategies, Uni-3DAR successfully unifies diverse {3D GU}
  tasks within a single autoregressive framework. Extensive experiments across
  multiple microscopic {3D GU} tasks, including molecules, proteins, polymers,
  and crystals, validate its effectiveness and versatility. Notably, Uni-3DAR
  surpasses previous state-of-the-art diffusion models by a substantial margin,
  achieving up to 256\% relative improvement while delivering inference speeds up
  to 21.8x faster. The code is publicly available at
  https://github.com/dptech-corp/Uni-3DAR.
paper_authors._id: 67dcc98b54dcfdc1fd17d9b6
paper_authors.name: Shuqi Lu
paper_authors._id: 67dcc98b54dcfdc1fd17d9b7
paper_authors.name: Haowei Lin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-08T06:57:39.558Z
paper_authors._id: 63453f02a05b51f7ded3c579
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/u7p-wTj8zyeWpSFRI8Go0.png
paper_authors.fullname: Andy Lin
paper_authors.type: user
paper_authors.user: pkuHaowei
paper_authors._id: 67dcc98b54dcfdc1fd17d9b8
paper_authors.name: Lin Yao
paper_authors._id: 67dcc98b54dcfdc1fd17d9b9
paper_authors.name: Zhifeng Gao
paper_authors._id: 67dcc98b54dcfdc1fd17d9ba
paper_authors.name: Xiaohong Ji
paper_authors._id: 67dcc98b54dcfdc1fd17d9bb
paper_authors.name: Weinan E
paper_authors._id: 67dcc98b54dcfdc1fd17d9bc
paper_authors.name: Linfeng Zhang
paper_authors._id: 67dcc98b54dcfdc1fd17d9bd
paper_authors.name: Guolin Ke
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-21T11:41:08.637Z
paper_authors._id: 6348de0c62c668c7b48d83c9
paper_authors.avatarUrl: /avatars/7296ea9bb301e19c10926022959b2023.svg
paper_authors.fullname: Guolin Ke
paper_authors.type: user
paper_authors.user: guolinke
paper_title: Uni-3DAR: Unified 3D Generation and Understanding via Autoregression on
    Compressed Spatial Tokens
paper_summary: Recent advancements in large language models and their multi-modal extensions
  have demonstrated the effectiveness of unifying generation and understanding
  through autoregressive next-token prediction. However, despite the critical
  role of 3D structural generation and understanding ({3D GU}) in AI for science,
  these tasks have largely evolved independently, with autoregressive methods
  remaining underexplored. To bridge this gap, we introduce Uni-3DAR, a unified
  framework that seamlessly integrates {3D GU} tasks via autoregressive
  prediction. At its core, Uni-3DAR employs a novel hierarchical tokenization
  that compresses 3D space using an octree, leveraging the inherent sparsity of
  3D structures. It then applies an additional tokenization for fine-grained
  structural details, capturing key attributes such as atom types and precise
  spatial coordinates in microscopic 3D structures. We further propose two
  optimizations to enhance efficiency and effectiveness. The first is a two-level
  subtree compression strategy, which reduces the octree token sequence by up to
  8x. The second is a masked next-token prediction mechanism tailored for
  dynamically varying token positions, significantly boosting model performance.
  By combining these strategies, Uni-3DAR successfully unifies diverse {3D GU}
  tasks within a single autoregressive framework. Extensive experiments across
  multiple microscopic {3D GU} tasks, including molecules, proteins, polymers,
  and crystals, validate its effectiveness and versatility. Notably, Uni-3DAR
  surpasses previous state-of-the-art diffusion models by a substantial margin,
  achieving up to 256\% relative improvement while delivering inference speeds up
  to 21.8x faster. The code is publicly available at
  https://github.com/dptech-corp/Uni-3DAR.
paper_ai_summary: Uni-3DAR is a unified autoregressive framework for 3D structural generation and understanding, using hierarchical tokenization and optimizations to outperform diffusion models.
paper_ai_keywords: unified framework 3D GU autoregressive next-token prediction hierarchical tokenization octree subtree compression masked next-token prediction diffusion models