FEAT: Full-Dimensional Efficient Attention Transformer for Medical Video
  Generation
Synthesizing high-quality dynamic medical videos remains a significant
challenge due to the need for modeling both spatial consistency and temporal
dynamics. Existing Transformer-based approaches face critical limitations,
including insufficient channel interactions, high computational complexity from
self-attention, and coarse denoising guidance from timestep embeddings when
handling varying noise levels. In this work, we propose FEAT, a
full-dimensional efficient attention Transformer, which addresses these issues
through three key innovations: (1) a unified paradigm with sequential
spatial-temporal-channel attention mechanisms to capture global dependencies
across all dimensions, (2) a linear-complexity design for attention mechanisms
in each dimension, utilizing weighted key-value attention and global channel
attention, and (3) a residual value guidance module that provides fine-grained
pixel-level guidance to adapt to different noise levels. We evaluate FEAT on
standard benchmarks and downstream tasks, demonstrating that FEAT-S, with only
23\% of the parameters of the state-of-the-art model Endora, achieves
comparable or even superior performance. Furthermore, FEAT-L surpasses all
comparison methods across multiple datasets, showcasing both superior
effectiveness and scalability. Code is available at
https://github.com/Yaziwel/FEAT.
6842e0874c601e90572c78a4
Huihan Wang
claimed_verified
2025-06-07T05:48:21.634Z
6842d2f8668a1d103cca0b2b
/avatars/bc61242617336ddf3e327534b396a2ce.svg
Huihan Wang
user
WTHH031230
6842e0874c601e90572c78a5
Zhiwen Yang
claimed_verified
2025-06-07T05:48:24.584Z
638a2bdb34cf0480e9abd6a4
/avatars/70e84c5c188688db8b29455061a410a1.svg
Zhiwen Yang
user
upyzwup
6842e0874c601e90572c78a6
Hui Zhang
6842e0874c601e90572c78a7
Dan Zhao
6842e0874c601e90572c78a8
Bingzheng Wei
6842e0874c601e90572c78a9
Yan Xu
FEAT: Full-Dimensional Efficient Attention Transformer for Medical Video
  Generation
Synthesizing high-quality dynamic medical videos remains a significant
challenge due to the need for modeling both spatial consistency and temporal
dynamics. Existing Transformer-based approaches face critical limitations,
including insufficient channel interactions, high computational complexity from
self-attention, and coarse denoising guidance from timestep embeddings when
handling varying noise levels. In this work, we propose FEAT, a
full-dimensional efficient attention Transformer, which addresses these issues
through three key innovations: (1) a unified paradigm with sequential
spatial-temporal-channel attention mechanisms to capture global dependencies
across all dimensions, (2) a linear-complexity design for attention mechanisms
in each dimension, utilizing weighted key-value attention and global channel
attention, and (3) a residual value guidance module that provides fine-grained
pixel-level guidance to adapt to different noise levels. We evaluate FEAT on
standard benchmarks and downstream tasks, demonstrating that FEAT-S, with only
23\% of the parameters of the state-of-the-art model Endora, achieves
comparable or even superior performance. Furthermore, FEAT-L surpasses all
comparison methods across multiple datasets, showcasing both superior
effectiveness and scalability. Code is available at
https://github.com/Yaziwel/FEAT.
FEAT, a full-dimensional efficient attention Transformer, addresses challenges in synthesizing high-quality dynamic medical videos by improving channel interactions, reducing computational complexity, and enhancing denoising guidance.
Transformer-based approaches sequential spatial-temporal-channel attention mechanisms linear-complexity design weighted key-value attention global channel attention residual value guidance module