Equivariant Image Modeling
Current generative models, such as autoregressive and diffusion approaches,
decompose high-dimensional data distribution learning into a series of simpler
subtasks. However, inherent conflicts arise during the joint optimization of
these subtasks, and existing solutions fail to resolve such conflicts without
sacrificing efficiency or scalability. We propose a novel equivariant image
modeling framework that inherently aligns optimization targets across subtasks
by leveraging the translation invariance of natural visual signals. Our method
introduces (1) column-wise tokenization which enhances translational symmetry
along the horizontal axis, and (2) windowed causal attention which enforces
consistent contextual relationships across positions. Evaluated on
class-conditioned ImageNet generation at 256x256 resolution, our approach
achieves performance comparable to state-of-the-art AR models while using fewer
computational resources. Systematic analysis demonstrates that enhanced
equivariance reduces inter-task conflicts, significantly improving zero-shot
generalization and enabling ultra-long image synthesis. This work establishes
the first framework for task-aligned decomposition in generative modeling,
offering insights into efficient parameter sharing and conflict-free
optimization. The code and models are publicly available at
https://github.com/drx-code/EquivariantModeling.
67e24217db11e1d382285cd4
Ruixiao Dong
admin_assigned
2025-03-25T09:20:32.045Z
6447a5806ffed6ece1fcf723
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/NjOA7G_QCa3bCluA69hSs.jpeg
Ruixiao Dong
user
dongruixiao
67e24217db11e1d382285cd5
Mengde Xu
admin_assigned
2025-03-25T09:20:25.156Z
63f5993afcf95ecac2b419b5
/avatars/a8c020080a84d9a663789c4fb19270e9.svg
Mengde Xu
user
Mendel192
67e24217db11e1d382285cd6
Zigang Geng
67e24217db11e1d382285cd7
Li Li
67e24217db11e1d382285cd8
Han Hu
admin_assigned
2025-03-25T09:20:10.271Z
665d88640e92f92b0e7eb17f
/avatars/ff3a410e1e7bfb00ff0ec8ce4d5b1463.svg
han hu
user
hanhu2
67e24217db11e1d382285cd9
Shuyang Gu
Equivariant Image Modeling
Current generative models, such as autoregressive and diffusion approaches,
decompose high-dimensional data distribution learning into a series of simpler
subtasks. However, inherent conflicts arise during the joint optimization of
these subtasks, and existing solutions fail to resolve such conflicts without
sacrificing efficiency or scalability. We propose a novel equivariant image
modeling framework that inherently aligns optimization targets across subtasks
by leveraging the translation invariance of natural visual signals. Our method
introduces (1) column-wise tokenization which enhances translational symmetry
along the horizontal axis, and (2) windowed causal attention which enforces
consistent contextual relationships across positions. Evaluated on
class-conditioned ImageNet generation at 256x256 resolution, our approach
achieves performance comparable to state-of-the-art AR models while using fewer
computational resources. Systematic analysis demonstrates that enhanced
equivariance reduces inter-task conflicts, significantly improving zero-shot
generalization and enabling ultra-long image synthesis. This work establishes
the first framework for task-aligned decomposition in generative modeling,
offering insights into efficient parameter sharing and conflict-free
optimization. The code and models are publicly available at
https://github.com/drx-code/EquivariantModeling.
A novel equivariant image modeling framework enhances generative performance by leveraging column-wise tokenization and windowed causal attention, improving efficiency, scalability, and zero-shot generalization.
equivariant image modeling column-wise tokenization windowed causal attention generative modeling autoregressive models diffusion approaches zero-shot generalization