title: Equivariant Image Modeling
summary: Current generative models, such as autoregressive and diffusion approaches,
  decompose high-dimensional data distribution learning into a series of simpler
  subtasks. However, inherent conflicts arise during the joint optimization of
  these subtasks, and existing solutions fail to resolve such conflicts without
  sacrificing efficiency or scalability. We propose a novel equivariant image
  modeling framework that inherently aligns optimization targets across subtasks
  by leveraging the translation invariance of natural visual signals. Our method
  introduces (1) column-wise tokenization which enhances translational symmetry
  along the horizontal axis, and (2) windowed causal attention which enforces
  consistent contextual relationships across positions. Evaluated on
  class-conditioned ImageNet generation at 256x256 resolution, our approach
  achieves performance comparable to state-of-the-art AR models while using fewer
  computational resources. Systematic analysis demonstrates that enhanced
  equivariance reduces inter-task conflicts, significantly improving zero-shot
  generalization and enabling ultra-long image synthesis. This work establishes
  the first framework for task-aligned decomposition in generative modeling,
  offering insights into efficient parameter sharing and conflict-free
  optimization. The code and models are publicly available at
  https://github.com/drx-code/EquivariantModeling.
paper_authors._id: 67e24217db11e1d382285cd4
paper_authors.name: Ruixiao Dong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-25T09:20:32.045Z
paper_authors._id: 6447a5806ffed6ece1fcf723
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/NjOA7G_QCa3bCluA69hSs.jpeg
paper_authors.fullname: Ruixiao Dong
paper_authors.type: user
paper_authors.user: dongruixiao
paper_authors._id: 67e24217db11e1d382285cd5
paper_authors.name: Mengde Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-25T09:20:25.156Z
paper_authors._id: 63f5993afcf95ecac2b419b5
paper_authors.avatarUrl: /avatars/a8c020080a84d9a663789c4fb19270e9.svg
paper_authors.fullname: Mengde Xu
paper_authors.type: user
paper_authors.user: Mendel192
paper_authors._id: 67e24217db11e1d382285cd6
paper_authors.name: Zigang Geng
paper_authors._id: 67e24217db11e1d382285cd7
paper_authors.name: Li Li
paper_authors._id: 67e24217db11e1d382285cd8
paper_authors.name: Han Hu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-25T09:20:10.271Z
paper_authors._id: 665d88640e92f92b0e7eb17f
paper_authors.avatarUrl: /avatars/ff3a410e1e7bfb00ff0ec8ce4d5b1463.svg
paper_authors.fullname: han hu
paper_authors.type: user
paper_authors.user: hanhu2
paper_authors._id: 67e24217db11e1d382285cd9
paper_authors.name: Shuyang Gu
paper_title: Equivariant Image Modeling
paper_summary: Current generative models, such as autoregressive and diffusion approaches,
  decompose high-dimensional data distribution learning into a series of simpler
  subtasks. However, inherent conflicts arise during the joint optimization of
  these subtasks, and existing solutions fail to resolve such conflicts without
  sacrificing efficiency or scalability. We propose a novel equivariant image
  modeling framework that inherently aligns optimization targets across subtasks
  by leveraging the translation invariance of natural visual signals. Our method
  introduces (1) column-wise tokenization which enhances translational symmetry
  along the horizontal axis, and (2) windowed causal attention which enforces
  consistent contextual relationships across positions. Evaluated on
  class-conditioned ImageNet generation at 256x256 resolution, our approach
  achieves performance comparable to state-of-the-art AR models while using fewer
  computational resources. Systematic analysis demonstrates that enhanced
  equivariance reduces inter-task conflicts, significantly improving zero-shot
  generalization and enabling ultra-long image synthesis. This work establishes
  the first framework for task-aligned decomposition in generative modeling,
  offering insights into efficient parameter sharing and conflict-free
  optimization. The code and models are publicly available at
  https://github.com/drx-code/EquivariantModeling.
paper_ai_summary: A novel equivariant image modeling framework enhances generative performance by leveraging column-wise tokenization and windowed causal attention, improving efficiency, scalability, and zero-shot generalization.
paper_ai_keywords: equivariant image modeling column-wise tokenization windowed causal attention generative modeling autoregressive models diffusion approaches zero-shot generalization