title: Distilling semantically aware orders for autoregressive image generation
summary: Autoregressive patch-based image generation has recently shown competitive
  results in terms of image quality and scalability. It can also be easily
  integrated and scaled within Vision-Language models. Nevertheless,
  autoregressive models require a defined order for patch generation. While a
  natural order based on the dictation of the words makes sense for text
  generation, there is no inherent generation order that exists for image
  generation. Traditionally, a raster-scan order (from top-left to bottom-right)
  guides autoregressive image generation models. In this paper, we argue that
  this order is suboptimal, as it fails to respect the causality of the image
  content: for instance, when conditioned on a visual description of a sunset, an
  autoregressive model may generate clouds before the sun, even though the color
  of clouds should depend on the color of the sun and not the inverse. In this
  work, we show that first by training a model to generate patches in
  any-given-order, we can infer both the content and the location (order) of each
  patch during generation. Secondly, we use these extracted orders to finetune
  the any-given-order model to produce better-quality images. Through our
  experiments, we show on two datasets that this new generation method produces
  better images than the traditional raster-scan approach, with similar training
  costs and no extra annotations.
paper_authors._id: 680b1b33388bb2cfd497ebdb
paper_authors.name: Rishav Pramanik
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-25T08:34:27.724Z
paper_authors._id: 62bb84f82ada492aa5775709
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62bb84f82ada492aa5775709/jv4yKL75t8QzHDHLbhBPT.png
paper_authors.fullname: Rishav Pramanik
paper_authors.type: user
paper_authors.user: rishavpramanik
paper_authors._id: 680b1b33388bb2cfd497ebdc
paper_authors.name: Antoine Poupon
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-26T08:53:34.585Z
paper_authors._id: 65bff50d5604c1d383d23ade
paper_authors.avatarUrl: /avatars/92982635885c46e90d4966245d066ef6.svg
paper_authors.fullname: Antoine Poupon
paper_authors.type: user
paper_authors.user: apoupon
paper_authors._id: 680b1b33388bb2cfd497ebdd
paper_authors.name: Juan A. Rodriguez
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-25T12:27:17.091Z
paper_authors._id: 63a614d264f470027818b066
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63a614d264f470027818b066/Q5Eih2VqD4NaHB_MkNp32.jpeg
paper_authors.fullname: Juan A. Rodriguez
paper_authors.type: user
paper_authors.user: joanrodai
paper_authors._id: 680b1b33388bb2cfd497ebde
paper_authors.name: Masih Aminbeidokhti
paper_authors._id: 680b1b33388bb2cfd497ebdf
paper_authors.name: David Vazquez
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-08T08:59:53.443Z
paper_authors._id: 62f6691f329d4d014d1b4087
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1660315866736-noauth.jpeg
paper_authors.fullname: David Vazquez
paper_authors.type: user
paper_authors.user: dvazquez
paper_authors._id: 680b1b33388bb2cfd497ebe0
paper_authors.name: Christopher Pal
paper_authors._id: 680b1b33388bb2cfd497ebe1
paper_authors.name: Zhaozheng Yin
paper_authors._id: 680b1b33388bb2cfd497ebe2
paper_authors.name: Marco Pedersoli
paper_title: Distilling semantically aware orders for autoregressive image generation
paper_summary: Autoregressive patch-based image generation has recently shown competitive
  results in terms of image quality and scalability. It can also be easily
  integrated and scaled within Vision-Language models. Nevertheless,
  autoregressive models require a defined order for patch generation. While a
  natural order based on the dictation of the words makes sense for text
  generation, there is no inherent generation order that exists for image
  generation. Traditionally, a raster-scan order (from top-left to bottom-right)
  guides autoregressive image generation models. In this paper, we argue that
  this order is suboptimal, as it fails to respect the causality of the image
  content: for instance, when conditioned on a visual description of a sunset, an
  autoregressive model may generate clouds before the sun, even though the color
  of clouds should depend on the color of the sun and not the inverse. In this
  work, we show that first by training a model to generate patches in
  any-given-order, we can infer both the content and the location (order) of each
  patch during generation. Secondly, we use these extracted orders to finetune
  the any-given-order model to produce better-quality images. Through our
  experiments, we show on two datasets that this new generation method produces
  better images than the traditional raster-scan approach, with similar training
  costs and no extra annotations.
paper_ai_summary: A new method for autoregressive image generation infers patch order during generation, improving image quality over traditional raster-scan approaches without extra annotations.
paper_ai_keywords: autoregressive patch-based image generation Vision-Language models raster-scan order causality of image content any-given-order model patch generation