title: Meissonic: Revitalizing Masked Generative Transformers for Efficient
    High-Resolution Text-to-Image Synthesis
summary: Diffusion models, such as Stable Diffusion, have made significant strides in
  visual generation, yet their paradigm remains fundamentally different from
  autoregressive language models, complicating the development of unified
  language-vision models. Recent efforts like LlamaGen have attempted
  autoregressive image generation using discrete VQVAE tokens, but the large
  number of tokens involved renders this approach inefficient and slow. In this
  work, we present Meissonic, which elevates non-autoregressive masked image
  modeling (MIM) text-to-image to a level comparable with state-of-the-art
  diffusion models like SDXL. By incorporating a comprehensive suite of
  architectural innovations, advanced positional encoding strategies, and
  optimized sampling conditions, Meissonic substantially improves MIM's
  performance and efficiency. Additionally, we leverage high-quality training
  data, integrate micro-conditions informed by human preference scores, and
  employ feature compression layers to further enhance image fidelity and
  resolution. Our model not only matches but often exceeds the performance of
  existing models like SDXL in generating high-quality, high-resolution images.
  Extensive experiments validate Meissonic's capabilities, demonstrating its
  potential as a new standard in text-to-image synthesis. We release a model
  checkpoint capable of producing 1024 times 1024 resolution images.
paper_authors._id: 670c859d0b54b420bb46330e
paper_authors.name: Jinbin Bai
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-14T07:48:20.758Z
paper_authors._id: 63fccdac93b993a4ebd7789a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63fccdac93b993a4ebd7789a/KRx8vpdoDjsZBRw0j8Vg8.jpeg
paper_authors.fullname: Jinbin Bai
paper_authors.type: user
paper_authors.user: BryanW
paper_authors._id: 670c859d0b54b420bb46330f
paper_authors.name: Tian Ye
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-14T07:48:17.347Z
paper_authors._id: 66015e8aa4d296af07de538e
paper_authors.avatarUrl: /avatars/a1295c631cc2646282c545859975ce4c.svg
paper_authors.fullname: Ye
paper_authors.type: user
paper_authors.user: Owen777
paper_authors._id: 670c859d0b54b420bb463310
paper_authors.name: Wei Chow
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-14T08:12:54.075Z
paper_authors._id: 644b71ddb2e7823a76abcf91
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/644b71ddb2e7823a76abcf91/JPF7Eqeq2jx8i79nQ962K.jpeg
paper_authors.fullname: zhou wei
paper_authors.type: user
paper_authors.user: WeiChow
paper_authors._id: 670c859d0b54b420bb463311
paper_authors.name: Enxin Song
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-14T08:13:06.950Z
paper_authors._id: 63a7b196e27a6dbd4861e275
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63a7b196e27a6dbd4861e275/QGn4hdDPNuSRgZVKxzDpq.jpeg
paper_authors.fullname: EnxinSong
paper_authors.type: user
paper_authors.user: Enxin
paper_authors._id: 670c859d0b54b420bb463312
paper_authors.name: Qing-Guo Chen
paper_authors._id: 670c859d0b54b420bb463313
paper_authors.name: Xiangtai Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-14T07:48:15.462Z
paper_authors._id: 63958b4414513eaf9029ebf1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/U1g5H071pWRswGAG9UTpo.png
paper_authors.fullname: Xiangtai Li
paper_authors.type: user
paper_authors.user: LXT
paper_authors._id: 670c859d0b54b420bb463314
paper_authors.name: Zhen Dong
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-14T07:48:19.035Z
paper_authors._id: 643ba2f725681c3afaa8f05e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/643ba2f725681c3afaa8f05e/2RnOdmbBM8WHYhiTGG-Cd.jpeg
paper_authors.fullname: Zhen Dong
paper_authors.type: user
paper_authors.user: zhendongucb
paper_authors._id: 670c859d0b54b420bb463315
paper_authors.name: Lei Zhu
paper_authors._id: 670c859d0b54b420bb463316
paper_authors.name: Shuicheng Yan
paper_title: Meissonic: Revitalizing Masked Generative Transformers for Efficient
    High-Resolution Text-to-Image Synthesis
paper_summary: Diffusion models, such as Stable Diffusion, have made significant strides in
  visual generation, yet their paradigm remains fundamentally different from
  autoregressive language models, complicating the development of unified
  language-vision models. Recent efforts like LlamaGen have attempted
  autoregressive image generation using discrete VQVAE tokens, but the large
  number of tokens involved renders this approach inefficient and slow. In this
  work, we present Meissonic, which elevates non-autoregressive masked image
  modeling (MIM) text-to-image to a level comparable with state-of-the-art
  diffusion models like SDXL. By incorporating a comprehensive suite of
  architectural innovations, advanced positional encoding strategies, and
  optimized sampling conditions, Meissonic substantially improves MIM's
  performance and efficiency. Additionally, we leverage high-quality training
  data, integrate micro-conditions informed by human preference scores, and
  employ feature compression layers to further enhance image fidelity and
  resolution. Our model not only matches but often exceeds the performance of
  existing models like SDXL in generating high-quality, high-resolution images.
  Extensive experiments validate Meissonic's capabilities, demonstrating its
  potential as a new standard in text-to-image synthesis. We release a model
  checkpoint capable of producing 1024 times 1024 resolution images.
paper_ai_keywords: masked image modeling (MIM) text-to-image diffusion models autoregressive image generation VQVAE tokens architectural innovations positional encoding strategies sampling conditions advanced positional encoding strategies feature compression layers