title: Muddit: Liberating Generation Beyond Text-to-Image with a Unified
    Discrete Diffusion Model
summary: Unified generation models aim to handle diverse tasks across modalities --
  such as text generation, image generation, and vision-language reasoning --
  within a single architecture and decoding paradigm. Autoregressive unified
  models suffer from slow inference due to sequential decoding, and
  non-autoregressive unified models suffer from weak generalization due to
  limited pretrained backbones. We introduce Muddit, a unified discrete diffusion
  transformer that enables fast and parallel generation across both text and
  image modalities. Unlike prior unified diffusion models trained from scratch,
  Muddit integrates strong visual priors from a pretrained text-to-image backbone
  with a lightweight text decoder, enabling flexible and high-quality multimodal
  generation under a unified architecture. Empirical results show that Muddit
  achieves competitive or superior performance compared to significantly larger
  autoregressive models in both quality and efficiency. The work highlights the
  potential of purely discrete diffusion, when equipped with strong visual
  priors, as a scalable and effective backbone for unified generation.
paper_authors._id: 6839189f4a3a71a917b0514e
paper_authors.name: Qingyu Shi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-30T06:53:45.160Z
paper_authors._id: 656724074f6ec72017754d33
paper_authors.avatarUrl: /avatars/e61de248f6f53719b2375077340dd033.svg
paper_authors.fullname: QingyuShi
paper_authors.type: user
paper_authors.user: QingyuShi
paper_authors._id: 6839189f4a3a71a917b0514f
paper_authors.name: Jinbin Bai
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-30T06:53:47.122Z
paper_authors._id: 63fccdac93b993a4ebd7789a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63fccdac93b993a4ebd7789a/KRx8vpdoDjsZBRw0j8Vg8.jpeg
paper_authors.fullname: Jinbin Bai
paper_authors.type: user
paper_authors.user: BryanW
paper_authors._id: 6839189f4a3a71a917b05150
paper_authors.name: Zhuoran Zhao
paper_authors._id: 6839189f4a3a71a917b05151
paper_authors.name: Wenhao Chai
paper_authors._id: 6839189f4a3a71a917b05152
paper_authors.name: Kaidong Yu
paper_authors._id: 6839189f4a3a71a917b05153
paper_authors.name: Jianzong Wu
paper_authors._id: 6839189f4a3a71a917b05154
paper_authors.name: Shuangyong Song
paper_authors._id: 6839189f4a3a71a917b05155
paper_authors.name: Yunhai Tong
paper_authors._id: 6839189f4a3a71a917b05156
paper_authors.name: Xiangtai Li
paper_authors._id: 6839189f4a3a71a917b05157
paper_authors.name: Xuelong Li
paper_authors._id: 6839189f4a3a71a917b05158
paper_authors.name: Shuicheng Yan
paper_title: Muddit: Liberating Generation Beyond Text-to-Image with a Unified
    Discrete Diffusion Model
paper_summary: Unified generation models aim to handle diverse tasks across modalities --
  such as text generation, image generation, and vision-language reasoning --
  within a single architecture and decoding paradigm. Autoregressive unified
  models suffer from slow inference due to sequential decoding, and
  non-autoregressive unified models suffer from weak generalization due to
  limited pretrained backbones. We introduce Muddit, a unified discrete diffusion
  transformer that enables fast and parallel generation across both text and
  image modalities. Unlike prior unified diffusion models trained from scratch,
  Muddit integrates strong visual priors from a pretrained text-to-image backbone
  with a lightweight text decoder, enabling flexible and high-quality multimodal
  generation under a unified architecture. Empirical results show that Muddit
  achieves competitive or superior performance compared to significantly larger
  autoregressive models in both quality and efficiency. The work highlights the
  potential of purely discrete diffusion, when equipped with strong visual
  priors, as a scalable and effective backbone for unified generation.
paper_ai_summary: Muddit, a unified discrete diffusion transformer, achieves fast and high-quality generation across text and image modalities by integrating pretrained visual priors with a lightweight text decoder.
paper_ai_keywords: unified generation models autoregressive models non-autoregressive models discrete diffusion diffusion transformer pretrained backbones flexible generation multimodal generation text generation image generation vision-language reasoning quality efficiency visual priors scalable backbone