Muddit: Liberating Generation Beyond Text-to-Image with a Unified
  Discrete Diffusion Model
Unified generation models aim to handle diverse tasks across modalities --
such as text generation, image generation, and vision-language reasoning --
within a single architecture and decoding paradigm. Autoregressive unified
models suffer from slow inference due to sequential decoding, and
non-autoregressive unified models suffer from weak generalization due to
limited pretrained backbones. We introduce Muddit, a unified discrete diffusion
transformer that enables fast and parallel generation across both text and
image modalities. Unlike prior unified diffusion models trained from scratch,
Muddit integrates strong visual priors from a pretrained text-to-image backbone
with a lightweight text decoder, enabling flexible and high-quality multimodal
generation under a unified architecture. Empirical results show that Muddit
achieves competitive or superior performance compared to significantly larger
autoregressive models in both quality and efficiency. The work highlights the
potential of purely discrete diffusion, when equipped with strong visual
priors, as a scalable and effective backbone for unified generation.
6839189f4a3a71a917b0514e
Qingyu Shi
claimed_verified
2025-05-30T06:53:45.160Z
656724074f6ec72017754d33
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/zLIgt4xvfbYPLZh0E3WWF.png
QingyuShi
user
QingyuShi
6839189f4a3a71a917b0514f
Jinbin Bai
claimed_verified
2025-05-30T06:53:47.122Z
63fccdac93b993a4ebd7789a
https://cdn-avatars.huggingface.co/v1/production/uploads/63fccdac93b993a4ebd7789a/KRx8vpdoDjsZBRw0j8Vg8.jpeg
Jinbin Bai
user
BryanW
6839189f4a3a71a917b05150
Zhuoran Zhao
6839189f4a3a71a917b05151
Wenhao Chai
claimed_verified
2025-06-16T15:51:01.740Z
637c7503fe115289cfecbe6b
https://cdn-avatars.huggingface.co/v1/production/uploads/1676361945047-637c7503fe115289cfecbe6b.jpeg
Wenhao Chai
user
wchai
6839189f4a3a71a917b05152
Kaidong Yu
6839189f4a3a71a917b05153
Jianzong Wu
6839189f4a3a71a917b05154
Shuangyong Song
6839189f4a3a71a917b05155
Yunhai Tong
6839189f4a3a71a917b05156
Xiangtai Li
claimed_verified
2025-06-11T08:37:11.380Z
63958b4414513eaf9029ebf1
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/U1g5H071pWRswGAG9UTpo.png
Xiangtai Li
user
LXT
6839189f4a3a71a917b05157
Xuelong Li
6839189f4a3a71a917b05158
Shuicheng Yan
Muddit: Liberating Generation Beyond Text-to-Image with a Unified
  Discrete Diffusion Model
Unified generation models aim to handle diverse tasks across modalities --
such as text generation, image generation, and vision-language reasoning --
within a single architecture and decoding paradigm. Autoregressive unified
models suffer from slow inference due to sequential decoding, and
non-autoregressive unified models suffer from weak generalization due to
limited pretrained backbones. We introduce Muddit, a unified discrete diffusion
transformer that enables fast and parallel generation across both text and
image modalities. Unlike prior unified diffusion models trained from scratch,
Muddit integrates strong visual priors from a pretrained text-to-image backbone
with a lightweight text decoder, enabling flexible and high-quality multimodal
generation under a unified architecture. Empirical results show that Muddit
achieves competitive or superior performance compared to significantly larger
autoregressive models in both quality and efficiency. The work highlights the
potential of purely discrete diffusion, when equipped with strong visual
priors, as a scalable and effective backbone for unified generation.
Muddit, a unified discrete diffusion transformer, achieves fast and high-quality generation across text and image modalities by integrating pretrained visual priors with a lightweight text decoder.
unified generation models autoregressive models non-autoregressive models discrete diffusion diffusion transformer pretrained backbones flexible generation multimodal generation text generation image generation vision-language reasoning quality efficiency visual priors scalable backbone