title: MMaDA: Multimodal Large Diffusion Language Models
summary: We introduce MMaDA, a novel class of multimodal diffusion foundation models
  designed to achieve superior performance across diverse domains such as textual
  reasoning, multimodal understanding, and text-to-image generation. The approach
  is distinguished by three key innovations: (i) MMaDA adopts a unified diffusion
  architecture with a shared probabilistic formulation and a modality-agnostic
  design, eliminating the need for modality-specific components. This
  architecture ensures seamless integration and processing across different data
  types. (ii) We implement a mixed long chain-of-thought (CoT) fine-tuning
  strategy that curates a unified CoT format across modalities. By aligning
  reasoning processes between textual and visual domains, this strategy
  facilitates cold-start training for the final reinforcement learning (RL)
  stage, thereby enhancing the model's ability to handle complex tasks from the
  outset. (iii) We propose UniGRPO, a unified policy-gradient-based RL algorithm
  specifically tailored for diffusion foundation models. Utilizing diversified
  reward modeling, UniGRPO unifies post-training across both reasoning and
  generation tasks, ensuring consistent performance improvements. Experimental
  results demonstrate that MMaDA-8B exhibits strong generalization capabilities
  as a unified multimodal foundation model. It surpasses powerful models like
  LLaMA-3-7B and Qwen2-7B in textual reasoning, outperforms Show-o and SEED-X in
  multimodal understanding, and excels over SDXL and Janus in text-to-image
  generation. These achievements highlight MMaDA's effectiveness in bridging the
  gap between pretraining and post-training within unified diffusion
  architectures, providing a comprehensive framework for future research and
  development. We open-source our code and trained models at:
  https://github.com/Gen-Verse/MMaDA
paper_authors._id: 682e7e061d7637a25846bf52
paper_authors.name: Ling Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-26T08:22:31.113Z
paper_authors._id: 64fde4e252e82dd432b74ce9
paper_authors.avatarUrl: /avatars/061a69d858b86d1600be916122cae7fc.svg
paper_authors.fullname: Ling Yang
paper_authors.type: user
paper_authors.user: Lingaaaaaaa
paper_authors._id: 682e7e061d7637a25846bf53
paper_authors.name: Ye Tian
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-22T07:17:11.786Z
paper_authors._id: 64e357dd825f4133e7427bf8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64e357dd825f4133e7427bf8/HwaWhINrzkbXG6SHG2oyf.jpeg
paper_authors.fullname: tyfeld
paper_authors.type: user
paper_authors.user: tyfeld
paper_authors._id: 682e7e061d7637a25846bf54
paper_authors.name: Bowen Li
paper_authors._id: 682e7e061d7637a25846bf55
paper_authors.name: Xinchen Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-22T07:17:14.114Z
paper_authors._id: 653e5d31ffd60206c8b64bb5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/653e5d31ffd60206c8b64bb5/JrfOSCunFSW39vdW7E59y.png
paper_authors.fullname: Xinchen Zhang
paper_authors.type: user
paper_authors.user: comin
paper_authors._id: 682e7e061d7637a25846bf56
paper_authors.name: Ke Shen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-26T08:22:28.227Z
paper_authors._id: 645604eebabbbbd3486dc615
paper_authors.avatarUrl: /avatars/17a5ca8274e2bfc8f183a4af9878a930.svg
paper_authors.fullname: shenke
paper_authors.type: user
paper_authors.user: shenke18
paper_authors._id: 682e7e061d7637a25846bf57
paper_authors.name: Yunhai Tong
paper_authors._id: 682e7e061d7637a25846bf58
paper_authors.name: Mengdi Wang
paper_title: MMaDA: Multimodal Large Diffusion Language Models
paper_summary: We introduce MMaDA, a novel class of multimodal diffusion foundation models
  designed to achieve superior performance across diverse domains such as textual
  reasoning, multimodal understanding, and text-to-image generation. The approach
  is distinguished by three key innovations: (i) MMaDA adopts a unified diffusion
  architecture with a shared probabilistic formulation and a modality-agnostic
  design, eliminating the need for modality-specific components. This
  architecture ensures seamless integration and processing across different data
  types. (ii) We implement a mixed long chain-of-thought (CoT) fine-tuning
  strategy that curates a unified CoT format across modalities. By aligning
  reasoning processes between textual and visual domains, this strategy
  facilitates cold-start training for the final reinforcement learning (RL)
  stage, thereby enhancing the model's ability to handle complex tasks from the
  outset. (iii) We propose UniGRPO, a unified policy-gradient-based RL algorithm
  specifically tailored for diffusion foundation models. Utilizing diversified
  reward modeling, UniGRPO unifies post-training across both reasoning and
  generation tasks, ensuring consistent performance improvements. Experimental
  results demonstrate that MMaDA-8B exhibits strong generalization capabilities
  as a unified multimodal foundation model. It surpasses powerful models like
  LLaMA-3-7B and Qwen2-7B in textual reasoning, outperforms Show-o and SEED-X in
  multimodal understanding, and excels over SDXL and Janus in text-to-image
  generation. These achievements highlight MMaDA's effectiveness in bridging the
  gap between pretraining and post-training within unified diffusion
  architectures, providing a comprehensive framework for future research and
  development. We open-source our code and trained models at:
  https://github.com/Gen-Verse/MMaDA
paper_ai_summary: MMaDA, a multimodal diffusion foundation model, achieves superior performance through a unified architecture, mixed long chain-of-thought fine-tuning, and a unified policy-gradient-based RL algorithm.
paper_ai_keywords: multimodal diffusion foundation models unified diffusion architecture modality-agnostic design mixed long chain-of-thought fine-tuning cold-start training reinforcement learning UniGRPO policy-gradient-based RL algorithm diversified reward modeling generalization capabilities textual reasoning multimodal understanding text-to-image generation