title: UniRL: Self-Improving Unified Multimodal Models via Supervised and
    Reinforcement Learning
summary: Unified multimodal large language models such as Show-o and Janus have
  achieved strong performance across both generation and understanding tasks.
  However, these models typically rely on large-scale datasets and require
  substantial computation during the pretraining stage. In addition, several
  post-training methods have been proposed, but they often depend on external
  data or are limited to task-specific customization. In this work, we introduce
  UniRL, a self-improving post-training approach. Our approach enables the model
  to generate images from prompts and use them as training data in each
  iteration, without relying on any external image data. Moreover, it enables the
  two tasks to enhance each other: the generated images are used for
  understanding, and the understanding results are used to supervise generation.
  We explore supervised fine-tuning (SFT) and Group Relative Policy Optimization
  (GRPO) to optimize the models. UniRL offers three key advantages: (1) it
  requires no external image data, as all training samples are generated by the
  model itself during training; (2) it not only improves individual task
  performance, but also reduces the imbalance between generation and
  understanding; and (3) it requires only several additional training steps
  during the post-training stage. We evaluate UniRL on top of Show-o and Janus,
  achieving a GenEval score of 0.77 for Show-o and 0.65 for Janus. Code and
  models will be released in https://github.com/showlab/UniRL.
paper_authors._id: 683906f81bf7a7a94309c5a5
paper_authors.name: Weijia Mao
paper_authors._id: 683906f81bf7a7a94309c5a6
paper_authors.name: Zhenheng Yang
paper_authors._id: 683906f81bf7a7a94309c5a7
paper_authors.name: Mike Zheng Shou
paper_title: UniRL: Self-Improving Unified Multimodal Models via Supervised and
    Reinforcement Learning
paper_summary: Unified multimodal large language models such as Show-o and Janus have
  achieved strong performance across both generation and understanding tasks.
  However, these models typically rely on large-scale datasets and require
  substantial computation during the pretraining stage. In addition, several
  post-training methods have been proposed, but they often depend on external
  data or are limited to task-specific customization. In this work, we introduce
  UniRL, a self-improving post-training approach. Our approach enables the model
  to generate images from prompts and use them as training data in each
  iteration, without relying on any external image data. Moreover, it enables the
  two tasks to enhance each other: the generated images are used for
  understanding, and the understanding results are used to supervise generation.
  We explore supervised fine-tuning (SFT) and Group Relative Policy Optimization
  (GRPO) to optimize the models. UniRL offers three key advantages: (1) it
  requires no external image data, as all training samples are generated by the
  model itself during training; (2) it not only improves individual task
  performance, but also reduces the imbalance between generation and
  understanding; and (3) it requires only several additional training steps
  during the post-training stage. We evaluate UniRL on top of Show-o and Janus,
  achieving a GenEval score of 0.77 for Show-o and 0.65 for Janus. Code and
  models will be released in https://github.com/showlab/UniRL.
paper_ai_summary: UniRL is a self-improving post-training method for unified multimodal large language models that uses generated images as training data, enhancing both generation and understanding tasks without external data.
paper_ai_keywords: Unified multimodal large language models Show-o Janus self-improving post-training prompts training data supervised fine-tuning Group Relative Policy Optimization GenEval score