title: HermesFlow: Seamlessly Closing the Gap in Multimodal Understanding and
    Generation
summary: The remarkable success of the autoregressive paradigm has made significant
  advancement in Multimodal Large Language Models (MLLMs), with powerful models
  like Show-o, Transfusion and Emu3 achieving notable progress in unified image
  understanding and generation. For the first time, we uncover a common
  phenomenon: the understanding capabilities of MLLMs are typically stronger than
  their generative capabilities, with a significant gap between the two. Building
  on this insight, we propose HermesFlow, a simple yet general framework designed
  to seamlessly bridge the gap between understanding and generation in MLLMs.
  Specifically, we take the homologous data as input to curate homologous
  preference data of both understanding and generation. Through Pair-DPO and
  self-play iterative optimization, HermesFlow effectively aligns multimodal
  understanding and generation using homologous preference data. Extensive
  experiments demonstrate the significant superiority of our approach over prior
  methods, particularly in narrowing the gap between multimodal understanding and
  generation. These findings highlight the potential of HermesFlow as a general
  alignment framework for next-generation multimodal foundation models. Code:
  https://github.com/Gen-Verse/HermesFlow
paper_authors._id: 67b40c8cdb88dfd19ab917f3
paper_authors.name: Ling Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-20T09:37:43.227Z
paper_authors._id: 64fde4e252e82dd432b74ce9
paper_authors.avatarUrl: /avatars/061a69d858b86d1600be916122cae7fc.svg
paper_authors.fullname: Ling Yang
paper_authors.type: user
paper_authors.user: Lingaaaaaaa
paper_authors._id: 67b40c8cdb88dfd19ab917f4
paper_authors.name: Xinchen Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-18T09:31:31.841Z
paper_authors._id: 653e5d31ffd60206c8b64bb5
paper_authors.avatarUrl: /avatars/5076795722ec1f9e031654f301d30e8f.svg
paper_authors.fullname: Xinchen Zhang
paper_authors.type: user
paper_authors.user: comin
paper_authors._id: 67b40c8cdb88dfd19ab917f5
paper_authors.name: Ye Tian
paper_authors._id: 67b40c8cdb88dfd19ab917f6
paper_authors.name: Chenming Shang
paper_authors._id: 67b40c8cdb88dfd19ab917f7
paper_authors.name: Minghao Xu
paper_authors._id: 67b40c8cdb88dfd19ab917f8
paper_authors.name: Wentao Zhang
paper_authors._id: 67b40c8cdb88dfd19ab917f9
paper_authors.name: Bin Cui
paper_title: HermesFlow: Seamlessly Closing the Gap in Multimodal Understanding and
    Generation
paper_summary: The remarkable success of the autoregressive paradigm has made significant
  advancement in Multimodal Large Language Models (MLLMs), with powerful models
  like Show-o, Transfusion and Emu3 achieving notable progress in unified image
  understanding and generation. For the first time, we uncover a common
  phenomenon: the understanding capabilities of MLLMs are typically stronger than
  their generative capabilities, with a significant gap between the two. Building
  on this insight, we propose HermesFlow, a simple yet general framework designed
  to seamlessly bridge the gap between understanding and generation in MLLMs.
  Specifically, we take the homologous data as input to curate homologous
  preference data of both understanding and generation. Through Pair-DPO and
  self-play iterative optimization, HermesFlow effectively aligns multimodal
  understanding and generation using homologous preference data. Extensive
  experiments demonstrate the significant superiority of our approach over prior
  methods, particularly in narrowing the gap between multimodal understanding and
  generation. These findings highlight the potential of HermesFlow as a general
  alignment framework for next-generation multimodal foundation models. Code:
  https://github.com/Gen-Verse/HermesFlow
paper_ai_keywords: autoregressive paradigm Multimodal Large Language Models (MLLMs) Show-o Transfusion Emu3 unified image understanding and generation homologous data homologous preference data Pair-DPO self-play iterative optimization HermesFlow multimodal understanding and generation next-generation multimodal foundation models