HermesFlow: Seamlessly Closing the Gap in Multimodal Understanding and
  Generation
The remarkable success of the autoregressive paradigm has made significant
advancement in Multimodal Large Language Models (MLLMs), with powerful models
like Show-o, Transfusion and Emu3 achieving notable progress in unified image
understanding and generation. For the first time, we uncover a common
phenomenon: the understanding capabilities of MLLMs are typically stronger than
their generative capabilities, with a significant gap between the two. Building
on this insight, we propose HermesFlow, a simple yet general framework designed
to seamlessly bridge the gap between understanding and generation in MLLMs.
Specifically, we take the homologous data as input to curate homologous
preference data of both understanding and generation. Through Pair-DPO and
self-play iterative optimization, HermesFlow effectively aligns multimodal
understanding and generation using homologous preference data. Extensive
experiments demonstrate the significant superiority of our approach over prior
methods, particularly in narrowing the gap between multimodal understanding and
generation. These findings highlight the potential of HermesFlow as a general
alignment framework for next-generation multimodal foundation models. Code:
https://github.com/Gen-Verse/HermesFlow
67b40c8cdb88dfd19ab917f3
Ling Yang
claimed_verified
2025-02-20T09:37:43.227Z
64fde4e252e82dd432b74ce9
https://cdn-avatars.huggingface.co/v1/production/uploads/64fde4e252e82dd432b74ce9/-CQZbBP7FsPPyawYrsi4z.jpeg
Ling Yang
user
Lingaaaaaaa
67b40c8cdb88dfd19ab917f4
Xinchen Zhang
claimed_verified
2025-02-18T09:31:31.841Z
653e5d31ffd60206c8b64bb5
https://cdn-avatars.huggingface.co/v1/production/uploads/653e5d31ffd60206c8b64bb5/bgztraPC27L6culMlJw4s.png
Xinchen Zhang
user
comin
67b40c8cdb88dfd19ab917f5
Ye Tian
claimed_verified
2025-03-25T08:21:53.116Z
64e357dd825f4133e7427bf8
https://cdn-avatars.huggingface.co/v1/production/uploads/64e357dd825f4133e7427bf8/HwaWhINrzkbXG6SHG2oyf.jpeg
tyfeld
user
tyfeld
67b40c8cdb88dfd19ab917f6
Chenming Shang
67b40c8cdb88dfd19ab917f7
Minghao Xu
67b40c8cdb88dfd19ab917f8
Wentao Zhang
67b40c8cdb88dfd19ab917f9
Bin Cui
HermesFlow: Seamlessly Closing the Gap in Multimodal Understanding and
  Generation
The remarkable success of the autoregressive paradigm has made significant
advancement in Multimodal Large Language Models (MLLMs), with powerful models
like Show-o, Transfusion and Emu3 achieving notable progress in unified image
understanding and generation. For the first time, we uncover a common
phenomenon: the understanding capabilities of MLLMs are typically stronger than
their generative capabilities, with a significant gap between the two. Building
on this insight, we propose HermesFlow, a simple yet general framework designed
to seamlessly bridge the gap between understanding and generation in MLLMs.
Specifically, we take the homologous data as input to curate homologous
preference data of both understanding and generation. Through Pair-DPO and
self-play iterative optimization, HermesFlow effectively aligns multimodal
understanding and generation using homologous preference data. Extensive
experiments demonstrate the significant superiority of our approach over prior
methods, particularly in narrowing the gap between multimodal understanding and
generation. These findings highlight the potential of HermesFlow as a general
alignment framework for next-generation multimodal foundation models. Code:
https://github.com/Gen-Verse/HermesFlow
HermesFlow aligns multimodal understanding and generation through homologous preference data, demonstrating superior performance compared to existing methods.
autoregressive paradigm Multimodal Large Language Models (MLLMs) Show-o Transfusion Emu3 unified image understanding unified image generation HermesFlow homologous data homologous preference data Pair-DPO self-play iterative optimization alignment framework next-generation multimodal foundation models