title: UniWorld: High-Resolution Semantic Encoders for Unified Visual
    Understanding and Generation
summary: Although existing unified models deliver strong performance on
  vision-language understanding and text-to-image generation, their models are
  limited in exploring image perception and manipulation tasks, which are
  urgently desired by users for wide applications. Recently, OpenAI released
  their powerful GPT-4o-Image model for comprehensive image perception and
  manipulation, achieving expressive capability and attracting community
  interests. By observing the performance of GPT-4o-Image in our carefully
  constructed experiments, we infer that GPT-4o-Image leverages features
  extracted by semantic encoders instead of VAE, while VAEs are considered
  essential components in many image manipulation models. Motivated by such
  inspiring observations, we present a unified generative framework named
  UniWorld based on semantic features provided by powerful visual-language models
  and contrastive semantic encoders. As a result, we build a strong unified model
  using only 1% amount of BAGEL's data, which consistently outperforms BAGEL on
  image editing benchmarks. UniWorld also maintains competitive image
  understanding and generation capabilities, achieving strong performance across
  multiple image perception tasks. We fully open-source our models, including
  model weights, training and evaluation scripts, and datasets.
paper_authors._id: 683fae55c6b71c5994ccd4fe
paper_authors.name: Bin Lin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-04T08:56:49.923Z
paper_authors._id: 6367a8175bb06007ea099b8f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6367a8175bb06007ea099b8f/IjG7HyWyWRlVt_XwRbxRW.jpeg
paper_authors.fullname: linbin
paper_authors.type: user
paper_authors.user: LanguageBind
paper_authors._id: 683fae55c6b71c5994ccd4ff
paper_authors.name: Zongjian Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-04T15:03:37.427Z
paper_authors._id: 646df3c04ad7f907279f14c3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/646df3c04ad7f907279f14c3/WZjSDtAmezmjbczLtCP2B.jpeg
paper_authors.fullname: Zongjian Li
paper_authors.type: user
paper_authors.user: chestnutlzj
paper_authors._id: 683fae55c6b71c5994ccd500
paper_authors.name: Xinhua Cheng
paper_authors._id: 683fae55c6b71c5994ccd501
paper_authors.name: Yuwei Niu
paper_authors._id: 683fae55c6b71c5994ccd502
paper_authors.name: Yang Ye
paper_authors._id: 683fae55c6b71c5994ccd503
paper_authors.name: Xianyi He
paper_authors._id: 683fae55c6b71c5994ccd504
paper_authors.name: Shenghai Yuan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-04T08:56:52.748Z
paper_authors._id: 63468720dd6d90d82ccf3450
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63468720dd6d90d82ccf3450/tVBFlmZNz8FRMkOrDaDID.jpeg
paper_authors.fullname: YSH
paper_authors.type: user
paper_authors.user: BestWishYsh
paper_authors._id: 683fae55c6b71c5994ccd505
paper_authors.name: Wangbo Yu
paper_authors._id: 683fae55c6b71c5994ccd506
paper_authors.name: Shaodong Wang
paper_authors._id: 683fae55c6b71c5994ccd507
paper_authors.name: Yunyang Ge
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-06-05T07:41:16.521Z
paper_authors._id: 646de6402fd5a8eb8c518aa6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/646de6402fd5a8eb8c518aa6/HYWb8-fT1kTm-ROBr1-0X.jpeg
paper_authors.fullname: yunyangge
paper_authors.type: user
paper_authors.user: yunyangge
paper_authors._id: 683fae55c6b71c5994ccd508
paper_authors.name: Yatian Pang
paper_authors._id: 683fae55c6b71c5994ccd509
paper_authors.name: Li Yuan
paper_title: UniWorld: High-Resolution Semantic Encoders for Unified Visual
    Understanding and Generation
paper_summary: Although existing unified models deliver strong performance on
  vision-language understanding and text-to-image generation, their models are
  limited in exploring image perception and manipulation tasks, which are
  urgently desired by users for wide applications. Recently, OpenAI released
  their powerful GPT-4o-Image model for comprehensive image perception and
  manipulation, achieving expressive capability and attracting community
  interests. By observing the performance of GPT-4o-Image in our carefully
  constructed experiments, we infer that GPT-4o-Image leverages features
  extracted by semantic encoders instead of VAE, while VAEs are considered
  essential components in many image manipulation models. Motivated by such
  inspiring observations, we present a unified generative framework named
  UniWorld based on semantic features provided by powerful visual-language models
  and contrastive semantic encoders. As a result, we build a strong unified model
  using only 1% amount of BAGEL's data, which consistently outperforms BAGEL on
  image editing benchmarks. UniWorld also maintains competitive image
  understanding and generation capabilities, achieving strong performance across
  multiple image perception tasks. We fully open-source our models, including
  model weights, training and evaluation scripts, and datasets.
paper_ai_summary: A unified generative framework called UniWorld uses semantic features from visual-language models for image perception and manipulation, outperforming BAGEL with reduced data.
paper_ai_keywords: GPT-4o-Image semantic encoders VAE UniWorld visual-language models contrastive semantic encoders image editing benchmarks image perception tasks