title: X-Prompt: Towards Universal In-Context Image Generation in
    Auto-Regressive Vision Language Foundation Models
summary: In-context generation is a key component of large language models' (LLMs)
  open-task generalization capability. By leveraging a few examples as context,
  LLMs can perform both in-domain and out-of-domain tasks. Recent advancements in
  auto-regressive vision-language models (VLMs) built upon LLMs have showcased
  impressive performance in text-to-image generation. However, the potential of
  in-context learning for general image generation tasks remains largely
  unexplored. To address this, we introduce X-Prompt, a purely auto-regressive
  large-vision language model designed to deliver competitive performance across
  a wide range of both seen and unseen image generation tasks, all within a
  unified in-context learning framework. X-Prompt incorporates a specialized
  design that efficiently compresses valuable features from in-context examples,
  supporting longer in-context token sequences and improving its ability to
  generalize to unseen tasks. A unified training task for both text and image
  prediction enables X-Prompt to handle general image generation with enhanced
  task awareness from in-context examples. Extensive experiments validate the
  model's performance across diverse seen image generation tasks and its capacity
  to generalize to previously unseen tasks.
paper_authors._id: 674e8f89ec384b629bbc0f60
paper_authors.name: Zeyi Sun
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-03T11:14:44.696Z
paper_authors._id: 63fda3fced9eead590ff6918
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1677566802735-noauth.jpeg
paper_authors.fullname: Zeyi Sun
paper_authors.type: user
paper_authors.user: Zery
paper_authors._id: 674e8f89ec384b629bbc0f61
paper_authors.name: Ziyang Chu
paper_authors._id: 674e8f89ec384b629bbc0f62
paper_authors.name: Pan Zhang
paper_authors._id: 674e8f89ec384b629bbc0f63
paper_authors.name: Tong Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-03T11:27:25.489Z
paper_authors._id: 62fae9328e137d7c4b896498
paper_authors.avatarUrl: /avatars/1bda39dec585c099417cc9daa9f53c42.svg
paper_authors.fullname: Tong Wu
paper_authors.type: user
paper_authors.user: tongwu2020
paper_authors._id: 674e8f89ec384b629bbc0f64
paper_authors.name: Xiaoyi Dong
paper_authors._id: 674e8f89ec384b629bbc0f65
paper_authors.name: Yuhang Zang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-03T11:27:44.758Z
paper_authors._id: 63859cf3b2906edaf83af9f0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63859cf3b2906edaf83af9f0/iUQm5FAomzqYi6fkqIn9F.jpeg
paper_authors.fullname: Yuhang Zang
paper_authors.type: user
paper_authors.user: yuhangzang
paper_authors._id: 674e8f89ec384b629bbc0f66
paper_authors.name: Yuanjun Xiong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-03T11:27:54.048Z
paper_authors._id: 66209605f44be495f123b055
paper_authors.avatarUrl: /avatars/b72081a0383a709899f9ba80b4211690.svg
paper_authors.fullname: Yuanjun Xiong
paper_authors.type: user
paper_authors.user: TheYJ
paper_authors._id: 674e8f89ec384b629bbc0f67
paper_authors.name: Dahua Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-03T11:28:00.539Z
paper_authors._id: 636317ed80c1a705a6eff396
paper_authors.avatarUrl: /avatars/3db090e101b916d9256d0d3e043db71d.svg
paper_authors.fullname: Dahua Lin
paper_authors.type: user
paper_authors.user: lindahua
paper_authors._id: 674e8f89ec384b629bbc0f68
paper_authors.name: Jiaqi Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-11T09:21:04.208Z
paper_authors._id: 64b4eec4faa3181a5eab9c46
paper_authors.avatarUrl: /avatars/bcc9bf5cbf67546ad2b4c9ec8b96ac96.svg
paper_authors.fullname: Jiaqi Wang
paper_authors.type: user
paper_authors.user: myownskyW7
paper_title: X-Prompt: Towards Universal In-Context Image Generation in
    Auto-Regressive Vision Language Foundation Models
paper_summary: In-context generation is a key component of large language models' (LLMs)
  open-task generalization capability. By leveraging a few examples as context,
  LLMs can perform both in-domain and out-of-domain tasks. Recent advancements in
  auto-regressive vision-language models (VLMs) built upon LLMs have showcased
  impressive performance in text-to-image generation. However, the potential of
  in-context learning for general image generation tasks remains largely
  unexplored. To address this, we introduce X-Prompt, a purely auto-regressive
  large-vision language model designed to deliver competitive performance across
  a wide range of both seen and unseen image generation tasks, all within a
  unified in-context learning framework. X-Prompt incorporates a specialized
  design that efficiently compresses valuable features from in-context examples,
  supporting longer in-context token sequences and improving its ability to
  generalize to unseen tasks. A unified training task for both text and image
  prediction enables X-Prompt to handle general image generation with enhanced
  task awareness from in-context examples. Extensive experiments validate the
  model's performance across diverse seen image generation tasks and its capacity
  to generalize to previously unseen tasks.
paper_ai_keywords: auto-regressive vision-language models (VLMs) in-context learning in-context token sequences unified in-double learning framework X-Prompt text-to-image generation in-context examples task awareness