X-Prompt: Towards Universal In-Context Image Generation in
  Auto-Regressive Vision Language Foundation Models
In-context generation is a key component of large language models' (LLMs)
open-task generalization capability. By leveraging a few examples as context,
LLMs can perform both in-domain and out-of-domain tasks. Recent advancements in
auto-regressive vision-language models (VLMs) built upon LLMs have showcased
impressive performance in text-to-image generation. However, the potential of
in-context learning for general image generation tasks remains largely
unexplored. To address this, we introduce X-Prompt, a purely auto-regressive
large-vision language model designed to deliver competitive performance across
a wide range of both seen and unseen image generation tasks, all within a
unified in-context learning framework. X-Prompt incorporates a specialized
design that efficiently compresses valuable features from in-context examples,
supporting longer in-context token sequences and improving its ability to
generalize to unseen tasks. A unified training task for both text and image
prediction enables X-Prompt to handle general image generation with enhanced
task awareness from in-context examples. Extensive experiments validate the
model's performance across diverse seen image generation tasks and its capacity
to generalize to previously unseen tasks.
674e8f89ec384b629bbc0f60
Zeyi Sun
claimed_verified
2024-12-03T11:14:44.696Z
63fda3fced9eead590ff6918
https://cdn-avatars.huggingface.co/v1/production/uploads/1677566802735-noauth.jpeg
Zeyi Sun
user
Zery
674e8f89ec384b629bbc0f61
Ziyang Chu
674e8f89ec384b629bbc0f62
Pan Zhang
674e8f89ec384b629bbc0f63
Tong Wu
admin_assigned
2024-12-03T11:27:25.489Z
62fae9328e137d7c4b896498
/avatars/1bda39dec585c099417cc9daa9f53c42.svg
Tong Wu
user
tongwu2020
674e8f89ec384b629bbc0f64
Xiaoyi Dong
674e8f89ec384b629bbc0f65
Yuhang Zang
admin_assigned
2024-12-03T11:27:44.758Z
63859cf3b2906edaf83af9f0
https://cdn-avatars.huggingface.co/v1/production/uploads/63859cf3b2906edaf83af9f0/mCgynEtoXdILvzyoPexii.png
Yuhang Zang
user
yuhangzang
674e8f89ec384b629bbc0f66
Yuanjun Xiong
admin_assigned
2024-12-03T11:27:54.048Z
66209605f44be495f123b055
/avatars/b72081a0383a709899f9ba80b4211690.svg
Yuanjun Xiong
user
TheYJ
674e8f89ec384b629bbc0f67
Dahua Lin
admin_assigned
2024-12-03T11:28:00.539Z
636317ed80c1a705a6eff396
/avatars/3db090e101b916d9256d0d3e043db71d.svg
Dahua Lin
user
lindahua
674e8f89ec384b629bbc0f68
Jiaqi Wang
claimed_verified
2024-12-11T09:21:04.208Z
64b4eec4faa3181a5eab9c46
/avatars/bcc9bf5cbf67546ad2b4c9ec8b96ac96.svg
Jiaqi Wang
user
myownskyW7
X-Prompt: Towards Universal In-Context Image Generation in
  Auto-Regressive Vision Language Foundation Models
In-context generation is a key component of large language models' (LLMs)
open-task generalization capability. By leveraging a few examples as context,
LLMs can perform both in-domain and out-of-domain tasks. Recent advancements in
auto-regressive vision-language models (VLMs) built upon LLMs have showcased
impressive performance in text-to-image generation. However, the potential of
in-context learning for general image generation tasks remains largely
unexplored. To address this, we introduce X-Prompt, a purely auto-regressive
large-vision language model designed to deliver competitive performance across
a wide range of both seen and unseen image generation tasks, all within a
unified in-context learning framework. X-Prompt incorporates a specialized
design that efficiently compresses valuable features from in-context examples,
supporting longer in-context token sequences and improving its ability to
generalize to unseen tasks. A unified training task for both text and image
prediction enables X-Prompt to handle general image generation with enhanced
task awareness from in-context examples. Extensive experiments validate the
model's performance across diverse seen image generation tasks and its capacity
to generalize to previously unseen tasks.
X-Prompt, an auto-regressive large-vision language model, uses in-context learning to achieve competitive performance in seen and unseen image generation tasks.
in-context generation large language models LLMs auto-regressive vision-language models VLMs text-to-image generation in-context learning large-vision language model in-context token sequences unified in-context learning framework unified training task text prediction image prediction task awareness image generation