Ovis-U1 Technical Report
In this report, we introduce Ovis-U1, a 3-billion-parameter unified model
that integrates multimodal understanding, text-to-image generation, and image
editing capabilities. Building on the foundation of the Ovis series, Ovis-U1
incorporates a diffusion-based visual decoder paired with a bidirectional token
refiner, enabling image generation tasks comparable to leading models like
GPT-4o. Unlike some previous models that use a frozen MLLM for generation
tasks, Ovis-U1 utilizes a new unified training approach starting from a
language model. Compared to training solely on understanding or generation
tasks, unified training yields better performance, demonstrating the
enhancement achieved by integrating these two tasks. Ovis-U1 achieves a score
of 69.6 on the OpenCompass Multi-modal Academic Benchmark, surpassing recent
state-of-the-art models such as Ristretto-3B and SAIL-VL-1.5-2B. In
text-to-image generation, it excels with scores of 83.72 and 0.89 on the
DPG-Bench and GenEval benchmarks, respectively. For image editing, it achieves
4.00 and 6.42 on the ImgEdit-Bench and GEdit-Bench-EN, respectively. As the
initial version of the Ovis unified model series, Ovis-U1 pushes the boundaries
of multimodal understanding, generation, and editing.
686347cd588cea0da970c87a
Guo-Hua Wang
claimed_verified
2025-07-01T10:13:51.516Z
636f4c6b5d2050767e4a1491
https://cdn-avatars.huggingface.co/v1/production/uploads/636f4c6b5d2050767e4a1491/qErBLQnYzL35EkbDAhvqN.jpeg
Guo-Hua Wang
user
Flourish
686347cd588cea0da970c87b
Shanshan Zhao
686347cd588cea0da970c87c
Xinjie Zhang
686347cd588cea0da970c87d
Liangfu Cao
686347cd588cea0da970c87e
Pengxin Zhan
686347cd588cea0da970c87f
Lunhao Duan
686347cd588cea0da970c880
Shiyin Lu
686347cd588cea0da970c881
Minghao Fu
686347cd588cea0da970c882
Xiaohao Chen
686347cd588cea0da970c883
Jianshan Zhao
686347cd588cea0da970c884
Yang Li
686347cd588cea0da970c885
Qing-Guo Chen
Ovis-U1 Technical Report
In this report, we introduce Ovis-U1, a 3-billion-parameter unified model
that integrates multimodal understanding, text-to-image generation, and image
editing capabilities. Building on the foundation of the Ovis series, Ovis-U1
incorporates a diffusion-based visual decoder paired with a bidirectional token
refiner, enabling image generation tasks comparable to leading models like
GPT-4o. Unlike some previous models that use a frozen MLLM for generation
tasks, Ovis-U1 utilizes a new unified training approach starting from a
language model. Compared to training solely on understanding or generation
tasks, unified training yields better performance, demonstrating the
enhancement achieved by integrating these two tasks. Ovis-U1 achieves a score
of 69.6 on the OpenCompass Multi-modal Academic Benchmark, surpassing recent
state-of-the-art models such as Ristretto-3B and SAIL-VL-1.5-2B. In
text-to-image generation, it excels with scores of 83.72 and 0.89 on the
DPG-Bench and GenEval benchmarks, respectively. For image editing, it achieves
4.00 and 6.42 on the ImgEdit-Bench and GEdit-Bench-EN, respectively. As the
initial version of the Ovis unified model series, Ovis-U1 pushes the boundaries
of multimodal understanding, generation, and editing.
Ovis-U1, a 3-billion-parameter unified model, integrates multimodal understanding, text-to-image generation, and image editing using a diffusion-based visual decoder and bidirectional token refiner, achieving state-of-the-art performance across various benchmarks.
diffusion-based visual decoder bidirectional token refiner unified training multimodal understanding text-to-image generation image editing OpenCompass Multi-modal Academic Benchmark DPG-Bench GenEval ImgEdit-Bench GEdit-Bench-EN