UniReal: Universal Image Generation and Editing via Learning Real-world
  Dynamics
We introduce UniReal, a unified framework designed to address various image
generation and editing tasks. Existing solutions often vary by tasks, yet share
fundamental principles: preserving consistency between inputs and outputs while
capturing visual variations. Inspired by recent video generation models that
effectively balance consistency and variation across frames, we propose a
unifying approach that treats image-level tasks as discontinuous video
generation. Specifically, we treat varying numbers of input and output images
as frames, enabling seamless support for tasks such as image generation,
editing, customization, composition, etc. Although designed for image-level
tasks, we leverage videos as a scalable source for universal supervision.
UniReal learns world dynamics from large-scale videos, demonstrating advanced
capability in handling shadows, reflections, pose variation, and object
interaction, while also exhibiting emergent capability for novel applications.
675904147ab15d62b77e40e0
Xi Chen
admin_assigned
2024-12-11T10:19:38.341Z
644a1b6401e18bf93a6f45c1
https://cdn-avatars.huggingface.co/v1/production/uploads/644a1b6401e18bf93a6f45c1/P0i_CgCrIzOS2tYRlxoE9.png
xichen
user
xichenhku
675904147ab15d62b77e40e1
Zhifei Zhang
675904147ab15d62b77e40e2
He Zhang
675904147ab15d62b77e40e3
Yuqian Zhou
675904147ab15d62b77e40e4
Soo Ye Kim
675904147ab15d62b77e40e5
Qing Liu
675904147ab15d62b77e40e6
Yijun Li
675904147ab15d62b77e40e7
Jianming Zhang
admin_assigned
2024-12-11T10:21:15.636Z
632d52047bad38a82f75daee
/avatars/56cc9b5dea95408a83cb80ee8c5565bf.svg
Jianming Zhang
user
jimmie33
675904147ab15d62b77e40e8
Nanxuan Zhao
675904147ab15d62b77e40e9
Yilin Wang
675904147ab15d62b77e40ea
Hui Ding
675904147ab15d62b77e40eb
Zhe Lin
675904147ab15d62b77e40ec
Hengshuang Zhao
UniReal: Universal Image Generation and Editing via Learning Real-world
  Dynamics
We introduce UniReal, a unified framework designed to address various image
generation and editing tasks. Existing solutions often vary by tasks, yet share
fundamental principles: preserving consistency between inputs and outputs while
capturing visual variations. Inspired by recent video generation models that
effectively balance consistency and variation across frames, we propose a
unifying approach that treats image-level tasks as discontinuous video
generation. Specifically, we treat varying numbers of input and output images
as frames, enabling seamless support for tasks such as image generation,
editing, customization, composition, etc. Although designed for image-level
tasks, we leverage videos as a scalable source for universal supervision.
UniReal learns world dynamics from large-scale videos, demonstrating advanced
capability in handling shadows, reflections, pose variation, and object
interaction, while also exhibiting emergent capability for novel applications.
UniReal treats image generation and editing tasks as discontinuous video generation to capture visual variations and consistency, learning from large-scale video data.
unifying approach discontinuous video generation image-level tasks universal supervision world dynamics