SkyReels-A2: Compose Anything in Video Diffusion Transformers
This paper presents SkyReels-A2, a controllable video generation framework
capable of assembling arbitrary visual elements (e.g., characters, objects,
backgrounds) into synthesized videos based on textual prompts while maintaining
strict consistency with reference images for each element. We term this task
elements-to-video (E2V), whose primary challenges lie in preserving the
fidelity of each reference element, ensuring coherent composition of the scene,
and achieving natural outputs. To address these, we first design a
comprehensive data pipeline to construct prompt-reference-video triplets for
model training. Next, we propose a novel image-text joint embedding model to
inject multi-element representations into the generative process, balancing
element-specific consistency with global coherence and text alignment. We also
optimize the inference pipeline for both speed and output stability. Moreover,
we introduce a carefully curated benchmark for systematic evaluation, i.e, A2
Bench. Experiments demonstrate that our framework can generate diverse,
high-quality videos with precise element control. SkyReels-A2 is the first
open-source commercial grade model for the generation of E2V, performing
favorably against advanced closed-source commercial models. We anticipate
SkyReels-A2 will advance creative applications such as drama and virtual
e-commerce, pushing the boundaries of controllable video generation.
67ef3dfae8b932ae7a832950
Zhengcong Fei
admin_assigned
2025-04-04T07:19:16.548Z
617ba1820e4237bd1731b867
/avatars/f9de06363e64bddd7dc977e96e85df8a.svg
zhengcong fei
user
onion
67ef3dfae8b932ae7a832951
Debang Li
admin_assigned
2025-04-04T07:19:27.042Z
65dc3a850af7e21ba40e939f
/avatars/e129c64617675edd05d4317d39604318.svg
Li
user
Debang
67ef3dfae8b932ae7a832952
Di Qiu
admin_assigned
2025-04-04T07:19:41.458Z
65bef422fdb8d33cefeaccc3
/avatars/d40b0d7dda21fa1a68c291d11bc357ec.svg
Qiu Di
user
diqiu7
67ef3dfae8b932ae7a832953
Jiahua Wang
67ef3dfae8b932ae7a832954
Yikun Dou
67ef3dfae8b932ae7a832955
Rui Wang
admin_assigned
2025-04-04T07:20:11.206Z
62e0f1314db2175cd270ad08
/avatars/1d3d6af6c63557f4abf0484e028fa942.svg
Rui Wang
user
ruiwang
67ef3dfae8b932ae7a832956
Jingtao Xu
admin_assigned
2025-04-04T07:20:20.880Z
666a674967c686801acf25bb
/avatars/c1f3edd63fd378dfb555e6413a966932.svg
jingtao xu
user
raul678
67ef3dfae8b932ae7a832957
Mingyuan Fan
admin_assigned
2025-04-04T07:20:32.597Z
634672bfb7b4e71c7f45360f
/avatars/4b646fc3e271be90b9ec619d42ce3e99.svg
Fan Mingyuan
user
MichaelFan
67ef3dfae8b932ae7a832958
Guibin Chen
67ef3dfae8b932ae7a832959
Yang Li
67ef3dfae8b932ae7a83295a
Yahui Zhou
SkyReels-A2: Compose Anything in Video Diffusion Transformers
This paper presents SkyReels-A2, a controllable video generation framework
capable of assembling arbitrary visual elements (e.g., characters, objects,
backgrounds) into synthesized videos based on textual prompts while maintaining
strict consistency with reference images for each element. We term this task
elements-to-video (E2V), whose primary challenges lie in preserving the
fidelity of each reference element, ensuring coherent composition of the scene,
and achieving natural outputs. To address these, we first design a
comprehensive data pipeline to construct prompt-reference-video triplets for
model training. Next, we propose a novel image-text joint embedding model to
inject multi-element representations into the generative process, balancing
element-specific consistency with global coherence and text alignment. We also
optimize the inference pipeline for both speed and output stability. Moreover,
we introduce a carefully curated benchmark for systematic evaluation, i.e, A2
Bench. Experiments demonstrate that our framework can generate diverse,
high-quality videos with precise element control. SkyReels-A2 is the first
open-source commercial grade model for the generation of E2V, performing
favorably against advanced closed-source commercial models. We anticipate
SkyReels-A2 will advance creative applications such as drama and virtual
e-commerce, pushing the boundaries of controllable video generation.
SkyReels-A2, an open-source framework, generates high-quality, element-controlled videos from textual prompts using a novel image-text embedding model, optimized inference pipeline, and A2 Bench for systematic evaluation.
elements-to-video (E2V) image-text joint embedding generative process prompt-reference-video triplets output stability A2 Bench