Magic 1-For-1: Generating One Minute Video Clips within One Minute
In this technical report, we present Magic 1-For-1 (Magic141), an efficient
video generation model with optimized memory consumption and inference latency.
The key idea is simple: factorize the text-to-video generation task into two
separate easier tasks for diffusion step distillation, namely text-to-image
generation and image-to-video generation. We verify that with the same
optimization algorithm, the image-to-video task is indeed easier to converge
over the text-to-video task. We also explore a bag of optimization tricks to
reduce the computational cost of training the image-to-video (I2V) models from
three aspects: 1) model convergence speedup by using a multi-modal prior
condition injection; 2) inference latency speed up by applying an adversarial
step distillation, and 3) inference memory cost optimization with parameter
sparsification. With those techniques, we are able to generate 5-second video
clips within 3 seconds. By applying a test time sliding window, we are able to
generate a minute-long video within one minute with significantly improved
visual quality and motion dynamics, spending less than 1 second for generating
1 second video clips on average. We conduct a series of preliminary
explorations to find out the optimal tradeoff between computational cost and
video quality during diffusion step distillation and hope this could be a good
foundation model for open-source explorations. The code and the model weights
are available at https://github.com/DA-Group-PKU/Magic-1-For-1.
67ac23166def89f9aae56abd
Hongwei Yi
67ac23166def89f9aae56abe
Shitong Shao
claimed_verified
2025-03-19T09:48:07.681Z
64aa48245fc663fee24125be
https://cdn-avatars.huggingface.co/v1/production/uploads/64aa48245fc663fee24125be/8Me4rbY3XdY48p9QyEpaA.jpeg
ShanShan
user
sst12345
67ac23166def89f9aae56abf
Tian Ye
claimed_verified
2025-02-12T09:16:12.141Z
66015e8aa4d296af07de538e
/avatars/a1295c631cc2646282c545859975ce4c.svg
Owen
user
Owen777
67ac23166def89f9aae56ac0
Jiantong Zhao
67ac23166def89f9aae56ac1
Qingyu Yin
67ac23166def89f9aae56ac2
Michael Lingelbach
67ac23166def89f9aae56ac3
Li Yuan
claimed_verified
2025-04-04T07:10:15.479Z
66135a5e50350afe76beebce
/avatars/370a4b83949355feb050c2cb0425c264.svg
yl2488
user
yl2488
67ac23166def89f9aae56ac4
Yonghong Tian
67ac23166def89f9aae56ac5
Enze Xie
67ac23166def89f9aae56ac6
Daquan Zhou
Magic 1-For-1: Generating One Minute Video Clips within One Minute
In this technical report, we present Magic 1-For-1 (Magic141), an efficient
video generation model with optimized memory consumption and inference latency.
The key idea is simple: factorize the text-to-video generation task into two
separate easier tasks for diffusion step distillation, namely text-to-image
generation and image-to-video generation. We verify that with the same
optimization algorithm, the image-to-video task is indeed easier to converge
over the text-to-video task. We also explore a bag of optimization tricks to
reduce the computational cost of training the image-to-video (I2V) models from
three aspects: 1) model convergence speedup by using a multi-modal prior
condition injection; 2) inference latency speed up by applying an adversarial
step distillation, and 3) inference memory cost optimization with parameter
sparsification. With those techniques, we are able to generate 5-second video
clips within 3 seconds. By applying a test time sliding window, we are able to
generate a minute-long video within one minute with significantly improved
visual quality and motion dynamics, spending less than 1 second for generating
1 second video clips on average. We conduct a series of preliminary
explorations to find out the optimal tradeoff between computational cost and
video quality during diffusion step distillation and hope this could be a good
foundation model for open-source explorations. The code and the model weights
are available at https://github.com/DA-Group-PKU/Magic-1-For-1.
Magic141 efficiently generates high-quality videos by factorizing the text-to-video task into text-to-image and image-to-video components, optimizing for memory, speed, and quality.
diffusion step distillation text-to-image generation image-to-video generation multi-modal prior condition injection adversarial step distillation parameter sparsification test time sliding window