AMD-Hummingbird: Towards an Efficient Text-to-Video Model
Text-to-Video (T2V) generation has attracted significant attention for its
ability to synthesize realistic videos from textual descriptions. However,
existing models struggle to balance computational efficiency and high visual
quality, particularly on resource-limited devices, e.g.,iGPUs and mobile
phones. Most prior work prioritizes visual fidelity while overlooking the need
for smaller, more efficient models suitable for real-world deployment. To
address this challenge, we propose a lightweight T2V framework, termed
Hummingbird, which prunes existing models and enhances visual quality through
visual feedback learning. Our approach reduces the size of the U-Net from 1.4
billion to 0.7 billion parameters, significantly improving efficiency while
preserving high-quality video generation. Additionally, we introduce a novel
data processing pipeline that leverages Large Language Models (LLMs) and Video
Quality Assessment (VQA) models to enhance the quality of both text prompts and
video data. To support user-driven training and style customization, we
publicly release the full training code, including data processing and model
training. Extensive experiments show that our method achieves a 31X speedup
compared to state-of-the-art models such as VideoCrafter2, while also attaining
the highest overall score on VBench. Moreover, our method supports the
generation of videos with up to 26 frames, addressing the limitations of
existing U-Net-based methods in long video generation. Notably, the entire
training process requires only four GPUs, yet delivers performance competitive
with existing leading methods. Hummingbird presents a practical and efficient
solution for T2V generation, combining high performance, scalability, and
flexibility for real-world applications.
67e22d5236076dc847989434
Takashi Isobe
67e22d5236076dc847989435
He Cui
67e22d5236076dc847989436
Dong Zhou
67e22d5236076dc847989437
Mengmeng Ge
admin_assigned
2025-03-25T09:30:24.973Z
6463685fd2044cd1d7c74b81
/avatars/334637e2d63efb7cc2129fec6ea54725.svg
gemengmeng
user
gemengmeng
67e22d5236076dc847989438
Dong Li
67e22d5236076dc847989439
Emad Barsoum
admin_assigned
2025-03-25T09:30:09.267Z
65adc9d086f88a686be41215
https://cdn-avatars.huggingface.co/v1/production/uploads/65adc9d086f88a686be41215/xizVHuZPkE0Gu8_ulx0Fm.jpeg
Emad Barsoum
user
ebarsoum
AMD-Hummingbird: Towards an Efficient Text-to-Video Model
Text-to-Video (T2V) generation has attracted significant attention for its
ability to synthesize realistic videos from textual descriptions. However,
existing models struggle to balance computational efficiency and high visual
quality, particularly on resource-limited devices, e.g.,iGPUs and mobile
phones. Most prior work prioritizes visual fidelity while overlooking the need
for smaller, more efficient models suitable for real-world deployment. To
address this challenge, we propose a lightweight T2V framework, termed
Hummingbird, which prunes existing models and enhances visual quality through
visual feedback learning. Our approach reduces the size of the U-Net from 1.4
billion to 0.7 billion parameters, significantly improving efficiency while
preserving high-quality video generation. Additionally, we introduce a novel
data processing pipeline that leverages Large Language Models (LLMs) and Video
Quality Assessment (VQA) models to enhance the quality of both text prompts and
video data. To support user-driven training and style customization, we
publicly release the full training code, including data processing and model
training. Extensive experiments show that our method achieves a 31X speedup
compared to state-of-the-art models such as VideoCrafter2, while also attaining
the highest overall score on VBench. Moreover, our method supports the
generation of videos with up to 26 frames, addressing the limitations of
existing U-Net-based methods in long video generation. Notably, the entire
training process requires only four GPUs, yet delivers performance competitive
with existing leading methods. Hummingbird presents a practical and efficient
solution for T2V generation, combining high performance, scalability, and
flexibility for real-world applications.
Hummingbird, a lightweight T2V framework, enhances visual quality and efficiency through model pruning and visual feedback learning, achieving high performance with low computational cost.
U-Net large language models video quality assessment visual feedback learning