title: AMD-Hummingbird: Towards an Efficient Text-to-Video Model
summary: Text-to-Video (T2V) generation has attracted significant attention for its
  ability to synthesize realistic videos from textual descriptions. However,
  existing models struggle to balance computational efficiency and high visual
  quality, particularly on resource-limited devices, e.g.,iGPUs and mobile
  phones. Most prior work prioritizes visual fidelity while overlooking the need
  for smaller, more efficient models suitable for real-world deployment. To
  address this challenge, we propose a lightweight T2V framework, termed
  Hummingbird, which prunes existing models and enhances visual quality through
  visual feedback learning. Our approach reduces the size of the U-Net from 1.4
  billion to 0.7 billion parameters, significantly improving efficiency while
  preserving high-quality video generation. Additionally, we introduce a novel
  data processing pipeline that leverages Large Language Models (LLMs) and Video
  Quality Assessment (VQA) models to enhance the quality of both text prompts and
  video data. To support user-driven training and style customization, we
  publicly release the full training code, including data processing and model
  training. Extensive experiments show that our method achieves a 31X speedup
  compared to state-of-the-art models such as VideoCrafter2, while also attaining
  the highest overall score on VBench. Moreover, our method supports the
  generation of videos with up to 26 frames, addressing the limitations of
  existing U-Net-based methods in long video generation. Notably, the entire
  training process requires only four GPUs, yet delivers performance competitive
  with existing leading methods. Hummingbird presents a practical and efficient
  solution for T2V generation, combining high performance, scalability, and
  flexibility for real-world applications.
paper_authors._id: 67e22d5236076dc847989434
paper_authors.name: Takashi Isobe
paper_authors._id: 67e22d5236076dc847989435
paper_authors.name: He Cui
paper_authors._id: 67e22d5236076dc847989436
paper_authors.name: Dong Zhou
paper_authors._id: 67e22d5236076dc847989437
paper_authors.name: Mengmeng Ge
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-25T09:30:24.973Z
paper_authors._id: 6463685fd2044cd1d7c74b81
paper_authors.avatarUrl: /avatars/334637e2d63efb7cc2129fec6ea54725.svg
paper_authors.fullname: gemengmeng
paper_authors.type: user
paper_authors.user: gemengmeng
paper_authors._id: 67e22d5236076dc847989438
paper_authors.name: Dong Li
paper_authors._id: 67e22d5236076dc847989439
paper_authors.name: Emad Barsoum
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-25T09:30:09.267Z
paper_authors._id: 65adc9d086f88a686be41215
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65adc9d086f88a686be41215/xizVHuZPkE0Gu8_ulx0Fm.jpeg
paper_authors.fullname: Emad Barsoum
paper_authors.type: user
paper_authors.user: ebarsoum
paper_title: AMD-Hummingbird: Towards an Efficient Text-to-Video Model
paper_summary: Text-to-Video (T2V) generation has attracted significant attention for its
  ability to synthesize realistic videos from textual descriptions. However,
  existing models struggle to balance computational efficiency and high visual
  quality, particularly on resource-limited devices, e.g.,iGPUs and mobile
  phones. Most prior work prioritizes visual fidelity while overlooking the need
  for smaller, more efficient models suitable for real-world deployment. To
  address this challenge, we propose a lightweight T2V framework, termed
  Hummingbird, which prunes existing models and enhances visual quality through
  visual feedback learning. Our approach reduces the size of the U-Net from 1.4
  billion to 0.7 billion parameters, significantly improving efficiency while
  preserving high-quality video generation. Additionally, we introduce a novel
  data processing pipeline that leverages Large Language Models (LLMs) and Video
  Quality Assessment (VQA) models to enhance the quality of both text prompts and
  video data. To support user-driven training and style customization, we
  publicly release the full training code, including data processing and model
  training. Extensive experiments show that our method achieves a 31X speedup
  compared to state-of-the-art models such as VideoCrafter2, while also attaining
  the highest overall score on VBench. Moreover, our method supports the
  generation of videos with up to 26 frames, addressing the limitations of
  existing U-Net-based methods in long video generation. Notably, the entire
  training process requires only four GPUs, yet delivers performance competitive
  with existing leading methods. Hummingbird presents a practical and efficient
  solution for T2V generation, combining high performance, scalability, and
  flexibility for real-world applications.
paper_ai_summary: Hummingbird, a lightweight T2V framework, enhances visual quality and efficiency through model pruning and visual feedback learning, achieving high performance with low computational cost.
paper_ai_keywords: U-Net large language models video quality assessment visual feedback learning