title: T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video
    Generation
summary: Text-to-video (T2V) generation models have advanced significantly, yet their
  ability to compose different objects, attributes, actions, and motions into a
  video remains unexplored. Previous text-to-video benchmarks also neglect this
  important ability for evaluation. In this work, we conduct the first systematic
  study on compositional text-to-video generation. We propose T2V-CompBench, the
  first benchmark tailored for compositional text-to-video generation.
  T2V-CompBench encompasses diverse aspects of compositionality, including
  consistent attribute binding, dynamic attribute binding, spatial relationships,
  motion binding, action binding, object interactions, and generative numeracy.
  We further carefully design evaluation metrics of MLLM-based metrics,
  detection-based metrics, and tracking-based metrics, which can better reflect
  the compositional text-to-video generation quality of seven proposed categories
  with 700 text prompts. The effectiveness of the proposed metrics is verified by
  correlation with human evaluations. We also benchmark various text-to-video
  generative models and conduct in-depth analysis across different models and
  different compositional categories. We find that compositional text-to-video
  generation is highly challenging for current models, and we hope that our
  attempt will shed light on future research in this direction.
paper_authors._id: 66a05d78cf52b98e1d286375
paper_authors.name: Kaiyue Sun
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-24T07:26:33.664Z
paper_authors._id: 63640ce5ff4b318d1b7b6f5c
paper_authors.avatarUrl: /avatars/9336d1dab1491f3c8e84b1ea287eb891.svg
paper_authors.fullname: Kaiyue Sun
paper_authors.type: user
paper_authors.user: Kaiyue
paper_authors._id: 66a05d78cf52b98e1d286376
paper_authors.name: Kaiyi Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-06T17:51:03.269Z
paper_authors._id: 637cba13b8e573d75be96ea6
paper_authors.avatarUrl: /avatars/5eca230e63d66947b2a05c1ff964a96c.svg
paper_authors.fullname: Nina
paper_authors.type: user
paper_authors.user: NinaKarine
paper_authors._id: 66a05d78cf52b98e1d286377
paper_authors.name: Xian Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-24T08:10:45.554Z
paper_authors._id: 6423f10a3fa01ecba6fca861
paper_authors.avatarUrl: /avatars/e4b4c7dddc5cf962ade45d8bdfa10b1a.svg
paper_authors.fullname: Xian Liu
paper_authors.type: user
paper_authors.user: alvinliu0
paper_authors._id: 66a05d78cf52b98e1d286378
paper_authors.name: Yue Wu
paper_authors._id: 66a05d78cf52b98e1d286379
paper_authors.name: Zihan Xu
paper_authors._id: 66a05d78cf52b98e1d28637a
paper_authors.name: Zhenguo Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-24T08:13:36.623Z
paper_authors._id: 639944bef7c4648254d124f0
paper_authors.avatarUrl: /avatars/c5cbb87cd51cc1341844f67d42c55151.svg
paper_authors.fullname: lzg
paper_authors.type: user
paper_authors.user: lizhenguo
paper_authors._id: 66a05d78cf52b98e1d28637b
paper_authors.name: Xihui Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-24T08:13:43.112Z
paper_authors._id: 65d5ec74cd05bc1eaa125040
paper_authors.avatarUrl: /avatars/2de1b1539a86452c2c89570eeb02f5ab.svg
paper_authors.fullname: Xihui Liu
paper_authors.type: user
paper_authors.user: XihuiLiu
paper_title: T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video
    Generation
paper_summary: Text-to-video (T2V) generation models have advanced significantly, yet their
  ability to compose different objects, attributes, actions, and motions into a
  video remains unexplored. Previous text-to-video benchmarks also neglect this
  important ability for evaluation. In this work, we conduct the first systematic
  study on compositional text-to-video generation. We propose T2V-CompBench, the
  first benchmark tailored for compositional text-to-video generation.
  T2V-CompBench encompasses diverse aspects of compositionality, including
  consistent attribute binding, dynamic attribute binding, spatial relationships,
  motion binding, action binding, object interactions, and generative numeracy.
  We further carefully design evaluation metrics of MLLM-based metrics,
  detection-based metrics, and tracking-based metrics, which can better reflect
  the compositional text-to-video generation quality of seven proposed categories
  with 700 text prompts. The effectiveness of the proposed metrics is verified by
  correlation with human evaluations. We also benchmark various text-to-video
  generative models and conduct in-depth analysis across different models and
  different compositional categories. We find that compositional text-to-video
  generation is highly challenging for current models, and we hope that our
  attempt will shed light on future research in this direction.
paper_ai_summary: This study introduces T2V-CompBench, a benchmark for evaluating the compositional capabilities of text-to-video models, including metrics for consistent and dynamic attributes, spatial relationships, motion and action binding, object interactions, and generative numeracy.
paper_ai_keywords: text-to-video T2V-CompBench compositional generation MLLM-based metrics detection-based metrics tracking-based metrics compositional categories