ETVA: Evaluation of Text-to-Video Alignment via Fine-grained Question
  Generation and Answering
Precisely evaluating semantic alignment between text prompts and generated
videos remains a challenge in Text-to-Video (T2V) Generation. Existing
text-to-video alignment metrics like CLIPScore only generate coarse-grained
scores without fine-grained alignment details, failing to align with human
preference. To address this limitation, we propose ETVA, a novel Evaluation
method of Text-to-Video Alignment via fine-grained question generation and
answering. First, a multi-agent system parses prompts into semantic scene
graphs to generate atomic questions. Then we design a knowledge-augmented
multi-stage reasoning framework for question answering, where an auxiliary LLM
first retrieves relevant common-sense knowledge (e.g., physical laws), and then
video LLM answers the generated questions through a multi-stage reasoning
mechanism. Extensive experiments demonstrate that ETVA achieves a Spearman's
correlation coefficient of 58.47, showing a much higher correlation with human
judgment than existing metrics which attain only 31.0. We also construct a
comprehensive benchmark specifically designed for text-to-video alignment
evaluation, featuring 2k diverse prompts and 12k atomic questions spanning 10
categories. Through a systematic evaluation of 15 existing text-to-video
models, we identify their key capabilities and limitations, paving the way for
next-generation T2V generation.
67e0d31f151ca9ed92898fff
Kaisi Guan
claimed_verified
2025-03-24T08:06:57.786Z
63bbf071d8d676a2299c7d0b
/avatars/4bb1c86ef8651c75b9761afee2865267.svg
Guan
user
Guan123
67e0d31f151ca9ed92899000
Zhengfeng Lai
admin_assigned
2025-03-24T12:49:46.588Z
66b5295f83425904fa7a1a6a
/avatars/a35568fb933ceef7451bd88fb3d5ab17.svg
Zhengfeng Lai
user
jefflai
67e0d31f151ca9ed92899001
Yuchong Sun
admin_assigned
2025-03-24T12:49:53.004Z
645e0f1aa45b4182d7f69176
/avatars/6b87f203e781b6ec734d2b58084ab013.svg
yuchong sun
user
ycsun
67e0d31f151ca9ed92899002
Peng Zhang
67e0d31f151ca9ed92899003
Wei Liu
67e0d31f151ca9ed92899004
Kieran Liu
67e0d31f151ca9ed92899005
Meng Cao
67e0d31f151ca9ed92899006
Ruihua Song
ETVA: Evaluation of Text-to-Video Alignment via Fine-grained Question
  Generation and Answering
Precisely evaluating semantic alignment between text prompts and generated
videos remains a challenge in Text-to-Video (T2V) Generation. Existing
text-to-video alignment metrics like CLIPScore only generate coarse-grained
scores without fine-grained alignment details, failing to align with human
preference. To address this limitation, we propose ETVA, a novel Evaluation
method of Text-to-Video Alignment via fine-grained question generation and
answering. First, a multi-agent system parses prompts into semantic scene
graphs to generate atomic questions. Then we design a knowledge-augmented
multi-stage reasoning framework for question answering, where an auxiliary LLM
first retrieves relevant common-sense knowledge (e.g., physical laws), and then
video LLM answers the generated questions through a multi-stage reasoning
mechanism. Extensive experiments demonstrate that ETVA achieves a Spearman's
correlation coefficient of 58.47, showing a much higher correlation with human
judgment than existing metrics which attain only 31.0. We also construct a
comprehensive benchmark specifically designed for text-to-video alignment
evaluation, featuring 2k diverse prompts and 12k atomic questions spanning 10
categories. Through a systematic evaluation of 15 existing text-to-video
models, we identify their key capabilities and limitations, paving the way for
next-generation T2V generation.
ETVA, a novel evaluation method for text-to-video alignment, uses fine-grained question generation and answering to provide detailed alignment, significantly outperforming existing metrics like CLIPScore.
ETVA Evaluation method Text-to-Video Alignment fine-grained question generation question answering multi-agent system semantic scene graphs knowledge-augmented multi-stage reasoning auxiliary LLM video LLM multi-stage reasoning Spearman's correlation coefficient text-to-video alignment benchmark diverse prompts atomic questions text-to-video models