EditBoard: Towards A Comprehensive Evaluation Benchmark for Text-based Video Editing Models
<|reference_start|>EditBoard: Towards A Comprehensive Evaluation Benchmark for Text-based Video Editing Models: The rapid development of diffusion models has significantly advanced AI-generated content (AIGC), particularly in Text-to-Image (T2I) and Text-to-Video (T2V) generation. Text-based video editing, leveraging these generative capabilities, has emerged as a promising field, enabling precise modifications to videos based on text prompts. Despite the proliferation of innovative video editing models, there is a conspicuous lack of comprehensive evaluation benchmarks that holistically assess these models' performance across various dimensions. Existing evaluations are limited and inconsistent, typically summarizing overall performance with a single score, which obscures models' effectiveness on individual editing tasks. To address this gap, we propose EditBoard, the first comprehensive evaluation benchmark for text-based video editing models. EditBoard encompasses nine automatic metrics across four dimensions, evaluating models on four task categories and introducing three new metrics to assess fidelity. This task-oriented benchmark facilitates objective evaluation by detailing model performance and providing insights into each model's strengths and weaknesses. By open-sourcing EditBoard, we aim to standardize evaluation and advance the development of robust video editing models.<|reference_end|>
@article{chen2024editboard:,
    title={EditBoard: Towards A Comprehensive Evaluation Benchmark for Text-based
  Video Editing Models},
    author={Yupeng Chen, Penglin Chen, Xiaoyu Zhang, Yixian Huang, Qian Xie},
    journal={arXiv preprint arXiv:2409.09668},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.09668},
    primaryClass={cs.CV}
}