What Matters in Detecting AI-Generated Videos like Sora?
Recent advancements in diffusion-based video generation have showcased
remarkable results, yet the gap between synthetic and real-world videos remains
under-explored. In this study, we examine this gap from three fundamental
perspectives: appearance, motion, and geometry, comparing real-world videos
with those generated by a state-of-the-art AI model, Stable Video Diffusion. To
achieve this, we train three classifiers using 3D convolutional networks, each
targeting distinct aspects: vision foundation model features for appearance,
optical flow for motion, and monocular depth for geometry. Each classifier
exhibits strong performance in fake video detection, both qualitatively and
quantitatively. This indicates that AI-generated videos are still easily
detectable, and a significant gap between real and fake videos persists.
Furthermore, utilizing the Grad-CAM, we pinpoint systematic failures of
AI-generated videos in appearance, motion, and geometry. Finally, we propose an
Ensemble-of-Experts model that integrates appearance, optical flow, and depth
information for fake video detection, resulting in enhanced robustness and
generalization ability. Our model is capable of detecting videos generated by
Sora with high accuracy, even without exposure to any Sora videos during
training. This suggests that the gap between real and fake videos can be
generalized across various video generative models. Project page:
https://justin-crchang.github.io/3DCNNDetection.github.io/
66851684fb78936cdbfd16d2
Chirui Chang
claimed_verified
2024-07-03T09:47:54.002Z
64a7fe360f4bad61a44aafbe
https://cdn-avatars.huggingface.co/v1/production/uploads/64a7fe360f4bad61a44aafbe/ocs7Qf3AT5q1S3WjGDYDh.jpeg
Chirui Chang
user
JustinSheung
66851684fb78936cdbfd16d3
Zhengzhe Liu
66851684fb78936cdbfd16d4
Xiaoyang Lyu
66851684fb78936cdbfd16d5
Xiaojuan Qi
claimed_verified
2025-07-15T19:15:05.086Z
6875266f9cd3191dfddc7071
/avatars/64c581910833b111e9a7bae5b8740229.svg
xiaojuan qi
user
xjqi
What Matters in Detecting AI-Generated Videos like Sora?
Recent advancements in diffusion-based video generation have showcased
remarkable results, yet the gap between synthetic and real-world videos remains
under-explored. In this study, we examine this gap from three fundamental
perspectives: appearance, motion, and geometry, comparing real-world videos
with those generated by a state-of-the-art AI model, Stable Video Diffusion. To
achieve this, we train three classifiers using 3D convolutional networks, each
targeting distinct aspects: vision foundation model features for appearance,
optical flow for motion, and monocular depth for geometry. Each classifier
exhibits strong performance in fake video detection, both qualitatively and
quantitatively. This indicates that AI-generated videos are still easily
detectable, and a significant gap between real and fake videos persists.
Furthermore, utilizing the Grad-CAM, we pinpoint systematic failures of
AI-generated videos in appearance, motion, and geometry. Finally, we propose an
Ensemble-of-Experts model that integrates appearance, optical flow, and depth
information for fake video detection, resulting in enhanced robustness and
generalization ability. Our model is capable of detecting videos generated by
Sora with high accuracy, even without exposure to any Sora videos during
training. This suggests that the gap between real and fake videos can be
generalized across various video generative models. Project page:
https://justin-crchang.github.io/3DCNNDetection.github.io/
An ensemble of 3D convolutional network classifiers detects fake videos by analyzing appearance, motion, and geometry, revealing persistent gaps between real and synthetic videos.
diffusion-based video generation 3D convolutional networks vision foundation model features optical flow monocular depth Grad-CAM fake video detection Ensemble-of-Experts model Sora