title: Vidi: Large Multimodal Models for Video Understanding and Editing
summary: Humans naturally share information with those they are connected to, and
  video has become one of the dominant mediums for communication and expression
  on the Internet. To support the creation of high-quality large-scale video
  content, a modern pipeline requires a comprehensive understanding of both the
  raw input materials (e.g., the unedited footage captured by cameras) and the
  editing components (e.g., visual effects). In video editing scenarios, models
  must process multiple modalities (e.g., vision, audio, text) with strong
  background knowledge and handle flexible input lengths (e.g., hour-long raw
  videos), which poses significant challenges for traditional models. In this
  report, we introduce Vidi, a family of Large Multimodal Models (LMMs) for a
  wide range of video understand editing scenarios. The first release focuses on
  temporal retrieval, i.e., identifying the time ranges within the input videos
  corresponding to a given text query, which plays a critical role in intelligent
  editing. The model is capable of processing hour-long videos with strong
  temporal understanding capability, e.g., retrieve time ranges for certain
  queries. To support a comprehensive evaluation in real-world scenarios, we also
  present the VUE-TR benchmark, which introduces five key advancements. 1) Video
  duration: significantly longer than existing temporal retrival datasets, 2)
  Audio support: includes audio-based queries, 3) Query format: diverse query
  lengths/formats, 4) Annotation quality: ground-truth time ranges are manually
  annotated. 5) Evaluation metric: a refined IoU metric to support evaluation
  over multiple time ranges. Remarkably, Vidi significantly outperforms leading
  proprietary models, e.g., GPT-4o and Gemini, on the temporal retrieval task,
  indicating its superiority in video editing scenarios.
paper_authors._id: 680846defa5a6cc6bd9d2cf3
paper_authors.name: Vidi Team
paper_authors._id: 680846defa5a6cc6bd9d2cf4
paper_authors.name: Celong Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-23T13:52:25.362Z
paper_authors._id: 64364ba31adb261e94e0e5b4
paper_authors.avatarUrl: /avatars/cb6ef2155d88b1e64c4ff36bd3a8f255.svg
paper_authors.fullname: Celong Liu
paper_authors.type: user
paper_authors.user: goddice
paper_authors._id: 680846defa5a6cc6bd9d2cf5
paper_authors.name: Chia-Wen Kuo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-23T13:52:32.272Z
paper_authors._id: 64eea9ff9884f64fd27544f5
paper_authors.avatarUrl: /avatars/65aac158a55f16c82605f4dafd59b3ea.svg
paper_authors.fullname: Chia-Wen Kuo
paper_authors.type: user
paper_authors.user: cwkuo
paper_authors._id: 680846defa5a6cc6bd9d2cf6
paper_authors.name: Dawei Du
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-23T08:28:04.152Z
paper_authors._id: 6476af4402fc644c810b29a2
paper_authors.avatarUrl: /avatars/68aefabe6b000443f4601137e6672187.svg
paper_authors.fullname: Dawei Du
paper_authors.type: user
paper_authors.user: daviddousa
paper_authors._id: 680846defa5a6cc6bd9d2cf7
paper_authors.name: Fan Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-06T08:34:33.789Z
paper_authors._id: 65c2a2f5b911dda8cb3ff13b
paper_authors.avatarUrl: /avatars/29ad2ed1433be78e1f1ceefbe7463a4e.svg
paper_authors.fullname: Fan Chen
paper_authors.type: user
paper_authors.user: fanchenBYT
paper_authors._id: 680846defa5a6cc6bd9d2cf8
paper_authors.name: Guang Chen
paper_authors._id: 680846defa5a6cc6bd9d2cf9
paper_authors.name: Jiamin Yuan
paper_authors._id: 680846defa5a6cc6bd9d2cfa
paper_authors.name: Lingxi Zhang
paper_authors._id: 680846defa5a6cc6bd9d2cfb
paper_authors.name: Lu Guo
paper_authors._id: 680846defa5a6cc6bd9d2cfc
paper_authors.name: Lusha Li
paper_authors._id: 680846defa5a6cc6bd9d2cfd
paper_authors.name: Longyin Wen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-23T13:53:13.545Z
paper_authors._id: 644df7eacfb40c94eae71186
paper_authors.avatarUrl: /avatars/1daa4967efd34d54c59aa95970093dbd.svg
paper_authors.fullname: Longyin Wen
paper_authors.type: user
paper_authors.user: lionwen
paper_authors._id: 680846defa5a6cc6bd9d2cfe
paper_authors.name: Qingyu Chen
paper_authors._id: 680846defa5a6cc6bd9d2cff
paper_authors.name: Rachel Deng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-23T13:53:24.615Z
paper_authors._id: 6509f43b1aece923f200ba50
paper_authors.avatarUrl: /avatars/d6f7c96222ae8442291d193a1aa768c3.svg
paper_authors.fullname: Rachel Deng
paper_authors.type: user
paper_authors.user: racheldeng
paper_authors._id: 680846defa5a6cc6bd9d2d00
paper_authors.name: Sijie Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-23T13:53:31.960Z
paper_authors._id: 642ee3316b75297bdcccc17e
paper_authors.avatarUrl: /avatars/2ca159e38c64b203835cd0086cf5abb1.svg
paper_authors.fullname: zhusijie
paper_authors.type: user
paper_authors.user: sijiezhu
paper_authors._id: 680846defa5a6cc6bd9d2d01
paper_authors.name: Stuart Siew
paper_authors._id: 680846defa5a6cc6bd9d2d02
paper_authors.name: Tong Jin
paper_authors._id: 680846defa5a6cc6bd9d2d03
paper_authors.name: Wei Lu
paper_authors._id: 680846defa5a6cc6bd9d2d04
paper_authors.name: Wen Zhong
paper_authors._id: 680846defa5a6cc6bd9d2d05
paper_authors.name: Xiaohui Shen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-23T13:53:43.973Z
paper_authors._id: 6430aa1b32a732121cd81f98
paper_authors.avatarUrl: /avatars/5419f8d6d4d36fa5ac83e30667b9fd99.svg
paper_authors.fullname: Xiaohui Shen
paper_authors.type: user
paper_authors.user: XiaohuiShen
paper_authors._id: 680846defa5a6cc6bd9d2d06
paper_authors.name: Xin Gu
paper_authors._id: 680846defa5a6cc6bd9d2d07
paper_authors.name: Xing Mei
paper_authors._id: 680846defa5a6cc6bd9d2d08
paper_authors.name: Xueqiong Qu
paper_title: Vidi: Large Multimodal Models for Video Understanding and Editing
paper_summary: Humans naturally share information with those they are connected to, and
  video has become one of the dominant mediums for communication and expression
  on the Internet. To support the creation of high-quality large-scale video
  content, a modern pipeline requires a comprehensive understanding of both the
  raw input materials (e.g., the unedited footage captured by cameras) and the
  editing components (e.g., visual effects). In video editing scenarios, models
  must process multiple modalities (e.g., vision, audio, text) with strong
  background knowledge and handle flexible input lengths (e.g., hour-long raw
  videos), which poses significant challenges for traditional models. In this
  report, we introduce Vidi, a family of Large Multimodal Models (LMMs) for a
  wide range of video understand editing scenarios. The first release focuses on
  temporal retrieval, i.e., identifying the time ranges within the input videos
  corresponding to a given text query, which plays a critical role in intelligent
  editing. The model is capable of processing hour-long videos with strong
  temporal understanding capability, e.g., retrieve time ranges for certain
  queries. To support a comprehensive evaluation in real-world scenarios, we also
  present the VUE-TR benchmark, which introduces five key advancements. 1) Video
  duration: significantly longer than existing temporal retrival datasets, 2)
  Audio support: includes audio-based queries, 3) Query format: diverse query
  lengths/formats, 4) Annotation quality: ground-truth time ranges are manually
  annotated. 5) Evaluation metric: a refined IoU metric to support evaluation
  over multiple time ranges. Remarkably, Vidi significantly outperforms leading
  proprietary models, e.g., GPT-4o and Gemini, on the temporal retrieval task,
  indicating its superiority in video editing scenarios.
paper_ai_summary: Vidi, a family of Large Multimodal Models, excels in temporal retrieval for video editing by processing long, multimodal video content and outperforming proprietary models on the VUE-TR benchmark.
paper_ai_keywords: LMMs temporal retrieval multimodal models video editing temporal understanding VUE-TR benchmark