title: Deep Video Discovery: Agentic Search with Tool Use for Long-form Video
    Understanding
summary: Long-form video understanding presents significant challenges due to
  extensive temporal-spatial complexity and the difficulty of question answering
  under such extended contexts. While Large Language Models (LLMs) have
  demonstrated considerable advancements in video analysis capabilities and long
  context handling, they continue to exhibit limitations when processing
  information-dense hour-long videos. To overcome such limitations, we propose
  the Deep Video Discovery agent to leverage an agentic search strategy over
  segmented video clips. Different from previous video agents manually designing
  a rigid workflow, our approach emphasizes the autonomous nature of agents. By
  providing a set of search-centric tools on multi-granular video database, our
  DVD agent leverages the advanced reasoning capability of LLM to plan on its
  current observation state, strategically selects tools, formulates appropriate
  parameters for actions, and iteratively refines its internal reasoning in light
  of the gathered information. We perform comprehensive evaluation on multiple
  long video understanding benchmarks that demonstrates the advantage of the
  entire system design. Our DVD agent achieves SOTA performance, significantly
  surpassing prior works by a large margin on the challenging LVBench dataset.
  Comprehensive ablation studies and in-depth tool analyses are also provided,
  yielding insights to further advance intelligent agents tailored for long-form
  video understanding tasks. The code will be released later.
paper_authors._id: 68354eec0830dfc6782ba1c4
paper_authors.name: Xiaoyi Zhang
paper_authors._id: 68354eec0830dfc6782ba1c5
paper_authors.name: Zhaoyang Jia
paper_authors._id: 68354eec0830dfc6782ba1c6
paper_authors.name: Zongyu Guo
paper_authors._id: 68354eec0830dfc6782ba1c7
paper_authors.name: Jiahao Li
paper_authors._id: 68354eec0830dfc6782ba1c8
paper_authors.name: Bin Li
paper_authors._id: 68354eec0830dfc6782ba1c9
paper_authors.name: Houqiang Li
paper_authors._id: 68354eec0830dfc6782ba1ca
paper_authors.name: Yan Lu
paper_title: Deep Video Discovery: Agentic Search with Tool Use for Long-form Video
    Understanding
paper_summary: Long-form video understanding presents significant challenges due to
  extensive temporal-spatial complexity and the difficulty of question answering
  under such extended contexts. While Large Language Models (LLMs) have
  demonstrated considerable advancements in video analysis capabilities and long
  context handling, they continue to exhibit limitations when processing
  information-dense hour-long videos. To overcome such limitations, we propose
  the Deep Video Discovery agent to leverage an agentic search strategy over
  segmented video clips. Different from previous video agents manually designing
  a rigid workflow, our approach emphasizes the autonomous nature of agents. By
  providing a set of search-centric tools on multi-granular video database, our
  DVD agent leverages the advanced reasoning capability of LLM to plan on its
  current observation state, strategically selects tools, formulates appropriate
  parameters for actions, and iteratively refines its internal reasoning in light
  of the gathered information. We perform comprehensive evaluation on multiple
  long video understanding benchmarks that demonstrates the advantage of the
  entire system design. Our DVD agent achieves SOTA performance, significantly
  surpassing prior works by a large margin on the challenging LVBench dataset.
  Comprehensive ablation studies and in-depth tool analyses are also provided,
  yielding insights to further advance intelligent agents tailored for long-form
  video understanding tasks. The code will be released later.
paper_ai_summary: The Deep Video Discovery agent uses an autonomous agentic search strategy with large language models to overcome limitations in long-form video understanding, achieving state-of-the-art results on benchmarks like LVBench.
paper_ai_keywords: Deep Video Discovery agent agentic search strategy segmented video clips multi-granular video database advanced reasoning capability LLM autonomous nature observation state search-centric tools internal reasoning long video understanding benchmarks LVBench ablation studies tool analyses