VideoChat: Chat-Centric Video Understanding
In this study, we initiate an exploration into video understanding by
introducing VideoChat, an end-to-end chat-centric video understanding system.
It integrates video foundation models and large language models via a learnable
neural interface, excelling in spatiotemporal reasoning, event localization,
and causal relationship inference. To instructively tune this system, we
propose a video-centric instruction dataset, composed of thousands of videos
matched with detailed descriptions and conversations. This dataset emphasizes
spatiotemporal reasoning and causal relationships, providing a valuable asset
for training chat-centric video understanding systems. Preliminary qualitative
experiments reveal our system's potential across a broad spectrum of video
applications and set the standard for future research. Access our code and data
at https://github.com/OpenGVLab/Ask-Anything
645c4d10a7db8b98eb397d1e
KunChang Li
admin_assigned
2023-06-02T14:48:06.848Z
61fb81006374891646732f37
https://cdn-avatars.huggingface.co/v1/production/uploads/1643872995181-61fb81006374891646732f37.jpeg
Kunchang Li
user
Andy1621
645c4d10a7db8b98eb397d1f
Yinan He
claimed_verified
2023-07-17T06:49:29.393Z
62aafa49f29ff279b51f0182
https://cdn-avatars.huggingface.co/v1/production/uploads/62aafa49f29ff279b51f0182/rQx8QFQGOY2qIhqJ8zSRj.jpeg
yinanhe
user
ynhe
645c4d10a7db8b98eb397d20
Yi Wang
645c4d10a7db8b98eb397d21
Yizhuo Li
claimed_verified
2025-06-04T09:07:31.789Z
630b094f8b327c7b8b94d24c
https://cdn-avatars.huggingface.co/v1/production/uploads/630b094f8b327c7b8b94d24c/Fd0ugLOHJZIi8oUZScOmX.jpeg
Yizhuo Li
user
liyz
645c4d10a7db8b98eb397d22
Wenhai Wang
admin_assigned
2023-06-02T14:48:54.344Z
644f58e6f8b353c94914c7e0
/avatars/16c7e80aafbea9a52cfc309068eedc9a.svg
wangwenhai
user
yicixi
645c4d10a7db8b98eb397d23
Ping Luo
645c4d10a7db8b98eb397d24
Yali Wang
645c4d10a7db8b98eb397d25
Limin Wang
admin_assigned
2023-06-02T14:49:37.149Z
62c77f4352d8ae531f5511f9
/avatars/50198ccb02ccd286975a4613fbabee28.svg
Limin Wang
user
lmwang
645c4d10a7db8b98eb397d26
Yu Qiao
VideoChat: Chat-Centric Video Understanding
In this study, we initiate an exploration into video understanding by
introducing VideoChat, an end-to-end chat-centric video understanding system.
It integrates video foundation models and large language models via a learnable
neural interface, excelling in spatiotemporal reasoning, event localization,
and causal relationship inference. To instructively tune this system, we
propose a video-centric instruction dataset, composed of thousands of videos
matched with detailed descriptions and conversations. This dataset emphasizes
spatiotemporal reasoning and causal relationships, providing a valuable asset
for training chat-centric video understanding systems. Preliminary qualitative
experiments reveal our system's potential across a broad spectrum of video
applications and set the standard for future research. Access our code and data
at https://github.com/OpenGVLab/Ask-Anything
VideoChat combines video foundation models and large language models with a learnable neural interface for tasks such as spatiotemporal reasoning and causal relationship inference, supported by a new video-centric instruction dataset.
video foundation models large language models neural interface spatiotemporal reasoning event localization causal relationship inference instructional tuning video-centric instruction dataset