Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Input
<|reference_start|>Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Input: Rapid advancements have been made in extending Large Language Models (LLMs) to Large Multi-modal Models (LMMs). However, extending input modality of LLMs to video data remains a challenging endeavor, especially for long videos. Due to insufficient access to large-scale high-quality video data and the excessive compression of visual features, current methods exhibit limitations in effectively processing long videos. In this paper, we introduce Kangaroo, a powerful Video LMM aimed at addressing these challenges. Confronted with issue of inadequate training data, we develop a data curation system to build a large-scale dataset with high-quality annotations for vision-language pre-training and instruction tuning. In addition, we design a curriculum training pipeline with gradually increasing resolution and number of input frames to accommodate long videos. Evaluation results demonstrate that, with 8B parameters, Kangaroo achieves state-of-the-art performance across a variety of video understanding benchmarks while exhibiting competitive results on others. Particularly, on benchmarks specialized for long videos, Kangaroo excels some larger models with over 10B parameters and proprietary models.<|reference_end|>
@article{liu2024kangaroo:,
    title={Kangaroo: A Powerful Video-Language Model Supporting Long-context Video
  Input},
    author={Jiajun Liu, Yibing Wang, Hanghang Ma, Xiaoping Wu, Xiaoqi Ma, Xiaoming
  Wei, Jianbin Jiao, Enhua Wu, Jie Hu},
    journal={arXiv preprint arXiv:2408.15542},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.15542},
    primaryClass={cs.CV cs.AI cs.MM}
}