Bootstrapping Referring Multi-Object Tracking
<|reference_start|>Bootstrapping Referring Multi-Object Tracking: Referring multi-object tracking (RMOT) aims at detecting and tracking multiple objects following human instruction represented by a natural language expression. Existing RMOT benchmarks are usually formulated through manual annotations, integrated with static regulations. This approach results in a dearth of notable diversity and a constrained scope of implementation. In this work, our key idea is to bootstrap the task of referring multi-object tracking by introducing discriminative language words as much as possible. In specific, we first develop Refer-KITTI into a large-scale dataset, named Refer-KITTI-V2. It starts with 2,719 manual annotations, addressing the issue of class imbalance and introducing more keywords to make it closer to real-world scenarios compared to Refer-KITTI. They are further expanded to a total of 9,758 annotations by prompting large language models, which create 617 different words, surpassing previous RMOT benchmarks. In addition, the end-to-end framework in RMOT is also bootstrapped by a simple yet elegant temporal advancement strategy, which achieves better performance than previous approaches. The source code and dataset is available at https://github.com/zyn213/TempRMOT.<|reference_end|>
@article{zhang2024bootstrapping,
    title={Bootstrapping Referring Multi-Object Tracking},
    author={Yani Zhang and Dongming Wu and Wencheng Han and Xingping Dong},
    journal={arXiv preprint arXiv:2406.05039},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.05039},
    primaryClass={cs.CV cs.CL}
}