MLS-Track: Multilevel Semantic Interaction in RMOT
<|reference_start|>MLS-Track: Multilevel Semantic Interaction in RMOT: The new trend in multi-object tracking task is to track objects of interest using natural language. However, the scarcity of paired prompt-instance data hinders its progress. To address this challenge, we propose a high-quality yet low-cost data generation method base on Unreal Engine 5 and construct a brand-new benchmark dataset, named Refer-UE-City, which primarily includes scenes from intersection surveillance videos, detailing the appearance and actions of people and vehicles. Specifically, it provides 14 videos with a total of 714 expressions, and is comparable in scale to the Refer-KITTI dataset. Additionally, we propose a multi-level semantic-guided multi-object framework called MLS-Track, where the interaction between the model and text is enhanced layer by layer through the introduction of Semantic Guidance Module (SGM) and Semantic Correlation Branch (SCB). Extensive experiments on Refer-UE-City and Refer-KITTI datasets demonstrate the effectiveness of our proposed framework and it achieves state-of-the-art performance. Code and datatsets will be available.<|reference_end|>
@article{ma2024mls-track:,
    title={MLS-Track: Multilevel Semantic Interaction in RMOT},
    author={Zeliang Ma, Song Yang, Zhe Cui, Zhicheng Zhao, Fei Su, Delong Liu,
  Jingyu Wang},
    journal={arXiv preprint arXiv:2404.12031},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.12031},
    primaryClass={cs.CV}
}