Masks and Boxes: Combining the Best of Both Worlds for Multi-Object Tracking
<|reference_start|>Masks and Boxes: Combining the Best of Both Worlds for Multi-Object Tracking: Multi-object tracking (MOT) involves identifying and consistently tracking objects across video sequences. Traditional tracking-by-detection methods, while effective, often require extensive tuning and lack generalizability. On the other hand, segmentation mask-based methods are more generic but struggle with tracking management, making them unsuitable for MOT. We propose a novel approach, McByte, which incorporates a temporally propagated segmentation mask as a strong association cue within a tracking-by-detection framework. By combining bounding box and mask information, McByte enhances robustness and generalizability without per-sequence tuning. Evaluated on four benchmark datasets - DanceTrack, MOT17, SoccerNet-tracking 2022, and KITTI-tracking - McByte demonstrates performance gain in all cases examined. At the same time, it outperforms existing mask-based methods. Implementation code will be provided upon acceptance.<|reference_end|>
@article{stanczyk2024masks,
    title={Masks and Boxes: Combining the Best of Both Worlds for Multi-Object
  Tracking},
    author={Tomasz Stanczyk, Francois Bremond},
    journal={arXiv preprint arXiv:2409.14220},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.14220},
    primaryClass={cs.CV}
}