FreeA: Human-object Interaction Detection using Free Annotation Labels
<|reference_start|>FreeA: Human-object Interaction Detection using Free Annotation Labels: Recent human-object interaction (HOI) detection approaches rely on high cost of manpower and require comprehensive annotated image datasets. In this paper, we propose a novel self-adaption language-driven HOI detection method, termed as FreeA, without labeling by leveraging the adaptability of CLIP to generate latent HOI labels. To be specific, FreeA matches image features of human-object pairs with HOI text templates, and a priori knowledge-based mask method is developed to suppress improbable interactions. In addition, FreeA utilizes the proposed interaction correlation matching method to enhance the likelihood of actions related to a specified action, further refine the generated HOI labels. Experiments on two benchmark datasets show that FreeA achieves state-of-the-art performance among weakly supervised HOI models. Our approach is +8.58 mean Average Precision (mAP) on HICO-DET and +1.23 mAP on V-COCO more accurate in localizing and classifying the interactive actions than the newest weakly model, and +1.68 mAP and +7.28 mAP than the latest weakly+ model, respectively. Code will be available at https://drliuqi.github.io/.<|reference_end|>
@article{wang2024freea:,
    title={FreeA: Human-object Interaction Detection using Free Annotation Labels},
    author={Yuxiao Wang, Zhenao Wei, Xinyu Jiang, Yu Lei, Weiying Xue, Jinxiu Liu,
  Qi Liu},
    journal={arXiv preprint arXiv:2403.01840},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.01840},
    primaryClass={cs.CV cs.AI}
}