HOI-M3:Capture Multiple Humans and Objects Interaction within Contextual Environment
<|reference_start|>HOI-M3:Capture Multiple Humans and Objects Interaction within Contextual Environment: Humans naturally interact with both others and the surrounding multiple objects, engaging in various social activities. However, recent advances in modeling human-object interactions mostly focus on perceiving isolated individuals and objects, due to fundamental data scarcity. In this paper, we introduce HOI-M3, a novel large-scale dataset for modeling the interactions of Multiple huMans and Multiple objects. Notably, it provides accurate 3D tracking for both humans and objects from dense RGB and object-mounted IMU inputs, covering 199 sequences and 181M frames of diverse humans and objects under rich activities. With the unique HOI-M3 dataset, we introduce two novel data-driven tasks with companion strong baselines: monocular capture and unstructured generation of multiple human-object interactions. Extensive experiments demonstrate that our dataset is challenging and worthy of further research about multiple human-object interactions and behavior analysis. Our HOI-M3 dataset, corresponding codes, and pre-trained models will be disseminated to the community for future research.<|reference_end|>
@article{zhang2024hoi-m3:capture,
    title={HOI-M3:Capture Multiple Humans and Objects Interaction within Contextual
  Environment},
    author={Juze Zhang, Jingyan Zhang, Zining Song, Zhanhe Shi, Chengfeng Zhao, Ye
  Shi, Jingyi Yu, Lan Xu, Jingya Wang},
    journal={arXiv preprint arXiv:2404.00299},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.00299},
    primaryClass={cs.CV}
}