Enhancing 3D Human Pose Estimation Amidst Severe Occlusion with Dual Transformer Fusion
<|reference_start|>Enhancing 3D Human Pose Estimation Amidst Severe Occlusion with Dual Transformer Fusion: In the field of 3D Human Pose Estimation from monocular videos, the presence of diverse occlusion types presents a formidable challenge. Prior research has made progress by harnessing spatial and temporal cues to infer 3D poses from 2D joint observations. This paper introduces a Dual Transformer Fusion (DTF) algorithm, a novel approach to obtain a holistic 3D pose estimation, even in the presence of severe occlusions. Confronting the issue of occlusion-induced missing joint data, we propose a temporal interpolation-based occlusion guidance mechanism. To enable precise 3D Human Pose Estimation, our approach leverages the innovative DTF architecture, which first generates a pair of intermediate views. Each intermediate-view undergoes spatial refinement through a self-refinement schema. Subsequently, these intermediate-views are fused to yield the final 3D human pose estimation. The entire system is end-to-end trainable. Through extensive experiments conducted on the Human3.6M and MPI-INF-3DHP datasets, our method's performance is rigorously evaluated. Notably, our approach outperforms existing state-of-the-art methods on both datasets, yielding substantial improvements. The code is available here: https://github.com/MehwishG/DTF.<|reference_end|>
@article{ghafoor2024enhancing,
    title={Enhancing 3D Human Pose Estimation Amidst Severe Occlusion with Dual
  Transformer Fusion},
    author={Mehwish Ghafoor, Arif Mahmood, Muhammad Bilal},
    journal={arXiv preprint arXiv:2410.04574},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.04574},
    primaryClass={cs.CV cs.LG}
}