RT-Pose: A 4D Radar Tensor-based 3D Human Pose Estimation and Localization Benchmark
<|reference_start|>RT-Pose: A 4D Radar Tensor-based 3D Human Pose Estimation and Localization Benchmark: Traditional methods for human localization and pose estimation (HPE), which mainly rely on RGB images as an input modality, confront substantial limitations in real-world applications due to privacy concerns. In contrast, radar-based HPE methods emerge as a promising alternative, characterized by distinctive attributes such as through-wall recognition and privacy-preserving, rendering the method more conducive to practical deployments. This paper presents a Radar Tensor-based human pose (RT-Pose) dataset and an open-source benchmarking framework. The RT-Pose dataset comprises 4D radar tensors, LiDAR point clouds, and RGB images, and is collected for a total of 72k frames across 240 sequences with six different complexity-level actions. The 4D radar tensor provides raw spatio-temporal information, differentiating it from other radar point cloud-based datasets. We develop an annotation process using RGB images and LiDAR point clouds to accurately label 3D human skeletons. In addition, we propose HRRadarPose, the first single-stage architecture that extracts the high-resolution representation of 4D radar tensors in 3D space to aid human keypoint estimation. HRRadarPose outperforms previous radar-based HPE work on the RT-Pose benchmark. The overall HRRadarPose performance on the RT-Pose dataset, as reflected in a mean per joint position error (MPJPE) of 9.91cm, indicates the persistent challenges in achieving accurate HPE in complex real-world scenarios. RT-Pose is available at https://huggingface.co/datasets/uwipl/RT-Pose.<|reference_end|>
@article{ho2024rt-pose:,
    title={RT-Pose: A 4D Radar Tensor-based 3D Human Pose Estimation and
  Localization Benchmark},
    author={Yuan-Hao Ho, Jen-Hao Cheng, Sheng Yao Kuan, Zhongyu Jiang, Wenhao
  Chai, Hsiang-Wei Huang, Chih-Lung Lin, Jenq-Neng Hwang},
    journal={arXiv preprint arXiv:2407.13930},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.13930},
    primaryClass={cs.CV cs.AI eess.SP}
}