title: HumanMM: Global Human Motion Recovery from Multi-shot Videos
summary: In this paper, we present a novel framework designed to reconstruct
  long-sequence 3D human motion in the world coordinates from in-the-wild videos
  with multiple shot transitions. Such long-sequence in-the-wild motions are
  highly valuable to applications such as motion generation and motion
  understanding, but are of great challenge to be recovered due to abrupt shot
  transitions, partial occlusions, and dynamic backgrounds presented in such
  videos. Existing methods primarily focus on single-shot videos, where
  continuity is maintained within a single camera view, or simplify multi-shot
  alignment in camera space only. In this work, we tackle the challenges by
  integrating an enhanced camera pose estimation with Human Motion Recovery (HMR)
  by incorporating a shot transition detector and a robust alignment module for
  accurate pose and orientation continuity across shots. By leveraging a custom
  motion integrator, we effectively mitigate the problem of foot sliding and
  ensure temporal consistency in human pose. Extensive evaluations on our created
  multi-shot dataset from public 3D human datasets demonstrate the robustness of
  our method in reconstructing realistic human motion in world coordinates.
paper_authors._id: 67d029c7f8577b7e578cc5ff
paper_authors.name: Yuhong Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-11T14:24:04.517Z
paper_authors._id: 65f8281209cf7381af189aa7
paper_authors.avatarUrl: /avatars/fc7d570d1ff4df5b87b38a082b16f1ab.svg
paper_authors.fullname: Yuhong Zhang
paper_authors.type: user
paper_authors.user: YuhongZhang
paper_authors._id: 67d029c7f8577b7e578cc600
paper_authors.name: Guanlin Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-11T14:24:11.123Z
paper_authors._id: 6523f611db79e539485521b1
paper_authors.avatarUrl: /avatars/64aca195312084a98162b08152841429.svg
paper_authors.fullname: Guanlin WU
paper_authors.type: user
paper_authors.user: Frank0930
paper_authors._id: 67d029c7f8577b7e578cc601
paper_authors.name: Ling-Hao Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-11T14:24:17.437Z
paper_authors._id: 63109a4d61cab0446e48c83b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63109a4d61cab0446e48c83b/JQlVkQp0ok586ND1GmB0w.png
paper_authors.fullname: Ling-Hao Chen
paper_authors.type: user
paper_authors.user: EvanTHU
paper_authors._id: 67d029c7f8577b7e578cc602
paper_authors.name: Zhuokai Zhao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-11T14:24:37.175Z
paper_authors._id: 63af25605fe9db73f67a0fb7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63af25605fe9db73f67a0fb7/mcQHbTdJVxi2GdHzRB9ad.jpeg
paper_authors.fullname: Zhuokai Zhao
paper_authors.type: user
paper_authors.user: zhuokai
paper_authors._id: 67d029c7f8577b7e578cc603
paper_authors.name: Jing Lin
paper_authors._id: 67d029c7f8577b7e578cc604
paper_authors.name: Xiaoke Jiang
paper_authors._id: 67d029c7f8577b7e578cc605
paper_authors.name: Jiamin Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-11T14:24:52.948Z
paper_authors._id: 61a48038cd3eb3043f38e402
paper_authors.avatarUrl: /avatars/21f33c82fcbbf49434345d522c8865d9.svg
paper_authors.fullname: Jiamin Wu
paper_authors.type: user
paper_authors.user: Bunny
paper_authors._id: 67d029c7f8577b7e578cc606
paper_authors.name: Zhuoheng Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-11T14:24:59.235Z
paper_authors._id: 66d9e708dc8d21114944e4f8
paper_authors.avatarUrl: /avatars/1f22d26eeb5b70d56a7031dca2d7b790.svg
paper_authors.fullname: Zhuoheng Li
paper_authors.type: user
paper_authors.user: Andy-LZH
paper_authors._id: 67d029c7f8577b7e578cc607
paper_authors.name: Hao Frank Yang
paper_authors._id: 67d029c7f8577b7e578cc608
paper_authors.name: Haoqian Wang
paper_authors._id: 67d029c7f8577b7e578cc609
paper_authors.name: Lei Zhang
paper_title: HumanMM: Global Human Motion Recovery from Multi-shot Videos
paper_summary: In this paper, we present a novel framework designed to reconstruct
  long-sequence 3D human motion in the world coordinates from in-the-wild videos
  with multiple shot transitions. Such long-sequence in-the-wild motions are
  highly valuable to applications such as motion generation and motion
  understanding, but are of great challenge to be recovered due to abrupt shot
  transitions, partial occlusions, and dynamic backgrounds presented in such
  videos. Existing methods primarily focus on single-shot videos, where
  continuity is maintained within a single camera view, or simplify multi-shot
  alignment in camera space only. In this work, we tackle the challenges by
  integrating an enhanced camera pose estimation with Human Motion Recovery (HMR)
  by incorporating a shot transition detector and a robust alignment module for
  accurate pose and orientation continuity across shots. By leveraging a custom
  motion integrator, we effectively mitigate the problem of foot sliding and
  ensure temporal consistency in human pose. Extensive evaluations on our created
  multi-shot dataset from public 3D human datasets demonstrate the robustness of
  our method in reconstructing realistic human motion in world coordinates.
paper_ai_summary: A framework for reconstructing long-sequence 3D human motion with multiple shot transitions by integrating enhanced camera pose estimation, Human Motion Recovery (HMR), shot transition detection, and a robust alignment module.
paper_ai_keywords: camera pose estimation Human Motion Recovery (HMR) shot transition detector robust alignment module multi-shot dataset foot sliding temporal consistency