title: WHAC: World-grounded Humans and Cameras
summary: Estimating human and camera trajectories with accurate scale in the world
  coordinate system from a monocular video is a highly desirable yet challenging
  and ill-posed problem. In this study, we aim to recover expressive parametric
  human models (i.e., SMPL-X) and corresponding camera poses jointly, by
  leveraging the synergy between three critical players: the world, the human,
  and the camera. Our approach is founded on two key observations. Firstly,
  camera-frame SMPL-X estimation methods readily recover absolute human depth.
  Secondly, human motions inherently provide absolute spatial cues. By
  integrating these insights, we introduce a novel framework, referred to as
  WHAC, to facilitate world-grounded expressive human pose and shape estimation
  (EHPS) alongside camera pose estimation, without relying on traditional
  optimization techniques. Additionally, we present a new synthetic dataset,
  WHAC-A-Mole, which includes accurately annotated humans and cameras, and
  features diverse interactive human motions as well as realistic camera
  trajectories. Extensive experiments on both standard and newly established
  benchmarks highlight the superiority and efficacy of our framework. We will
  make the code and dataset publicly available.
paper_authors._id: 67bc67e17727595ca5b7ddb4
paper_authors.name: Wanqi Yin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-27T09:17:51.355Z
paper_authors._id: 668f51fb8e8d87dbdd23caa9
paper_authors.avatarUrl: /avatars/0b17cb0f3ad2c729f185cdccdad94e48.svg
paper_authors.fullname: Yin
paper_authors.type: user
paper_authors.user: waanqii
paper_authors._id: 67bc67e17727595ca5b7ddb5
paper_authors.name: Zhongang Cai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-24T16:14:30.375Z
paper_authors._id: 652d06833b5997ed71ce5c46
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/xZTXEcnEogEmBm_ledJQr.jpeg
paper_authors.fullname: Zhongang Cai
paper_authors.type: user
paper_authors.user: caizhongang
paper_authors._id: 67bc67e17727595ca5b7ddb6
paper_authors.name: Ruisi Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-24T16:14:42.508Z
paper_authors._id: 65b74305e602b6c2c9125480
paper_authors.avatarUrl: /avatars/d36909e0f245bfeb632a4afc9d3fceca.svg
paper_authors.fullname: wang ruisi
paper_authors.type: user
paper_authors.user: wruisi
paper_authors._id: 67bc67e17727595ca5b7ddb7
paper_authors.name: Fanzhou Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-24T16:14:48.396Z
paper_authors._id: 66ac94b7ccd0aed70992a8be
paper_authors.avatarUrl: /avatars/98db346b7bfcb2040d4b58727a73d18b.svg
paper_authors.fullname: Fanzhou Wang
paper_authors.type: user
paper_authors.user: wentww
paper_authors._id: 67bc67e17727595ca5b7ddb8
paper_authors.name: Chen Wei
paper_authors._id: 67bc67e17727595ca5b7ddb9
paper_authors.name: Haiyi Mei
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-24T16:14:59.208Z
paper_authors._id: 635a895ef4a106ecd9203b2d
paper_authors.avatarUrl: /avatars/0ed9967f559582c2d93b5471b39f731a.svg
paper_authors.fullname: haiyimei
paper_authors.type: user
paper_authors.user: haiyimei
paper_authors._id: 67bc67e17727595ca5b7ddba
paper_authors.name: Weiye Xiao
paper_authors._id: 67bc67e17727595ca5b7ddbb
paper_authors.name: Zhitao Yang
paper_authors._id: 67bc67e17727595ca5b7ddbc
paper_authors.name: Qingping Sun
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-24T16:15:19.079Z
paper_authors._id: 65e20f88bb9c052e4171b857
paper_authors.avatarUrl: /avatars/5dcb3fe293c53e051842baf9024d589b.svg
paper_authors.fullname: Qingping SUN
paper_authors.type: user
paper_authors.user: ttxskk
paper_authors._id: 67bc67e17727595ca5b7ddbd
paper_authors.name: Atsushi Yamashita
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-24T16:15:25.136Z
paper_authors._id: 6485d56c77076d551d4adedc
paper_authors.avatarUrl: /avatars/0154474a6c6ec39a08f354b5dd69e1e3.svg
paper_authors.fullname: atsushi yamashita
paper_authors.type: user
paper_authors.user: atsu-yama
paper_authors._id: 67bc67e17727595ca5b7ddbe
paper_authors.name: Ziwei Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-24T16:14:15.911Z
paper_authors._id: 62ab1ac1d48b4d8b048a3473
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
paper_authors.fullname: Ziwei Liu
paper_authors.type: user
paper_authors.user: liuziwei7
paper_authors._id: 67bc67e17727595ca5b7ddbf
paper_authors.name: Lei Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-10T08:11:07.737Z
paper_authors._id: 6626a471430a124253f197c8
paper_authors.avatarUrl: /avatars/f5747fdbe495d1296fed9d16d8c95857.svg
paper_authors.fullname: yl-1993
paper_authors.type: user
paper_authors.user: yl-1993
paper_title: WHAC: World-grounded Humans and Cameras
paper_summary: Estimating human and camera trajectories with accurate scale in the world
  coordinate system from a monocular video is a highly desirable yet challenging
  and ill-posed problem. In this study, we aim to recover expressive parametric
  human models (i.e., SMPL-X) and corresponding camera poses jointly, by
  leveraging the synergy between three critical players: the world, the human,
  and the camera. Our approach is founded on two key observations. Firstly,
  camera-frame SMPL-X estimation methods readily recover absolute human depth.
  Secondly, human motions inherently provide absolute spatial cues. By
  integrating these insights, we introduce a novel framework, referred to as
  WHAC, to facilitate world-grounded expressive human pose and shape estimation
  (EHPS) alongside camera pose estimation, without relying on traditional
  optimization techniques. Additionally, we present a new synthetic dataset,
  WHAC-A-Mole, which includes accurately annotated humans and cameras, and
  features diverse interactive human motions as well as realistic camera
  trajectories. Extensive experiments on both standard and newly established
  benchmarks highlight the superiority and efficacy of our framework. We will
  make the code and dataset publicly available.
paper_ai_summary: A new framework, WHAC, jointly estimates expressive human models and camera poses from monocular video using synergies between human motion and camera perspectives without traditional optimization.
paper_ai_keywords: SMPL-X expressive human models EHPS camera poses WHAC WHAC-A-Mole synthetic dataset monocular video human motion camera trajectories