title: Humans in 4D: Reconstructing and Tracking Humans with Transformers
summary: We present an approach to reconstruct humans and track them over time. At the
  core of our approach, we propose a fully "transformerized" version of a network
  for human mesh recovery. This network, HMR 2.0, advances the state of the art
  and shows the capability to analyze unusual poses that have in the past been
  difficult to reconstruct from single images. To analyze video, we use 3D
  reconstructions from HMR 2.0 as input to a tracking system that operates in 3D.
  This enables us to deal with multiple people and maintain identities through
  occlusion events. Our complete approach, 4DHumans, achieves state-of-the-art
  results for tracking people from monocular video. Furthermore, we demonstrate
  the effectiveness of HMR 2.0 on the downstream task of action recognition,
  achieving significant improvements over previous pose-based action recognition
  approaches. Our code and models are available on the project website:
  https://shubham-goel.github.io/4dhumans/.
paper_authors._id: 64781e7f168cb428e012dfb6
paper_authors.name: Shubham Goel
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-01T16:27:46.133Z
paper_authors._id: 64787257d946c2a46a584928
paper_authors.avatarUrl: /avatars/445dd463ab784b7458eca62448d58814.svg
paper_authors.fullname: Shubham Goel
paper_authors.type: user
paper_authors.user: shubham-goel
paper_authors._id: 64781e7f168cb428e012dfb7
paper_authors.name: Georgios Pavlakos
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2023-06-01T17:23:51.925Z
paper_authors._id: 6478d3433b7f8b1f6249b469
paper_authors.avatarUrl: /avatars/11e7d7a94ae26500c1c2ad62e760726f.svg
paper_authors.fullname: Georgios Pavlakos
paper_authors.type: user
paper_authors.user: geopavlakos
paper_authors._id: 64781e7f168cb428e012dfb8
paper_authors.name: Jathushan Rajasegaran
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-01-10T04:08:10.724Z
paper_authors._id: 63895b3a43d8b0797a0d0406
paper_authors.avatarUrl: /avatars/04d9952dff70f85d4e481ec80ae818cb.svg
paper_authors.fullname: Jathushan Rajasegaran
paper_authors.type: user
paper_authors.user: brjathu
paper_authors._id: 64781e7f168cb428e012dfb9
paper_authors.name: Angjoo Kanazawa
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2023-06-01T17:04:52.139Z
paper_authors._id: 6478cf7150ff7001631679c3
paper_authors.avatarUrl: /avatars/65ec385a9cc44c972e6caf952e759ff1.svg
paper_authors.fullname: Angjoo Kanazawa
paper_authors.type: user
paper_authors.user: akanazawa
paper_authors._id: 64781e7f168cb428e012dfba
paper_authors.name: Jitendra Malik
paper_title: Humans in 4D: Reconstructing and Tracking Humans with Transformers
paper_summary: We present an approach to reconstruct humans and track them over time. At the
  core of our approach, we propose a fully "transformerized" version of a network
  for human mesh recovery. This network, HMR 2.0, advances the state of the art
  and shows the capability to analyze unusual poses that have in the past been
  difficult to reconstruct from single images. To analyze video, we use 3D
  reconstructions from HMR 2.0 as input to a tracking system that operates in 3D.
  This enables us to deal with multiple people and maintain identities through
  occlusion events. Our complete approach, 4DHumans, achieves state-of-the-art
  results for tracking people from monocular video. Furthermore, we demonstrate
  the effectiveness of HMR 2.0 on the downstream task of action recognition,
  achieving significant improvements over previous pose-based action recognition
  approaches. Our code and models are available on the project website:
  https://shubham-goel.github.io/4dhumans/.
paper_ai_summary: A fully transformerized network for human mesh recovery and tracking achieves state-of-the-art results in monocular video tracking and action recognition.
paper_ai_keywords: transformerized human mesh recovery HMR 2.0 3D reconstructions tracking system 3D 4DHumans action recognition