Humans in 4D: Reconstructing and Tracking Humans with Transformers
We present an approach to reconstruct humans and track them over time. At the
core of our approach, we propose a fully "transformerized" version of a network
for human mesh recovery. This network, HMR 2.0, advances the state of the art
and shows the capability to analyze unusual poses that have in the past been
difficult to reconstruct from single images. To analyze video, we use 3D
reconstructions from HMR 2.0 as input to a tracking system that operates in 3D.
This enables us to deal with multiple people and maintain identities through
occlusion events. Our complete approach, 4DHumans, achieves state-of-the-art
results for tracking people from monocular video. Furthermore, we demonstrate
the effectiveness of HMR 2.0 on the downstream task of action recognition,
achieving significant improvements over previous pose-based action recognition
approaches. Our code and models are available on the project website:
https://shubham-goel.github.io/4dhumans/.
64781e7f168cb428e012dfb6
Shubham Goel
claimed_verified
2023-06-01T16:27:46.133Z
64787257d946c2a46a584928
/avatars/445dd463ab784b7458eca62448d58814.svg
Shubham Goel
user
shubham-goel
64781e7f168cb428e012dfb7
Georgios Pavlakos
extracted_confirmed
2023-06-01T17:23:51.925Z
6478d3433b7f8b1f6249b469
/avatars/11e7d7a94ae26500c1c2ad62e760726f.svg
Georgios Pavlakos
user
geopavlakos
64781e7f168cb428e012dfb8
Jathushan Rajasegaran
extracted_confirmed
2025-01-10T04:08:10.724Z
63895b3a43d8b0797a0d0406
/avatars/04d9952dff70f85d4e481ec80ae818cb.svg
Jathushan Rajasegaran
user
brjathu
64781e7f168cb428e012dfb9
Angjoo Kanazawa
extracted_confirmed
2023-06-01T17:04:52.139Z
6478cf7150ff7001631679c3
/avatars/65ec385a9cc44c972e6caf952e759ff1.svg
Angjoo Kanazawa
user
akanazawa
64781e7f168cb428e012dfba
Jitendra Malik
Humans in 4D: Reconstructing and Tracking Humans with Transformers
We present an approach to reconstruct humans and track them over time. At the
core of our approach, we propose a fully "transformerized" version of a network
for human mesh recovery. This network, HMR 2.0, advances the state of the art
and shows the capability to analyze unusual poses that have in the past been
difficult to reconstruct from single images. To analyze video, we use 3D
reconstructions from HMR 2.0 as input to a tracking system that operates in 3D.
This enables us to deal with multiple people and maintain identities through
occlusion events. Our complete approach, 4DHumans, achieves state-of-the-art
results for tracking people from monocular video. Furthermore, we demonstrate
the effectiveness of HMR 2.0 on the downstream task of action recognition,
achieving significant improvements over previous pose-based action recognition
approaches. Our code and models are available on the project website:
https://shubham-goel.github.io/4dhumans/.
A fully transformerized network for human mesh recovery and tracking achieves state-of-the-art results in monocular video tracking and action recognition.
transformerized human mesh recovery HMR 2.0 3D reconstructions tracking system 3D 4DHumans action recognition