Our approach relies on a novel formulation of dimen-
sionality reduction of sets of keypoints, via an afﬁne-
combining autoencoder with guaranteed built-in equivari-
ances to common transformations. By regularizing a 3D
human pose estimator’s output to stay close to the learned
latent space discovered by the autoencoder, we can more
effectively share information between the different datasets,
resulting in an overall more accurate and consistent pose es-
timator. We release code for data processing and training, as
well as trained models to serve as high-quality off-the-shelf
methods for downstream research.
Acknowledgments. This work was supported by the ERC
Consolidator Grant project “DeeViSe” (ERC-CoG-2017-
773161) and by Robert Bosch GmbH under the project
“Context Understanding for Autonomous Systems.”

8
Learning 3D Human Pose Estimation from Dozens of Datasets using a Geometry-Aware Autoencoder to Bridge Between Skeleton Formats