In summary, we make the following contributions in
this paper. (1) We assemble the largest scale meta-dataset
for 3D human pose estimation to date, consisting of 28 in-
dividual datasets, and release scripts for reproducing the
process. We call special attention to the problem of dis-

2

Table 1: We study the extreme multi-dataset setting of 3D
human pose estimation, using all datasets below in one
training process. We deﬁne three dataset combinations (in-

dicated by♢,♣, and,♡) to study the effect of training data

amount. (* marks custom dataset-speciﬁc skeletons.)

Real images with marker-based MoCap

#RealSubj.

Real images with markerless MoCap

Dataset name

MuCo-3DHP [59]
CMU-Panoptic [38]
AIST-Dance++ [45, 75]
HUMBI [81]
MPI-INF-3DHP [58]
RICH [31]
BEHAVE [6]
ASPset [61]
3DOH50K [83]
IKEA ASM [5]

Human3.6M [10, 32]
TotalCapture [74]
BML-MoVi [23]
Berkeley-MHAD [63]
UMPM [1]
Fit3D [22]
GPA [78]
HumanSC3D [21]
CHI3D [20]
Human4D [11]
MADS [84]
Learning 3D Human Pose Estimation from Dozens of Datasets using a Geometry-Aware Autoencoder to Bridge Between Skeleton Formats