tion [33, 34]. In 3D, Jakab et al. [35] discover control points
for deforming 3D shapes. Rhodin et al. [69] learn a 3D hu-
man representation that consists of a set of 3D points, which
encode both pose and appearance, optimizing for the unsu-
pervised auxiliary task of novel view synthesis. Loper et al.
[51] optimize the placement of sparse markers on the body
to best capture both human shape and pose.
Linear Subspace Learning. Linear dimensionality reduc-
tion has a long history, with principal component analysis
being the best known representative [66]. Its relation to au-
toencoders was discovered by Bourlard and Kamp [9], and
a recent paper by the same ﬁrst author reviews the develop-
ments since [8]. Linear autoencoders have been employed
in robust and sparse [25] variants, a detailed overview is
presented in [16]. Our proposed afﬁne-combining autoen-
coders are related, but have different constraints, tailored to
our use case, i.e., that the weights sum to unity, and there is
Learning 3D Human Pose Estimation from Dozens of Datasets using a Geometry-Aware Autoencoder to Bridge Between Skeleton Formats