  Skeleton-based action recognition is a central task of human-computer
interaction. However, most of the previous methods suffer from two issues: (i)
semantic ambiguity arising from spatiotemporal information mixture; and (ii)
overlooking the explicit exploitation of the latent data distributions (i.e.,
the intra-class variations and inter-class relations), thereby leading to local
optimum solutions of the skeleton encoders. To mitigate this, we propose a
spatial-temporal decoupling contrastive learning (STD-CL) framework to obtain
discriminative and semantically distinct representations from the sequences,
which can be incorporated into almost all previous skeleton encoders and have
no impact on the skeleton encoders when testing. Specifically, we decouple the
global features into spatial-specific and temporal-specific features to reduce
the spatiotemporal coupling of features. Furthermore, to explicitly exploit the
latent data distributions, we employ the attentive features to contrastive
learning, which models the cross-sequence semantic relations by pulling
together the features from the positive pairs and pushing away the negative
pairs. Extensive experiments show that STD-CL with four various skeleton
encoders (HCN, 2S-AGCN, CTR-GCN, and Hyperformer) achieves solid improvement on
NTU60, NTU120, and NW-UCLA benchmarks. The code will be released.

Computer Vision and Pattern Recognition