  The motion analysis of human skeletons is crucial for human action
recognition, which is one of the most active topics in computer vision. In this
paper, we propose a fully end-to-end action-attending graphic neural network
(A$^2$GNN) for skeleton-based action recognition, in which each irregular
skeleton is structured as an undirected attribute graph. To extract high-level
semantic representation from skeletons, we perform the local spectral graph
filtering on the constructed attribute graphs like the standard image
convolution operation. Considering not all joints are informative for action
analysis, we design an action-attending layer to detect those salient action
units (AUs) by adaptively weighting skeletal joints. Herein the filtering
responses are parameterized into a weighting function irrelevant to the order
of input nodes. To further encode continuous motion variations, the deep
features learnt from skeletal graphs are gathered along consecutive temporal
slices and then fed into a recurrent gated network. Finally, the spectral graph
filtering, action-attending and recurrent temporal encoding are integrated
together to jointly train for the sake of robust action recognition as well as
the intelligibility of human actions. To evaluate our A$^2$GNN, we conduct
extensive experiments on four benchmark skeleton-based action datasets,
including the large-scale challenging NTU RGB+D dataset. The experimental
results demonstrate that our network achieves the state-of-the-art
performances.

Computer Vision and Pattern Recognition