  Video-based 3D human pose and shape estimations are evaluated by intra-frame
accuracy and inter-frame smoothness. Although these two metrics are responsible
for different ranges of temporal consistency, existing state-of-the-art methods
treat them as a unified problem and use monotonous modeling structures (e.g.,
RNN or attention-based block) to design their networks. However, using a single
kind of modeling structure is difficult to balance the learning of short-term
and long-term temporal correlations, and may bias the network to one of them,
leading to undesirable predictions like global location shift, temporal
inconsistency, and insufficient local details. To solve these problems, we
propose to structurally decouple the modeling of long-term and short-term
correlations in an end-to-end framework, Global-to-Local Transformer (GLoT).
First, a global transformer is introduced with a Masked Pose and Shape
Estimation strategy for long-term modeling. The strategy stimulates the global
transformer to learn more inter-frame correlations by randomly masking the
features of several frames. Second, a local transformer is responsible for
exploiting local details on the human mesh and interacting with the global
transformer by leveraging cross-attention. Moreover, a Hierarchical Spatial
Correlation Regressor is further introduced to refine intra-frame estimations
by decoupled global-local representation and implicit kinematic constraints.
Our GLoT surpasses previous state-of-the-art methods with the lowest model
parameters on popular benchmarks, i.e., 3DPW, MPI-INF-3DHP, and Human3.6M.
Codes are available at https://github.com/sxl142/GLoT.

Computer Vision and Pattern Recognition