  In recent years, video semantic segmentation has made great progress with
advanced deep neural networks. However, there still exist two main challenges
\ie, information inconsistency and computation cost. To deal with the two
difficulties, we propose a novel motion-state alignment framework for video
semantic segmentation to keep both motion and state consistency. In the
framework, we first construct a motion alignment branch armed with an efficient
decoupled transformer to capture dynamic semantics, guaranteeing region-level
temporal consistency. Then, a state alignment branch composed of a stage
transformer is designed to enrich feature spaces for the current frame to
extract static semantics and achieve pixel-level state consistency. Next, by a
semantic assignment mechanism, the region descriptor of each semantic category
is gained from dynamic semantics and linked with pixel descriptors from static
semantics. Benefiting from the alignment of these two kinds of effective
information, the proposed method picks up dynamic and static semantics in a
targeted way, so that video semantic regions are consistently segmented to
obtain precise locations with low computational complexity. Extensive
experiments on Cityscapes and CamVid datasets show that the proposed approach
outperforms state-of-the-art methods and validates the effectiveness of the
motion-state alignment framework.

Computer Vision and Pattern Recognition