  Contextual information is vital in visual understanding problems, such as
semantic segmentation and object detection. We propose a Criss-Cross Network
(CCNet) for obtaining full-image contextual information in a very effective and
efficient way. Concretely, for each pixel, a novel criss-cross attention module
harvests the contextual information of all the pixels on its criss-cross path.
By taking a further recurrent operation, each pixel can finally capture the
full-image dependencies. Besides, a category consistent loss is proposed to
enforce the criss-cross attention module to produce more discriminative
features. Overall, CCNet is with the following merits: 1) GPU memory friendly.
Compared with the non-local block, the proposed recurrent criss-cross attention
module requires 11x less GPU memory usage. 2) High computational efficiency.
The recurrent criss-cross attention significantly reduces FLOPs by about 85% of
the non-local block. 3) The state-of-the-art performance. We conduct extensive
experiments on semantic segmentation benchmarks including Cityscapes, ADE20K,
human parsing benchmark LIP, instance segmentation benchmark COCO, video
segmentation benchmark CamVid. In particular, our CCNet achieves the mIoU
scores of 81.9%, 45.76% and 55.47% on the Cityscapes test set, the ADE20K
validation set and the LIP validation set respectively, which are the new
state-of-the-art results. The source codes are available at
\url{https://github.com/speedinghzl/CCNet}.

Computer Vision and Pattern Recognition