  Non-local operations are usually used to capture long-range dependencies via
aggregating global context to each position recently. However, most of the
methods cannot preserve object shapes since they only focus on feature
similarity but ignore proximity between central and other positions for
capturing long-range dependencies, while shape-awareness is beneficial to many
computer vision tasks. In this paper, we propose a Semi-Global Shape-aware
Network (SGSNet) considering both feature similarity and proximity for
preserving object shapes when modeling long-range dependencies. A hierarchical
way is taken to aggregate global context. In the first level, each position in
the whole feature map only aggregates contextual information in vertical and
horizontal directions according to both similarity and proximity. And then the
result is input into the second level to do the same operations. By this
hierarchical way, each central position gains supports from all other
positions, and the combination of similarity and proximity makes each position
gain supports mostly from the same semantic object. Moreover, we also propose a
linear time algorithm for the aggregation of contextual information, where each
of rows and columns in the feature map is treated as a binary tree to reduce
similarity computation cost. Experiments on semantic segmentation and image
retrieval show that adding SGSNet to existing networks gains solid improvements
on both accuracy and efficiency.

Computer Vision and Pattern Recognition