  Conventional point cloud semantic segmentation methods usually employ an
encoder-decoder architecture, where mid-level features are locally aggregated
to extract geometric information. However, the over-reliance on these
class-agnostic local geometric representations may raise confusion between
local parts from different categories that are similar in appearance or
spatially adjacent. To address this issue, we argue that mid-level features can
be further enhanced with semantic information, and propose semantic-affine
transformation that transforms features of mid-level points belonging to
different categories with class-specific affine parameters. Based on this
technique, we propose SemAffiNet for point cloud semantic segmentation, which
utilizes the attention mechanism in the Transformer module to implicitly and
explicitly capture global structural knowledge within local parts for overall
comprehension of each category. We conduct extensive experiments on the
ScanNetV2 and NYUv2 datasets, and evaluate semantic-affine transformation on
various 3D point cloud and 2D image segmentation baselines, where both
qualitative and quantitative results demonstrate the superiority and
generalization ability of our proposed approach. Code is available at
https://github.com/wangzy22/SemAffiNet.

Computer Vision and Pattern Recognition;Artificial Intelligence;Machine Learning