  Off-road semantic segmentation with fine-grained labels is necessary for
autonomous vehicles to understand driving scenes, as the coarse-grained road
detection can not satisfy off-road vehicles with various mechanical properties.
Fine-grained semantic segmentation in off-road scenes usually has no unified
category definition due to ambiguous nature environments, and the cost of
pixel-wise labeling is extremely high. Furthermore, semantic properties of
off-road scenes can be very changeable due to various precipitations,
temperature, defoliation, etc. To address these challenges, this research
proposes an active and contrastive learning-based method that does not rely on
pixel-wise labels, but only on patch-based weak annotations for model learning.
There is no need for predefined semantic categories, the contrastive
learning-based feature representation and adaptive clustering will discover the
category model from scene data. In order to actively adapt to new scenes, a
risk evaluation method is proposed to discover and select hard frames with
high-risk predictions for supplemental labeling, so as to update the model
efficiently. Experiments conducted on our self-developed off-road dataset and
DeepScene dataset demonstrate that fine-grained semantic segmentation can be
learned with only dozens of weakly labeled frames, and the model can
efficiently adapt across scenes by weak supervision, while achieving almost the
same level of performance as typical fully supervised baselines.

Computer Vision and Pattern Recognition;Robotics