Moving object segmentation is a crucial task for autonomous vehicles as it can be used to segment objects in a class agnostic manner based on its motion cues. It will enable the detection of objects unseen during training (e.g., moose or a construction truck) generically based on their motion. Although pixel-wise motion segmentation has been studied in the literature, it is not dealt with at instance level, which would help separate connected segments of moving objects leading to better trajectory planning.  In this paper, we proposed a motion-based instance segmentation task and created a new annotated dataset based on KITTI , which will be released publicly. We make use of the YOLACT model to solve the instance motion segmentationnetwork by feeding inflow and image as input and instance motion masks as output. We extend it to a multi-task model that learns semantic and motion instance segmentation in a computationally efficient manner. Our model is based on sharing a prototype generation network between the two tasks and learning separate prototype coefficients per task . To obtain real-time performance, we study different efficient encoders and obtain 39 fps on a Titan Xp GPU using MobileNetV2 with an improvement of 10\% mAP relative to the baseline. A video demonstration of our work is available in URL
Moving object segmentation is a crucial task for autonomous vehicles as it can be used to segment objects in a class agnostic manner based on their motion cues. It enables the detection of unknown objects during training (e.g., moose or a construction truck) based on their motion. Although pixel-wise motion segmentation has been studied in autonomous driving literature, it is not dealt with on the instance level, which would help separate connected segments of moving objects leading to better trajectory planning. Other generic video object segmentation tasks have dealt with instance-wise motion segmentation but on much simpler setting and have not dealt with the multi-task learning problem for both semantic and class agnostic instance segmentation. In this paper, we propose a motion-based instance segmentation task and provide a new annotated dataset based on KITTIMoSeg , which will be released publicly. Our dataset provide extra class annotations which is crucial for studying class agnostic segmentation. We further propose an efficient multi-task learning approach that learns an extra class agnostic instance segmentation head through sharing the prototype generation network with the semantic head. The model then learns separate prototype coefficients within the class agnostic and semantic heads . To obtain real-time performance, we study different efficient encoders and obtain 39 fps on a Titan Xp GPU using MobileNetV2 with an improvement of 10\% mAP relative to the baseline. Our model outperforms state of the art motion segmentation methods with 3.3\% improvement. We summarize our work in a short video with qualitative results at URL
R
its
their
R
will enable
enables
R
objects unseen
unknown objects
D
generically
R
the
autonomous driving
R
at
on the
A
Other generic video object segmentation tasks have dealt with instance-wise motion segmentation but on much simpler setting and have not dealt with the multi-task learning problem for both semantic and class agnostic instance segmentation.
R
proposed
propose
R
created
provide
R
KITTI
KITTIMoSeg
R
We make use of the YOLACT model to solve the instance motion segmentationnetwork by feeding inflow and image as input and instance motion masks as output. We extend it to a
Our dataset provide extra class annotations which is crucial for studying class agnostic segmentation. We further propose an efficient
R
model that learns semantic and motion instance segmentation in a computationally efficient manner. Our model is based on sharing a
learning approach that learns an extra class agnostic instance segmentation head through sharing the
R
between the two tasks and learning
with the semantic head. The model then learns
R
per task
within the class agnostic and semantic heads
R
A video demonstration of our work is available in
Our model outperforms state of the art motion segmentation methods with 3.3\% improvement. We summarize our work in a short video with qualitative results at