Most recently, Li et al. [26] combined the segment proposal system in [8] and object detection system in [11] for
“fully convolutional instance segmentation” (FCIS). The
common idea in [8, 11, 26] is to predict a set of positionsensitive output channels fully convolutionally. These
channels simultaneously address object classes, boxes, and
masks, making the system fast. But FCIS exhibits systematic errors on overlapping instances and creates spurious
edges (Figure 6), showing that it is challenged by the fundamental difﬁculties of segmenting instances.
2
Another family of solutions [23, 4, 3, 29] to instance segmentation are driven by the success of semantic segmentation. Starting from per-pixel classiﬁcation results ( e.g.,
FCN outputs), these methods attempt to cut the pixels of
the same category into different instances. In contrast to the
segmentation-ﬁrst strategy of these methods, Mask R-CNN
is based on an instance-ﬁrst strategy. We expect a deeper incorporation of both strategies will be studied in the future.
3. Mask R-CNN
Mask R-CNN is conceptually simple: Faster R-CNN has
two outputs for each candidate object, a class label and a
We present a conceptually simple, flexible, and general framework for object
instance segmentation. Our approach efficiently detects objects in an image
while simultaneously generating a high-quality segmentation mask for each
instance. The method, called Mask R-CNN, extends Faster R-CNN by adding a
branch for predicting an object mask in parallel with the existing branch for
bounding box recognition. Mask R-CNN is simple to train and adds only a small
overhead to Faster R-CNN, running at 5 fps. Moreover, Mask R-CNN is easy to
generalize to other tasks, e.g., allowing us to estimate human poses in the
same framework. We show top results in all three tracks of the COCO suite of
challenges, including instance segmentation, bounding-box object detection, and
person keypoint detection. Without bells and whistles, Mask R-CNN outperforms
all existing, single-model entries on every task, including the COCO 2016
challenge winners. We hope our simple and effective approach will serve as a
solid baseline and help ease future research in instance-level recognition.
Code has been made available at: https://github.com/facebookresearch/Detectron
Kaiming He Georgia Gkioxari Piotr Dollár Ross Girshick