practice when applying FCNs [30] to semantic segmentation, which typically uses a per-pixel softmax and a multinomial cross-entropy loss. In that case, masks across classes
compete; in our case, with a per-pixel sigmoid and a binary
loss, they do not. We show by experiments that this formulation is key for good instance segmentation results.
Mask Representation: A mask encodes an input object’s
spatial layout. Thus, unlike class labels or box offsets
that are inevitably collapsed into short output vectors by
fully-connected ( fc) layers, extracting the spatial structure
of masks can be addressed naturally by the pixel-to-pixel
correspondence provided by convolutions.
Speciﬁcally, we predict an mmmask from each RoI
using an FCN [30]. This allows each layer in the mask
branch to maintain the explicit mmobject spatial layout without collapsing it into a vector representation that
lacks spatial dimensions. Unlike previous methods that resort to fclayers for mask prediction [33, 34, 10], our fully
convolutional representation requires fewer parameters, and
is more accurate as demonstrated by experiments.
This pixel-to-pixel behavior requires our RoI features,
which themselves are small feature maps, to be well aligned
We present a conceptually simple, flexible, and general framework for object
instance segmentation. Our approach efficiently detects objects in an image
while simultaneously generating a high-quality segmentation mask for each
instance. The method, called Mask R-CNN, extends Faster R-CNN by adding a
branch for predicting an object mask in parallel with the existing branch for
bounding box recognition. Mask R-CNN is simple to train and adds only a small
overhead to Faster R-CNN, running at 5 fps. Moreover, Mask R-CNN is easy to
generalize to other tasks, e.g., allowing us to estimate human poses in the
same framework. We show top results in all three tracks of the COCO suite of
challenges, including instance segmentation, bounding-box object detection, and
person keypoint detection. Without bells and whistles, Mask R-CNN outperforms
all existing, single-model entries on every task, including the COCO 2016
challenge winners. We hope our simple and effective approach will serve as a
solid baseline and help ease future research in instance-level recognition.
Code has been made available at: https://github.com/facebookresearch/Detectron
Kaiming He Georgia Gkioxari Piotr Dollár Ross Girshick