masks of resolution mm, one for each of the Kclasses.
To this we apply a per-pixel sigmoid, and deﬁne Lmask as
the average binary cross-entropy loss. For an RoI associated
with ground-truth class k,Lmask is only deﬁned on the k-th
mask (other mask outputs do not contribute to the loss).
Our deﬁnition of Lmask allows the network to generate
masks for every class without competition among classes;
we rely on the dedicated classiﬁcation branch to predict the
Figure 3. RoIAlign: The dashed grid represents a feature map, the solid lines an RoI
(with 22 bins in this example), and the dots
the 4 sampling points in each bin. RoIAlign
computes the value of each sampling point
by bilinear interpolation from the nearby grid
points on the feature map. No quantization is
performed on any coordinates involved in the
RoI, its bins, or the sampling points.
class label used to select the output mask. This decouples
mask and class prediction. This is different from common
practice when applying FCNs [30] to semantic segmentation, which typically uses a per-pixel softmax and a multinomial cross-entropy loss. In that case, masks across classes
We present a conceptually simple, flexible, and general framework for object
instance segmentation. Our approach efficiently detects objects in an image
while simultaneously generating a high-quality segmentation mask for each
instance. The method, called Mask R-CNN, extends Faster R-CNN by adding a
branch for predicting an object mask in parallel with the existing branch for
bounding box recognition. Mask R-CNN is simple to train and adds only a small
overhead to Faster R-CNN, running at 5 fps. Moreover, Mask R-CNN is easy to
generalize to other tasks, e.g., allowing us to estimate human poses in the
same framework. We show top results in all three tracks of the COCO suite of
challenges, including instance segmentation, bounding-box object detection, and
person keypoint detection. Without bells and whistles, Mask R-CNN outperforms
all existing, single-model entries on every task, including the COCO 2016
challenge winners. We hope our simple and effective approach will serve as a
solid baseline and help ease future research in instance-level recognition.
Code has been made available at: https://github.com/facebookresearch/Detectron
Kaiming He Georgia Gkioxari Piotr Dollár Ross Girshick