  Recently, query based object detection frameworks achieve comparable
performance with previous state-of-the-art object detectors. However, how to
fully leverage such frameworks to perform instance segmentation remains an open
problem. In this paper, we present QueryInst (Instances as Queries), a query
based instance segmentation method driven by parallel supervision on dynamic
mask heads. The key insight of QueryInst is to leverage the intrinsic
one-to-one correspondence in object queries across different stages, as well as
one-to-one correspondence between mask RoI features and object queries in the
same stage. This approach eliminates the explicit multi-stage mask head
connection and the proposal distribution inconsistency issues inherent in
non-query based multi-stage instance segmentation methods. We conduct extensive
experiments on three challenging benchmarks, i.e., COCO, CityScapes, and
YouTube-VIS to evaluate the effectiveness of QueryInst in instance segmentation
and video instance segmentation (VIS) task. Specifically, using ResNet-101-FPN
backbone, QueryInst obtains 48.1 box AP and 42.8 mask AP on COCO test-dev,
which is 2 points higher than HTC in terms of both box AP and mask AP, while
runs 2.4 times faster. For video instance segmentation, QueryInst achieves the
best performance among all online VIS approaches and strikes a decent
speed-accuracy trade-off. Code is available at
\url{https://github.com/hustvl/QueryInst}.

Computer Vision and Pattern Recognition