  Prompt-OVD is an efficient and effective framework for open-vocabulary object
detection that utilizes class embeddings from CLIP as prompts, guiding the
Transformer decoder to detect objects in both base and novel classes.
Additionally, our novel RoI-based masked attention and RoI pruning techniques
help leverage the zero-shot classification ability of the Vision
Transformer-based CLIP, resulting in improved detection performance at minimal
computational cost. Our experiments on the OV-COCO and OVLVIS datasets
demonstrate that Prompt-OVD achieves an impressive 21.2 times faster inference
speed than the first end-to-end open-vocabulary detection method (OV-DETR),
while also achieving higher APs than four two-stage-based methods operating
within similar inference time ranges. Code will be made available soon.

Computer Vision and Pattern Recognition