  Modern object detectors usually suffer from low accuracy issues, as
foregrounds always drown in tons of backgrounds and become hard examples during
training. Compared with those proposal-based ones, real-time detectors are in
far more serious trouble since they renounce the use of region-proposing stage
which is used to filter a majority of backgrounds for achieving real-time
rates. Though foregrounds as hard examples are in urgent need of being mined
from tons of backgrounds, a considerable number of state-of-the-art real-time
detectors, like YOLO series, have yet to profit from existing hard example
mining methods, as using these methods need detectors fit series of
prerequisites. In this paper, we propose a general hard example mining method
named Loss Rank Mining (LRM) to fill the gap. LRM is a general method for
real-time detectors, as it utilizes the final feature map which exists in all
real-time detectors to mine hard examples. By using LRM, some elements
representing easy examples in final feature map are filtered and detectors are
forced to concentrate on hard examples during training. Extensive experiments
validate the effectiveness of our method. With our method, the improvements of
YOLOv2 detector on auto-driving related dataset KITTI and more general dataset
PASCAL VOC are over 5% and 2% mAP, respectively. In addition, LRM is the first
hard example mining strategy which could fit YOLOv2 perfectly and make it
better applied in series of real scenarios where both real-time rates and
accurate detection are strongly demanded.

Computer Vision and Pattern Recognition