SNIPER: Efficient Multi-Scale Training
We present SNIPER, an algorithm for performing efficient multi-scale training in instance level visual recognition tasks. Instead of processing every pixel in an image pyramid, SNIPER processes context regions around ground-truth instances (referred to as chips) at the appropriate scale. For background sampling, these context-regions are generated using proposals extracted from a region proposal network trained with a short learning schedule. Hence, the number of chips generated per image during training adaptively changes based on the scene complexity. SNIPER only processes 30% more pixels compared to the commonly used single scale training at 800x1333 pixels on the COCO dataset. But, it also observes samples from extreme resolutions of the image pyramid, like 1400x2000 pixels. As SNIPER operates on resampled low resolution chips (512x512 pixels), it can have a batch size as large as 20 on a single GPU even with a ResNet-101 backbone. Therefore it can benefit from batch-normalization during training without the need for synchronizing batch-normalization statistics across GPUs. SNIPER brings training of instance level recognition tasks like object detection closer to the protocol for image classification and suggests that the commonly accepted guideline that it is important to train on high resolution images for instance level visual recognition tasks might not be correct. Our implementation based on Faster-RCNN with a ResNet-101 backbone obtains an mAP of 47.6% on the COCO dataset for bounding box detection and can process 5 images per second during inference with a single GPU. Code is available at https://github.com/mahyarnajibi/SNIPER/.
#1. Summary This paper introduces practical techniques, called SNIPER, for speed-up training of object detection networks based on Region Proposal Networks (RPNs). The idea is simple: Instead of considering numerous object candidate boxes in the entire input image, SNIPER samples sub-images called “chips” and train detection models to localize objects in such chips. Specifically, positive chips are generated to contain ground-truth object boxes of a certain scale range, and negative ones are sub-images that has object proposals likely but not actual ground-truth. Since the scale of chips are commonly normalized to a small scale, object detectors can be scale invariant (as in SNIP [24]) and its training procedure becomes more efficient and faster than before.  #2. Quality 1) Experimental evaluation is weak, and the advantages of SNIPER claimed in the manuscript are not empirically justified. First, SNIPER is compared only with SNIP, its previous version, and current state of the arts are not listed in the tables. Moreover, even though the speed-up of training is the most important and unique contribution of SNIPER, there is no quantitative analysis on that aspect at all.  2) Its originality is quite limited. Please see #3 below for more details. 3) The quality of presentation is not good enough. Please see #4 below for more details. >> Thus, overall the quality of this submission seems below the standard of NIPS.   #3. Originality The originality of SNIPER seems quite limited since it is an extension of SNIP [24]. Specifically, it can be regarded as a set of practices that are designed to speed-up of SNIP, which is slower than conventional pipelines for training object detectors due to its extensive use of image pyramid for scale-invariant learning. That is, the component improving detection accuracy by scale-invariant learning is the contribution of SNIP, not of SNIPER.   #4. Clarity The paper is readable, but sometimes difficult to follow due to the following issues: 1) The algorithm for drawing chips is not presented. The manuscript simply enumerates a few conditions to be a positive/negative chip, and no detail of the procedure is introduced. 2) Some techniques taken by SNIPER are not well justified. For example, it is unknown what is the advantages of taking the strategy introduced in line 164~165, why a ground-truth box that is partially overlapped with a chip is not ignored, and so on. 3) Often previous approaches are not cited, and their abbreviations are not given as well. For example, OHEM in line 169, SNIP in line 171, synchronized batch normalization in line 183, training on 128 GPUs in line 184, OpenImages v4 in line 195, and so on. >> Due to the above issues, this submission looks not ready for publication. Also, they damage reproducibility of SNIPER.    #5. Significance 1) Strength: The proposed techniques can improve the efficiency of training deep object detection networks significantly.  2) Weakness: The main idea of learning detectors with carefully generated chips is good and reasonable, but is implemented by a set of simple practical techniques. 3) Weakness: This is an extension of SNIP [24], and focuses mostly on speed-up. Thus its novelty is significantly limited.