...) and apply the same deep network. When the
input image is at different scales, the network (with
the same ﬁlter sizes) will extract features at different
scales. The scales play important roles in traditional
methods, e.g., the SIFT vectors are often extracted at
multiple scales [29], [27] (determined by the sizes of
the patches and Gaussian ﬁlters). We will show that
the scales are also important for the accuracy of deep
networks.
Interestingly, the coarsest pyramid level has a single
bin that covers the entire image. This is in fact a
“global pooling” operation, which is also investigated
in several concurrent works. In [31], [32] a global
average pooling is used to reduce the model size
and also reduce overﬁtting; in [33], a global average
pooling is used on the testing stage after all fc layers
to improve accuracy; in [34], a global max pooling is
used for weakly supervised object recognition. The
global pooling operation corresponds to the traditional Bag-of-Words method.
2.3 Training the Network
Theoretically, the above network structure can be
trained with standard back-propagation [1], regardless of the input image size. But in practice the GPU
Existing deep convolutional neural networks (CNNs) require a fixed-size
(e.g., 224x224) input image. This requirement is "artificial" and may reduce
the recognition accuracy for the images or sub-images of an arbitrary
size/scale. In this work, we equip the networks with another pooling strategy,
"spatial pyramid pooling", to eliminate the above requirement. The new network
structure, called SPP-net, can generate a fixed-length representation
regardless of image size/scale. Pyramid pooling is also robust to object
deformations. With these advantages, SPP-net should in general improve all
CNN-based image classification methods. On the ImageNet 2012 dataset, we
demonstrate that SPP-net boosts the accuracy of a variety of CNN architectures
despite their different designs. On the Pascal VOC 2007 and Caltech101
datasets, SPP-net achieves state-of-the-art classification results using a
single full-image representation and no fine-tuning.
  The power of SPP-net is also significant in object detection. Using SPP-net,
we compute the feature maps from the entire image only once, and then pool
features in arbitrary regions (sub-images) to generate fixed-length
representations for training the detectors. This method avoids repeatedly
computing the convolutional features. In processing test images, our method is
24-102x faster than the R-CNN method, while achieving better or comparable
accuracy on Pascal VOC 2007.
  In ImageNet Large Scale Visual Recognition Challenge (ILSVRC) 2014, our
methods rank #2 in object detection and #3 in image classification among all 38
teams. This manuscript also introduces the improvement made for this
competition.
Kaiming He Xiangyu Zhang Shaoqing Ren Jian Sun