scenarios. Having outlined details speciﬁc to each,
general methodologies which apply to all three
scenarios are reviewed, such as data augmentation
and feature normalisation, together with the linear
classiﬁer (trained with a standard hinge loss). We
also specify here the benchmark datasets used in
the evaluation.
2.1 Scenario 1: Shallow representation (IFV)
Our reference shallow image representation is the
IFV [3]. Our choice is motivated by the fact that
IFV usually outperforms related encoding methods
such as BoVW, LLC [12], and VLAD [18]. Given an
imageI, the IFVFV(I)is obtained by extracting
1. http://www.robots.ox.ac.uk/vgg/research/deep eval/a dense collection of patches and corresponding
local descriptors xi2RD(e.g. SIFT) from the image
at multiple scales. Each descriptor xiis then softquantized using a Gaussian Mixture Model with
Kcomponents. First and second order differences
between each descriptor xiand its Gaussian cluster
meankare accumulated in corresponding blocks
uk,vkin the vector FV(I)2R2KD, appropriately
weighed by the Gaussian soft-assignments and
The latest generation of Convolutional Neural Networks (CNN) have achieved
impressive results in challenging benchmarks on image recognition and object
detection, significantly raising the interest of the community in these
methods. Nevertheless, it is still unclear how different CNN methods compare
with each other and with previous state-of-the-art shallow representations such
as the Bag-of-Visual-Words and the Improved Fisher Vector. This paper conducts
a rigorous evaluation of these new techniques, exploring different deep
architectures and comparing them on a common ground, identifying and disclosing
important implementation details. We identify several useful properties of
CNN-based representations, including the fact that the dimensionality of the
CNN output layer can be reduced significantly without having an adverse effect
on performance. We also identify aspects of deep and shallow methods that can
be successfully shared. In particular, we show that the data augmentation
techniques commonly applied to CNN-based methods can also be applied to shallow
methods, and result in an analogous performance boost. Source code and models
to reproduce the experiments in the paper is made publicly available.
Ken Chatfield Karen Simonyan Andrea Vedaldi Andrew Zisserman