details . For example, state-of-the-art methods such
as [13] not only involve the use of a CNN, but
also include other improvements such as the use
of very large scale datasets, GPU computation, and
data augmentation (also known as data jittering or
virtual sampling). These improvements could also
transfer to shallow representations such as the IFV ,
potentially explaining a part of the performancearXiv:1405.3531v4  [cs.CV]  5 Nov 2014
2
gap [14].
In this study we analyse and empirically clarify
these issues, conducting a large set of rigorous
experiments (Sect. 4), in many ways picking up
the story where it last ended in [12] with the
comparison of shallow encoders. We focus on methods to construct image representations ,i.e. encoding functions mapping an image Ito a vector
(I)2Rdsuitable for analysis with a linear classiﬁer, such as an SVM. We consider three scenarios
(Sect. 2, Sect. 3): shallow image representations,
deep representations pre-trained on outside data,
and deep representation pre-trained and then ﬁnetuned on the target dataset. As part of our tests,
we explore generally-applicable best practices that
The latest generation of Convolutional Neural Networks (CNN) have achieved
impressive results in challenging benchmarks on image recognition and object
detection, significantly raising the interest of the community in these
methods. Nevertheless, it is still unclear how different CNN methods compare
with each other and with previous state-of-the-art shallow representations such
as the Bag-of-Visual-Words and the Improved Fisher Vector. This paper conducts
a rigorous evaluation of these new techniques, exploring different deep
architectures and comparing them on a common ground, identifying and disclosing
important implementation details. We identify several useful properties of
CNN-based representations, including the fact that the dimensionality of the
CNN output layer can be reduced significantly without having an adverse effect
on performance. We also identify aspects of deep and shallow methods that can
be successfully shared. In particular, we show that the data augmentation
techniques commonly applied to CNN-based methods can also be applied to shallow
methods, and result in an analogous performance boost. Source code and models
to reproduce the experiments in the paper is made publicly available.
Ken Chatfield Karen Simonyan Andrea Vedaldi Andrew Zisserman