Numerous CNN architectures that improve the
previous state of the art obtained using shallow
representations have been proposed, but choosing
the best one remains an open question. Many are inspired by [13]: DeCAF [7], [11], Caffe [20], Oquab et
al. [8]. Others use larger networks with a smaller
stride of the ﬁrst convolutional layer: Zeiler and
Fergus [19] and OverFeat [10], [9]. Other differences
include the CNN pre-training protocols. Here we
adopt a single learning framework and experiment
with architectures of different complexity exploring
their performance-speed trade-off.
2.3 Scenario 3: Deep representation (CNN) with
pre-training and ﬁne-tuning
In Scenario 2 features are trained on one (large)
dataset and applied to another (usually smaller).
However, it was demonstrated [11] that ﬁne-tuning
a pre-trained CNN on the target data can signiﬁcantly improve the performance. We consider
this scenario separately from that of Scenario 2, as
the image features become dataset-speciﬁc after the
ﬁne-tuning.
3
2.4 Commonalities
We now turn to what is in common across the
scenarios.
The latest generation of Convolutional Neural Networks (CNN) have achieved
impressive results in challenging benchmarks on image recognition and object
detection, significantly raising the interest of the community in these
methods. Nevertheless, it is still unclear how different CNN methods compare
with each other and with previous state-of-the-art shallow representations such
as the Bag-of-Visual-Words and the Improved Fisher Vector. This paper conducts
a rigorous evaluation of these new techniques, exploring different deep
architectures and comparing them on a common ground, identifying and disclosing
important implementation details. We identify several useful properties of
CNN-based representations, including the fact that the dimensionality of the
CNN output layer can be reduced significantly without having an adverse effect
on performance. We also identify aspects of deep and shallow methods that can
be successfully shared. In particular, we show that the data augmentation
techniques commonly applied to CNN-based methods can also be applied to shallow
methods, and result in an analogous performance boost. Source code and models
to reproduce the experiments in the paper is made publicly available.
Ken Chatfield Karen Simonyan Andrea Vedaldi Andrew Zisserman