that deeper layers may be more domain-speciﬁc and therefore potentially less transferrable than shallower layers.
The second contribution of this paper is FV-CNN
(Sect. 3), a pooling method that overcomes these drawbacks . The idea is to regard the convolutional layers of a
CNN as a ﬁlter bank and build an orderless representation
using FV as a pooling mechanism, as is commonly done
in the bag-of-words approaches. Although the suggested
change is simple, the approach is remarkably ﬂexible and
effective. First, pooling is orderless and multi-scale, hence
suitable for textures. Second, any image size can be processed by convolutional layers, avoiding costly resizing operations. Third, convolutional ﬁlters, pooled by FV-CNN,
are shown to transfer more easily than fully-connected ones
even without ﬁne-tuning. While other authors [15, 18] have
recently proposed alternative pooling strategies for CNNs,
we show that our method is more natural, faster and often
signiﬁcantly more accurate.
Thethird contribution of the paper is a thorough evaluation of these descriptors on a variety of benchmarks, from
textures to objects (Sect. 4). In textures, we evaluate material and describable attributes recognition and segmentation
Research in texture recognition often concentrates on the problem of material
recognition in uncluttered conditions, an assumption rarely met by
applications. In this work we conduct a first study of material and describable
texture at- tributes recognition in clutter, using a new dataset derived from
the OpenSurface texture repository. Motivated by the challenge posed by this
problem, we propose a new texture descriptor, D-CNN, obtained by Fisher Vector
pooling of a Convolutional Neural Network (CNN) filter bank. D-CNN
substantially improves the state-of-the-art in texture, mate- rial and scene
recognition. Our approach achieves 82.3% accuracy on Flickr material dataset
and 81.1% accuracy on MIT indoor scenes, providing absolute gains of more than
10% over existing approaches. D-CNN easily trans- fers across domains without
requiring feature adaptation as for methods that build on the fully-connected
layers of CNNs. Furthermore, D-CNN can seamlessly incorporate multi-scale
information and describe regions of arbitrary shapes and sizes. Our approach is
particularly suited at lo- calizing stuff categories and obtains
state-of-the-art re- sults on MSRC segmentation dataset, as well as promising
results on recognizing materials and surface attributes in clutter on the
OpenSurfaces dataset.
Mircea Cimpoi Subhransu Maji Andrea Vedaldi