detection [14] and segmentation [16] on a number of widely
used benchmarks. Key to their success is the ability to leverage large labelled datasets to learn increasingly complex
transformations of the input to capture invariances. Importantly, CNNs pre-trained on such large datasets have been
shown [6, 14, 30] to contain general-purpose feature extractors, transferrable to many other domains.
Domain transfer in CNNs is usually achieved by using
as features the output of a deep, fully-connected layer of
the network. From the perspective of textures, however,
this choice has three drawbacks. The ﬁrst one (I) is that,
while the convolutional layers are akin to non-linear ﬁlter
banks, the fully connected layers capture their spatial layout. While this may be useful for representing the shape of
an object, it may not be as useful for representing texture. A
second drawback (II) is that the input to the CNN has to be
of ﬁxed size to be compatible with the fully connected layers, which requires an expensive resizing of the input image,
particularly when features are computed for many different
regions [14, 15]. A third and more subtle drawback (III) is
that deeper layers may be more domain-speciﬁc and therefore potentially less transferrable than shallower layers.
Research in texture recognition often concentrates on the problem of material
recognition in uncluttered conditions, an assumption rarely met by
applications. In this work we conduct a first study of material and describable
texture at- tributes recognition in clutter, using a new dataset derived from
the OpenSurface texture repository. Motivated by the challenge posed by this
problem, we propose a new texture descriptor, D-CNN, obtained by Fisher Vector
pooling of a Convolutional Neural Network (CNN) filter bank. D-CNN
substantially improves the state-of-the-art in texture, mate- rial and scene
recognition. Our approach achieves 82.3% accuracy on Flickr material dataset
and 81.1% accuracy on MIT indoor scenes, providing absolute gains of more than
10% over existing approaches. D-CNN easily trans- fers across domains without
requiring feature adaptation as for methods that build on the fully-connected
layers of CNNs. Furthermore, D-CNN can seamlessly incorporate multi-scale
information and describe regions of arbitrary shapes and sizes. Our approach is
particularly suited at lo- calizing stuff categories and obtains
state-of-the-art re- sults on MSRC segmentation dataset, as well as promising
results on recognizing materials and surface attributes in clutter on the
OpenSurfaces dataset.
Mircea Cimpoi Subhransu Maji Andrea Vedaldi