attributes from the DTD [7]. Since the OS segments do
not trigger with sufﬁcient frequency all the 47 attributes,
the evaluation is restricted to eleven of them for which it
was possible to identify at least 100 matching segments.2
The attributes were manually labelled in the 53,915 segments retained for materials. We refer to this data as
OSA. The complete list of images, segments, labels, and
splits are available at http://www.robots.ox.ac.
uk/˜vgg/data/wildtex/ .
3. Method
This section describes the methodological contributions
of this paper: region description and segmentation.
3.1. Region description
This section introduces a number of visual descriptors
suitable to model the appearance of image regions. Texture
is traditionally described by orderless pooling of ﬁlter bank
responses as, unlike in objects, the overall shape information is usually unimportant. However, small under-sampled
textures may beneﬁt if recognized in the context of an object. Thus, the primacy of orderless pooling may not always
hold in the recognition of textures in natural conditions.
In order to explore the interplay between shape and orderless pooling, we evaluate two corresponding region descriptors: FC-CNN for shape and FV-CNN for texture.
Research in texture recognition often concentrates on the problem of material
recognition in uncluttered conditions, an assumption rarely met by
applications. In this work we conduct a first study of material and describable
texture at- tributes recognition in clutter, using a new dataset derived from
the OpenSurface texture repository. Motivated by the challenge posed by this
problem, we propose a new texture descriptor, D-CNN, obtained by Fisher Vector
pooling of a Convolutional Neural Network (CNN) filter bank. D-CNN
substantially improves the state-of-the-art in texture, mate- rial and scene
recognition. Our approach achieves 82.3% accuracy on Flickr material dataset
and 81.1% accuracy on MIT indoor scenes, providing absolute gains of more than
10% over existing approaches. D-CNN easily trans- fers across domains without
requiring feature adaptation as for methods that build on the fully-connected
layers of CNNs. Furthermore, D-CNN can seamlessly incorporate multi-scale
information and describe regions of arbitrary shapes and sizes. Our approach is
particularly suited at lo- calizing stuff categories and obtains
state-of-the-art re- sults on MSRC segmentation dataset, as well as promising
results on recognizing materials and surface attributes in clutter on the
OpenSurfaces dataset.
Mircea Cimpoi Subhransu Maji Andrea Vedaldi