graphics. OS comprises 25,357 images, each containing a
number of high-quality texture/material segments. Many of
these segments are annotated with additional attributes such
as the material name, the viewpoint, the BRDF, and the object class. Not all segments have a complete set of annotations; the experiments in this paper focus on the 58,928 that
contain material names. Since material classes are highly
unbalanced, only the materials that contain at least 400 examples are considered. This result in 53,915 annotated
material segments in 10,422 images spanning 22 different
classes.1Images are split evenly into training, validation,
and test subsets with 3,474 images each. Segment sizes are
highly variable, with half of them being relatively small,
with an area smaller than 6464pixels. While the lack
of exhaustive annotations makes it impossible to deﬁne a
complete background class, several less common materials
(including for example segments that annotators could not
assign to a material) are merged in an “other” class that acts
as pseudo-background.
In order to study perceptual properties as well as materials, we augment the OS dataset with some of the 47
attributes from the DTD [7]. Since the OS segments do
not trigger with sufﬁcient frequency all the 47 attributes,
Research in texture recognition often concentrates on the problem of material
recognition in uncluttered conditions, an assumption rarely met by
applications. In this work we conduct a first study of material and describable
texture at- tributes recognition in clutter, using a new dataset derived from
the OpenSurface texture repository. Motivated by the challenge posed by this
problem, we propose a new texture descriptor, D-CNN, obtained by Fisher Vector
pooling of a Convolutional Neural Network (CNN) filter bank. D-CNN
substantially improves the state-of-the-art in texture, mate- rial and scene
recognition. Our approach achieves 82.3% accuracy on Flickr material dataset
and 81.1% accuracy on MIT indoor scenes, providing absolute gains of more than
10% over existing approaches. D-CNN easily trans- fers across domains without
requiring feature adaptation as for methods that build on the fully-connected
layers of CNNs. Furthermore, D-CNN can seamlessly incorporate multi-scale
information and describe regions of arbitrary shapes and sizes. Our approach is
particularly suited at lo- calizing stuff categories and obtains
state-of-the-art re- sults on MSRC segmentation dataset, as well as promising
results on recognizing materials and surface attributes in clutter on the
OpenSurfaces dataset.
Mircea Cimpoi Subhransu Maji Andrea Vedaldi