and architectures as ﬁlter banks, concluding that: SIFT is
competitive only with the ﬁrst few layers of a CNN (Fig. 4)
and that signiﬁcant improvement to the underlying CNN architecture, such as the ones achieved by the very deep models of Simonyan and Zisserman [39], directly translate into
much better ﬁlter banks for texture recognition.
2. Texture recognition in clutter
A contribution of this work is the analysis of materials
and texture attributes in realistic imaging conditions. Earlier datasets such as KTH-TIPS were collected in controlled
conditions, which makes their applicability to natural images unclear. More recent datasets such as FMD and DTD
remove this limitation by building on images downloaded
from the Internet, dubbed images “in the wild”. However,
in these datasets texture always ﬁll the ﬁeld of view of the
camera. In this paper we remove this limitation by experimenting for the ﬁrst time with a large dataset of textures
collected in the wild and in cluttered conditions.
In particular, we build on the Open Surfaces (OS) dataset
that was recently introduced by Bell et al. [4] in computer
graphics. OS comprises 25,357 images, each containing a
number of high-quality texture/material segments. Many of
Research in texture recognition often concentrates on the problem of material
recognition in uncluttered conditions, an assumption rarely met by
applications. In this work we conduct a first study of material and describable
texture at- tributes recognition in clutter, using a new dataset derived from
the OpenSurface texture repository. Motivated by the challenge posed by this
problem, we propose a new texture descriptor, D-CNN, obtained by Fisher Vector
pooling of a Convolutional Neural Network (CNN) filter bank. D-CNN
substantially improves the state-of-the-art in texture, mate- rial and scene
recognition. Our approach achieves 82.3% accuracy on Flickr material dataset
and 81.1% accuracy on MIT indoor scenes, providing absolute gains of more than
10% over existing approaches. D-CNN easily trans- fers across domains without
requiring feature adaptation as for methods that build on the fully-connected
layers of CNNs. Furthermore, D-CNN can seamlessly incorporate multi-scale
information and describe regions of arbitrary shapes and sizes. Our approach is
particularly suited at lo- calizing stuff categories and obtains
state-of-the-art re- sults on MSRC segmentation dataset, as well as promising
results on recognizing materials and surface attributes in clutter on the
OpenSurfaces dataset.
Mircea Cimpoi Subhransu Maji Andrea Vedaldi