based on the Fisher Vector (FV). Differently from FC-CNN,
FV pools local features densely within the described regions
removing global spatial information , and is therefore more
apt at describing textures than objects. Here FV is computed
on the output of a single (last) convolutional layer of the
CNN, but we compared features from other layers as well
(Sect 4.4). By avoiding the computation of the fully connected layers, the input image does not need to be rescaled
to a speciﬁc size; in fact, the dense convolutional features
are extracted at multiple scales and pooled into a single FV
just like for SIFT. The pooled convolutional features are extracted immediately after the last linear ﬁltering operator
and are not otherwise normalised.
The FV-CNN descriptor is related to the one proposed
by Gong et al. [15]. There VLAD pooling, which is similar to FV , is applied to FC-CNN-like descriptors extracted
from densely sampled image windows. A key difference of
FV-CNN is that dense features are extracted from the convolutional rather than fully-connected layers. This is more
natural, signiﬁcantly more efﬁcient (as it does not require
Research in texture recognition often concentrates on the problem of material
recognition in uncluttered conditions, an assumption rarely met by
applications. In this work we conduct a first study of material and describable
texture at- tributes recognition in clutter, using a new dataset derived from
the OpenSurface texture repository. Motivated by the challenge posed by this
problem, we propose a new texture descriptor, D-CNN, obtained by Fisher Vector
pooling of a Convolutional Neural Network (CNN) filter bank. D-CNN
substantially improves the state-of-the-art in texture, mate- rial and scene
recognition. Our approach achieves 82.3% accuracy on Flickr material dataset
and 81.1% accuracy on MIT indoor scenes, providing absolute gains of more than
10% over existing approaches. D-CNN easily trans- fers across domains without
requiring feature adaptation as for methods that build on the fully-connected
layers of CNNs. Furthermore, D-CNN can seamlessly incorporate multi-scale
information and describe regions of arbitrary shapes and sizes. Our approach is
particularly suited at lo- calizing stuff categories and obtains
state-of-the-art re- sults on MSRC segmentation dataset, as well as promising
results on recognizing materials and surface attributes in clutter on the
OpenSurfaces dataset.
Mircea Cimpoi Subhransu Maji Andrea Vedaldi