SIFTFigure 4. CNN ﬁlterbank analysis for VGG-M. Performance of
ﬁlter banks extracted from various layers network are shown on
various datasets. For each layer convf1; : : : ; 5gwe show, the size
of the receptive ﬁeld [ NN], and FBD, where FBis the size
of ﬁlter bank and Dis the dictionary size in the FV representation.
The performance using SIFT is shown in black plus (+) marks.
brackets (FC-CNN was too slow for our experiments). The
results are comparable to those using crisp regions, and we
obtain 55:7%accuracy on the OS dataset. Our initial attempts at schemes such as non-maximum suppression of
overlapping regions that are quite successful for object segmentation [16] performed rather poorly. We believe this is
because unlike objects, material information is fairly localized and highly irregularly shaped in an image. However,
there is room for improvement by combining evidence from
multiple segmentations.
4.4. Convolutional layer analysis
We study the performance of ﬁlter banks extracted from
different layers of a CNN in the FV-CNN framework. We
Research in texture recognition often concentrates on the problem of material
recognition in uncluttered conditions, an assumption rarely met by
applications. In this work we conduct a first study of material and describable
texture at- tributes recognition in clutter, using a new dataset derived from
the OpenSurface texture repository. Motivated by the challenge posed by this
problem, we propose a new texture descriptor, D-CNN, obtained by Fisher Vector
pooling of a Convolutional Neural Network (CNN) filter bank. D-CNN
substantially improves the state-of-the-art in texture, mate- rial and scene
recognition. Our approach achieves 82.3% accuracy on Flickr material dataset
and 81.1% accuracy on MIT indoor scenes, providing absolute gains of more than
10% over existing approaches. D-CNN easily trans- fers across domains without
requiring feature adaptation as for methods that build on the fully-connected
layers of CNNs. Furthermore, D-CNN can seamlessly incorporate multi-scale
information and describe regions of arbitrary shapes and sizes. Our approach is
particularly suited at lo- calizing stuff categories and obtains
state-of-the-art re- sults on MSRC segmentation dataset, as well as promising
results on recognizing materials and surface attributes in clutter on the
OpenSurfaces dataset.
Mircea Cimpoi Subhransu Maji Andrea Vedaldi