orderless pooling of convolutional neural network features
is a remarkably good texture descriptor, versatile enough to
dubbed as a scene and object descriptor, resulting in new
state-of-the-art performance in several benchmarks.
brick cardboard carpet_rug ceramic concrete fabric_cloth food glass
granite_marble hair other laminate leather metal painted paper_tissue
plastic_clear plastic_opaque stone tile wallpaper wood(a) (b) (c) (d) (e) (f)
Figure 2. OS material recognition results. Example test image with material recognition and segmentation on the OS dataset. (a) original
image. (b) ground truth segmentations from the OpenSurfaces repository (note that not all pixels are annotated). (c) FC-CNN and crispregion proposals segmentation results. (d) incorrectly predicted pixels (restricted to the ones annotated). (e-f) the same, but for FV-CNN.
building grass tree cow sheep sky aeroplane water
face car bicycle flower sign bird book chair
road cat dog body boat
(a) (b) (c) (d) (e) (f)
Figure 3. MSRC object segmentation results. (a) image, (b) ground-truth, (c-d) FC-CNN, (d-e) FV-CNN segmentation and errors.
Research in texture recognition often concentrates on the problem of material
recognition in uncluttered conditions, an assumption rarely met by
applications. In this work we conduct a first study of material and describable
texture at- tributes recognition in clutter, using a new dataset derived from
the OpenSurface texture repository. Motivated by the challenge posed by this
problem, we propose a new texture descriptor, D-CNN, obtained by Fisher Vector
pooling of a Convolutional Neural Network (CNN) filter bank. D-CNN
substantially improves the state-of-the-art in texture, mate- rial and scene
recognition. Our approach achieves 82.3% accuracy on Flickr material dataset
and 81.1% accuracy on MIT indoor scenes, providing absolute gains of more than
10% over existing approaches. D-CNN easily trans- fers across domains without
requiring feature adaptation as for methods that build on the fully-connected
layers of CNNs. Furthermore, D-CNN can seamlessly incorporate multi-scale
information and describe regions of arbitrary shapes and sizes. Our approach is
particularly suited at lo- calizing stuff categories and obtains
state-of-the-art re- sults on MSRC segmentation dataset, as well as promising
results on recognizing materials and surface attributes in clutter on the
OpenSurfaces dataset.
Mircea Cimpoi Subhransu Maji Andrea Vedaldi