Deep Filter Banks for Texture Recognition and Segmentation
Mircea Cimpoi
University of Oxford
mircea@robots.ox.ac.ukSubhransu Maji
University of Massachusetts, Amherst
smaji@cs.umass.eduAndrea Vedaldi
University of Oxford
vedaldi@robots.ox.ac.uk
Abstract
Research in texture recognition often concentrates on the
problem of material recognition in uncluttered conditions,
an assumption rarely met by applications. In this work we
conduct a ﬁrst study of material and describable texture attributes recognition in clutter, using a new dataset derived
from the OpenSurface texture repository. Motivated by the
challenge posed by this problem, we propose a new texture
descriptor, FV-CNN, obtained by Fisher Vector pooling of a
Convolutional Neural Network (CNN) ﬁlter bank. FV-CNN
substantially improves the state-of-the-art in texture, material and scene recognition. Our approach achieves 79.8%
accuracy on Flickr material dataset and 81% accuracy on
MIT indoor scenes, providing absolute gains of more than
10% over existing approaches. FV-CNN easily transfers
across domains without requiring feature adaptation as for
methods that build on the fully-connected layers of CNNs.
Research in texture recognition often concentrates on the problem of material
recognition in uncluttered conditions, an assumption rarely met by
applications. In this work we conduct a first study of material and describable
texture at- tributes recognition in clutter, using a new dataset derived from
the OpenSurface texture repository. Motivated by the challenge posed by this
problem, we propose a new texture descriptor, D-CNN, obtained by Fisher Vector
pooling of a Convolutional Neural Network (CNN) filter bank. D-CNN
substantially improves the state-of-the-art in texture, mate- rial and scene
recognition. Our approach achieves 82.3% accuracy on Flickr material dataset
and 81.1% accuracy on MIT indoor scenes, providing absolute gains of more than
10% over existing approaches. D-CNN easily trans- fers across domains without
requiring feature adaptation as for methods that build on the fully-connected
layers of CNNs. Furthermore, D-CNN can seamlessly incorporate multi-scale
information and describe regions of arbitrary shapes and sizes. Our approach is
particularly suited at lo- calizing stuff categories and obtains
state-of-the-art re- sults on MSRC segmentation dataset, as well as promising
results on recognizing materials and surface attributes in clutter on the
OpenSurfaces dataset.
Mircea Cimpoi Subhransu Maji Andrea Vedaldi