object recognition and scene classiﬁcation. Material recognition is similarly poised for advancement through large-scale
data and learning. To date, progress in material recognition
has been facilitated by moderate-sized datasets like the Flickr
Material Database (FMD) [ 26]. FMD contains ten material
categories, each with 100 samples drawn from Flickr photos.
These images were carefully selected to illustrate a wide
range of appearances for these categories. FMD has been
used in research on new features and learning methods for
material perception and recognition [ 17,10,20,25]. While
FMD was an important step towards material recognition, it
is not sufﬁcient for classifying materials in real-world im1arXiv:1412.0623v2  [cs.CV]  14 Apr 2015
agery. This is due to the relatively small set of categories,
the relatively small number of images per category, and also
because the dataset has been designed around hand-picked
iconic images of materials. The OpenSurfaces dataset [ 1]
addresses some of these problems by introducing 105,000
material segmentations from real-world images, and is signiﬁcantly larger than FMD. However, in OpenSurfaces many
material categories are under-sampled, with only tens of
images.
Recognizing materials in real-world images is a challenging task. Real-world
materials have rich surface texture, geometry, lighting conditions, and
clutter, which combine to make the problem particularly difficult. In this
paper, we introduce a new, large-scale, open dataset of materials in the wild,
the Materials in Context Database (MINC), and combine this dataset with deep
learning to achieve material recognition and segmentation of images in the
wild.
  MINC is an order of magnitude larger than previous material databases, while
being more diverse and well-sampled across its 23 categories. Using MINC, we
train convolutional neural networks (CNNs) for two tasks: classifying materials
from patches, and simultaneous material recognition and segmentation in full
images. For patch-based classification on MINC we found that the best
performing CNN architectures can achieve 85.2% mean class accuracy. We convert
these trained CNN classifiers into an efficient fully convolutional framework
combined with a fully connected conditional random field (CRF) to predict the
material at every pixel in an image, achieving 73.1% mean class accuracy. Our
experiments demonstrate that having a large, well-sampled dataset such as MINC
is crucial for real-world material recognition and segmentation.
Sean Bell Paul Upchurch Noah Snavely Kavita Bala