at material recognition.
Some lessons we have learned are:
Training on a dataset which includes the surrounding
context is crucial for real-world material classiﬁcation.
Labeled clicks are cheap and sufﬁcient to train a CNN
alone. However, to obtain high quality segmentation
results, training a CRF on polygons results in much
better boundaries than training on clicks.
Many future avenues of work remain. Expanding the
dataset to a broader range of categories will require new
ways to mine images that have more variety, and new annotation tasks that are cost-effective. Inspired by attributes
for textures [ 3], in the future we would like to identify material attributes and expand our database to include them. We
also believe that further exploration of joint material and object classiﬁcation and segmentation will be fruitful [ 10] and
lead to improvements in both tasks. Our database, trained
models, and all experimental results are available online at
http://minc.cs.cornell.edu/.
Acknowledgements. This work was supported in part by
Google, Amazon AWS for Education, a NSERC PGS-D
scholarship, the National Science Foundation (grants IIS1149393, IIS-1011919, IIS-1161645), and the Intel Science
and Technology Center for Visual Computing.
8
References
Recognizing materials in real-world images is a challenging task. Real-world
materials have rich surface texture, geometry, lighting conditions, and
clutter, which combine to make the problem particularly difficult. In this
paper, we introduce a new, large-scale, open dataset of materials in the wild,
the Materials in Context Database (MINC), and combine this dataset with deep
learning to achieve material recognition and segmentation of images in the
wild.
  MINC is an order of magnitude larger than previous material databases, while
being more diverse and well-sampled across its 23 categories. Using MINC, we
train convolutional neural networks (CNNs) for two tasks: classifying materials
from patches, and simultaneous material recognition and segmentation in full
images. For patch-based classification on MINC we found that the best
performing CNN architectures can achieve 85.2% mean class accuracy. We convert
these trained CNN classifiers into an efficient fully convolutional framework
combined with a fully connected conditional random field (CRF) to predict the
material at every pixel in an image, achieving 73.1% mean class accuracy. Our
experiments demonstrate that having a large, well-sampled dataset such as MINC
is crucial for real-world material recognition and segmentation.
Sean Bell Paul Upchurch Noah Snavely Kavita Bala