building features that are invariant to pose and illumination.
Later, databases with more diverse examples from each ma-terial category began to appear, such as KTH-TIPS [ 9,2],
and led explorations of how to generalize from one example
of a material to another—from one sample of wood to a completely different sample, for instance. Real-world texture
attributes have also recently been explored [3].
In the domain of categorical material databases, Sharan et
al. released FMD [ 26] (described above). Subsequently,
Bell et al. released OpenSurfaces [ 1] which contains over
20,000 real-world scenes labeled with both materials and objects, using a multi-stage crowdsourcing pipeline. Because
OpenSurfaces images are drawn from consumer photos on
Flickr, material samples have real-world context, in contrast
to prior databases (CUReT, KTH-TIPS, FMD) which feature cropped stand-alone samples. While OpenSurfaces is a
good starting point for a material database, we substantially
expand it with millions of new labels.
Material recognition. Much prior work on material recognition has focused on the classiﬁcation problem (categorizing
an image patch into a set of material categories), often using
Recognizing materials in real-world images is a challenging task. Real-world
materials have rich surface texture, geometry, lighting conditions, and
clutter, which combine to make the problem particularly difficult. In this
paper, we introduce a new, large-scale, open dataset of materials in the wild,
the Materials in Context Database (MINC), and combine this dataset with deep
learning to achieve material recognition and segmentation of images in the
wild.
  MINC is an order of magnitude larger than previous material databases, while
being more diverse and well-sampled across its 23 categories. Using MINC, we
train convolutional neural networks (CNNs) for two tasks: classifying materials
from patches, and simultaneous material recognition and segmentation in full
images. For patch-based classification on MINC we found that the best
performing CNN architectures can achieve 85.2% mean class accuracy. We convert
these trained CNN classifiers into an efficient fully convolutional framework
combined with a fully connected conditional random field (CRF) to predict the
material at every pixel in an image, achieving 73.1% mean class accuracy. Our
experiments demonstrate that having a large, well-sampled dataset such as MINC
is crucial for real-world material recognition and segmentation.
Sean Bell Paul Upchurch Noah Snavely Kavita Bala