material categories are under-sampled, with only tens of
images.
A major contribution of our paper is a new, well-sampled
material dataset, called the Materials in Context Database
(MINC ), with 3 million material samples. MINC is more
diverse, has more examples in less common categories, and
is much larger than existing datasets. MINC draws data from
both Flickr images, which include many “regular” scenes,
as well as Houzz images from professional photographers of
staged interiors. These sources of images each have different
characteristics that together increase the range of materials
that can be recognized. See Figure 2 for examples of our
data. We make our full dataset available online at http:
//minc.cs.cornell.edu/.
We use this data for material recognition by training different CNN architectures on this new dataset. We perform
experiments that illustrate the effect of network architecture, image context, and training data size on subregions
(i.e., patches) of a full scene image. Further, we build on
our patch classiﬁcation results and demonstrate simultaneous material recognition and segmentation of an image by
performing dense classiﬁcation over the image with a fully
connected conditional random ﬁeld (CRF) model [ 12]. By
Recognizing materials in real-world images is a challenging task. Real-world
materials have rich surface texture, geometry, lighting conditions, and
clutter, which combine to make the problem particularly difficult. In this
paper, we introduce a new, large-scale, open dataset of materials in the wild,
the Materials in Context Database (MINC), and combine this dataset with deep
learning to achieve material recognition and segmentation of images in the
wild.
  MINC is an order of magnitude larger than previous material databases, while
being more diverse and well-sampled across its 23 categories. Using MINC, we
train convolutional neural networks (CNNs) for two tasks: classifying materials
from patches, and simultaneous material recognition and segmentation in full
images. For patch-based classification on MINC we found that the best
performing CNN architectures can achieve 85.2% mean class accuracy. We convert
these trained CNN classifiers into an efficient fully convolutional framework
combined with a fully connected conditional random field (CRF) to predict the
material at every pixel in an image, achieving 73.1% mean class accuracy. Our
experiments demonstrate that having a large, well-sampled dataset such as MINC
is crucial for real-world material recognition and segmentation.
Sean Bell Paul Upchurch Noah Snavely Kavita Bala