workers would get frustrated if the material was absent in
too many of the photos. Thus, we added an initial ﬁrst
stage where workers ﬁlter out such photos. To increase
the accuracy of our labels, we verify the click labels by
asking different workers to specify the material for each
click without providing them with the label from the previous
stage.
To ensure that we obtain high quality annotations and
avoid collecting labels from workers who are not making an
effort, we include secret known answers (sentinels) in the
ﬁrst and third stages, and block workers with an accuracy
below 50% and 85% respectively. We do not use sentinels
in the second stage since it would require per-pixel ground
truth labels, and it turned out not to be necessary. Workers
generally performed all three tasks so we could identify bad
workers in the ﬁrst or third task.Patches Category Patches Category Patches Category
564,891 Wood 114,085 Polished stone 35,246 Skin
465,076 Painted 98,891 Carpet 29,616 Stone
397,982 Fabric 83,644 Leather 28,108 Ceramic
216,368 Glass 75,084 Mirror 26,103 Hair
188,491 Metal 64,454 Brick 25,498 Food
147,346 Tile 55,364 Water 23,779 Paper
Recognizing materials in real-world images is a challenging task. Real-world
materials have rich surface texture, geometry, lighting conditions, and
clutter, which combine to make the problem particularly difficult. In this
paper, we introduce a new, large-scale, open dataset of materials in the wild,
the Materials in Context Database (MINC), and combine this dataset with deep
learning to achieve material recognition and segmentation of images in the
wild.
  MINC is an order of magnitude larger than previous material databases, while
being more diverse and well-sampled across its 23 categories. Using MINC, we
train convolutional neural networks (CNNs) for two tasks: classifying materials
from patches, and simultaneous material recognition and segmentation in full
images. For patch-based classification on MINC we found that the best
performing CNN architectures can achieve 85.2% mean class accuracy. We convert
these trained CNN classifiers into an efficient fully convolutional framework
combined with a fully connected conditional random field (CRF) to predict the
material at every pixel in an image, achieving 73.1% mean class accuracy. Our
experiments demonstrate that having a large, well-sampled dataset such as MINC
is crucial for real-world material recognition and segmentation.
Sean Bell Paul Upchurch Noah Snavely Kavita Bala