216,368 Glass 75,084 Mirror 26,103 Hair
188,491 Metal 64,454 Brick 25,498 Food
147,346 Tile 55,364 Water 23,779 Paper
142,150 Sky 39,612 Other 14,954 Wallpaper
120,957 Foliage 38,975 Plastic
Table 1. MINC patch counts by category. Patches were created
from both OpenSurfaces segments and our newly collected clicks .
See Section 3.2 for details.
Material clicks were collected for both OpenSurfaces
images and the new Houzz images. This allowed us to use
labels from OpenSurfaces to generate the sentinel data; we
included 4 sentinels per task. With this streamlined pipeline
we collected 2,341,473 annotations at an average cost of
$0.00306 per annotation (stage 1: $0.02 / 40 images, stage
2: $0.10 / 50 images, 2, stage 3: $0.10 / 50 points).
Patches. Labeled segments and clicks form the core of
MINC . For training CNNs and other types of classiﬁers,
it is useful to have data in the form of ﬁxed-sized patches.
We convert both forms of data into a uniﬁed dataset format:
square image patches. We use a patch center andpatch scale
Recognizing materials in real-world images is a challenging task. Real-world
materials have rich surface texture, geometry, lighting conditions, and
clutter, which combine to make the problem particularly difficult. In this
paper, we introduce a new, large-scale, open dataset of materials in the wild,
the Materials in Context Database (MINC), and combine this dataset with deep
learning to achieve material recognition and segmentation of images in the
wild.
  MINC is an order of magnitude larger than previous material databases, while
being more diverse and well-sampled across its 23 categories. Using MINC, we
train convolutional neural networks (CNNs) for two tasks: classifying materials
from patches, and simultaneous material recognition and segmentation in full
images. For patch-based classification on MINC we found that the best
performing CNN architectures can achieve 85.2% mean class accuracy. We convert
these trained CNN classifiers into an efficient fully convolutional framework
combined with a fully connected conditional random field (CRF) to predict the
material at every pixel in an image, achieving 73.1% mean class accuracy. Our
experiments demonstrate that having a large, well-sampled dataset such as MINC
is crucial for real-world material recognition and segmentation.
Sean Bell Paul Upchurch Noah Snavely Kavita Bala