and total accuracy on the test set. When training, we optimize the
CRF parameters for mean class accuracy, but report both mean class
and total accuracy (mean accuracy across all examples). In one
experiment (a), we train and test only on segments; in a separate
experiment (b), we train and test only on clicks. Accuracies for
segments are averaged across all pixels that fall in that segment.
combined with either arithmetic or geometric mean.
Since we have two types of data, clicks andsegments , we
run two sets of experiments: (a) we train and test only on
segments, and in a separate experiment (b) we train and test
only on clicks. These two training objectives result in very
different behavior, as illustrated in Figure 9. In experiment
(a), the accuracy across segments are optimized, producing
clean boundaries. In experiment (b), the CRF maximizes
accuracy only at click locations, thus resulting in sloppy
boundaries. As shown in Table 4, the numerical scores for
the two experiments are also very different: segments are
more challenging than clicks. While clicks are sufﬁcient to
train a CNN, they are not sufﬁcient to train a CRF.
Focusing on segmentation accuracy, we see from Table 4(a) that our best single model is GoogLeNet without
Recognizing materials in real-world images is a challenging task. Real-world
materials have rich surface texture, geometry, lighting conditions, and
clutter, which combine to make the problem particularly difficult. In this
paper, we introduce a new, large-scale, open dataset of materials in the wild,
the Materials in Context Database (MINC), and combine this dataset with deep
learning to achieve material recognition and segmentation of images in the
wild.
  MINC is an order of magnitude larger than previous material databases, while
being more diverse and well-sampled across its 23 categories. Using MINC, we
train convolutional neural networks (CNNs) for two tasks: classifying materials
from patches, and simultaneous material recognition and segmentation in full
images. For patch-based classification on MINC we found that the best
performing CNN architectures can achieve 85.2% mean class accuracy. We convert
these trained CNN classifiers into an efficient fully convolutional framework
combined with a fully connected conditional random field (CRF) to predict the
material at every pixel in an image, achieving 73.1% mean class accuracy. Our
experiments demonstrate that having a large, well-sampled dataset such as MINC
is crucial for real-world material recognition and segmentation.
Sean Bell Paul Upchurch Noah Snavely Kavita Bala