lines: other categories. CNN: AlexNet . While most materials are
optimally recognized at 23.3% or 32% patch scale, recognition of
sky,wallpaper andmirror improve with increasing context.
contextual information. Holding patch centers ﬁxed we varied scale and evaluated classiﬁcation accuracy with AlexNet .
Results and a visualization of patch scales are shown in Figure 5. Scale 32% performs the best. Individual categories
had peaks at middle scale with some exceptions; we ﬁnd
thatmirror ,wallpaper andskyimprove with increasing context (Figure 7). We used 23.3% (which has nearly the same
accuracy but higher spatial resolution) for our experiments.
Dataset size. To measure the effect of size on patch classiﬁcation accuracy we trained AlexNet with patches from
randomly sampled subsets of all 369,104 training images
and tested on our full test set (Figure 6). As expected, using
more data improved performance. In addition, we still have
not saturated performance with 2.5 million training patches;
even higher accuracies may be possible with more training
data (though with diminishing returns).
Dataset balance. Although we’ve shown that more data is
better we also ﬁnd that a balanced dataset is more effective.
We trained AlexNet with all patches of our smallest category
Recognizing materials in real-world images is a challenging task. Real-world
materials have rich surface texture, geometry, lighting conditions, and
clutter, which combine to make the problem particularly difficult. In this
paper, we introduce a new, large-scale, open dataset of materials in the wild,
the Materials in Context Database (MINC), and combine this dataset with deep
learning to achieve material recognition and segmentation of images in the
wild.
  MINC is an order of magnitude larger than previous material databases, while
being more diverse and well-sampled across its 23 categories. Using MINC, we
train convolutional neural networks (CNNs) for two tasks: classifying materials
from patches, and simultaneous material recognition and segmentation in full
images. For patch-based classification on MINC we found that the best
performing CNN architectures can achieve 85.2% mean class accuracy. We convert
these trained CNN classifiers into an efficient fully convolutional framework
combined with a fully connected conditional random field (CRF) to predict the
material at every pixel in an image, achieving 73.1% mean class accuracy. Our
experiments demonstrate that having a large, well-sampled dataset such as MINC
is crucial for real-world material recognition and segmentation.
Sean Bell Paul Upchurch Noah Snavely Kavita Bala