Input image (a)Labels from CRF (b)Labels from CRF
(test set) trained on segments trained on clicks
Figure 9. Optimizing for click accuracy leads to sloppy boundaries. In(a), we optimize for mean class accuracy across segments,
resulting in high quality boundaries. In (b), we optimize for mean
class accuracy at click locations. Since the clicks are not necessarily close to object boundaries, there is no penalty for sloppy
boundaries. CNN: GoogLeNet (without average pooling).
5.2. Full scene material segmentation
The full test set for our patch dataset contains 41,801
photos, but most of them contain only a few labels. Since
we want to evaluate the per-pixel classiﬁcation performance,
we select a subset of 5,000 test photos such that each photo
contains a large number of segments and clicks, and small
categories are well sampled. We greedily solve for the best
such set of photos. We similarly select 2,500 of 25,844
validation photos. Our splits for all experiments are included
online with the dataset. To train the CRF for our model, we
try various parameter settings ( p,ab,L,wp) and select the
model that performs best on the validation set. In total, we
Recognizing materials in real-world images is a challenging task. Real-world
materials have rich surface texture, geometry, lighting conditions, and
clutter, which combine to make the problem particularly difficult. In this
paper, we introduce a new, large-scale, open dataset of materials in the wild,
the Materials in Context Database (MINC), and combine this dataset with deep
learning to achieve material recognition and segmentation of images in the
wild.
  MINC is an order of magnitude larger than previous material databases, while
being more diverse and well-sampled across its 23 categories. Using MINC, we
train convolutional neural networks (CNNs) for two tasks: classifying materials
from patches, and simultaneous material recognition and segmentation in full
images. For patch-based classification on MINC we found that the best
performing CNN architectures can achieve 85.2% mean class accuracy. We convert
these trained CNN classifiers into an efficient fully convolutional framework
combined with a fully connected conditional random field (CRF) to predict the
material at every pixel in an image, achieving 73.1% mean class accuracy. Our
experiments demonstrate that having a large, well-sampled dataset such as MINC
is crucial for real-world material recognition and segmentation.
Sean Bell Paul Upchurch Noah Snavely Kavita Bala