Adding half-strides resulted in a minor 0.2% improvement
in mean class accuracy across segments (after applying the
dense CRF, described below), and about the same mean class
accuracy at click locations.
The input image is resized so that a patch maps to a
256x256 square. Thus, for a network trained at patch scale
s, the resized input has smaller dimension d= 256=s. Note
thatdis inversely proportional to scale, so increased context
leads to lower spatial resolution. We then add padding so
that the output probability map is aligned with the input
when upsampled. We repeat this at 3 different scales (smaller
dimensiond=p
2,d,dp
2), upsample each output probability
map with bilinear interpolation, and average the predictions.
To make the next step more efﬁcient, we upsample the output
to a ﬁxed smaller dimension of 550.
We then use the dense CRF of Kr¨ahenb ¨uhlet al. [12] to
predict a label at every pixel, using the following energy:
E(xjI) =X
i i(xi) +X
i<j ij(xi;xj) (1)
 i(xi) =logpi(xi) (2)
Recognizing materials in real-world images is a challenging task. Real-world
materials have rich surface texture, geometry, lighting conditions, and
clutter, which combine to make the problem particularly difficult. In this
paper, we introduce a new, large-scale, open dataset of materials in the wild,
the Materials in Context Database (MINC), and combine this dataset with deep
learning to achieve material recognition and segmentation of images in the
wild.
  MINC is an order of magnitude larger than previous material databases, while
being more diverse and well-sampled across its 23 categories. Using MINC, we
train convolutional neural networks (CNNs) for two tasks: classifying materials
from patches, and simultaneous material recognition and segmentation in full
images. For patch-based classification on MINC we found that the best
performing CNN architectures can achieve 85.2% mean class accuracy. We convert
these trained CNN classifiers into an efficient fully convolutional framework
combined with a fully connected conditional random field (CRF) to predict the
material at every pixel in an image, achieving 73.1% mean class accuracy. Our
experiments demonstrate that having a large, well-sampled dataset such as MINC
is crucial for real-world material recognition and segmentation.
Sean Bell Paul Upchurch Noah Snavely Kavita Bala