3.1. Sources of data
We decided to start with the public, crowdsourced
OpenSurfaces dataset [ 1] as the seed for MINC since it is
drawn from Flickr imagery of everyday, real-world scenes
with reasonable diversity. Furthermore, it has a large number
of categories and the most samples of all prior databases.
While OpenSurfaces data is a good start, it has a few limitations. Many categories in OpenSurfaces are not well sampled. While the largest category, wood , has nearly 20K samples, smaller categories, such as water , have only tens of examples. This imbalance is due to the way the OpenSurfaces
dataset was annotated; workers on Amazon Mechanical Turk
(AMT) were free to choose any material subregion to segment. Workers often gravitated towards certain common
types of materials or salient objects, rather than being encouraged to label a diverse set of materials. Further, the
images come from a single source (Flickr).
We decided to augment OpenSurfaces with substantially
more data, especially for underrepresented material cate-gories, with the initial goal of gathering at least 10K samples
per material category. We decided to gather this data from
another source of imagery, professional photos on the interior design website Houzz (houzz.com). Our motivation for
using this different source of data was that, despite Houzz
Recognizing materials in real-world images is a challenging task. Real-world
materials have rich surface texture, geometry, lighting conditions, and
clutter, which combine to make the problem particularly difficult. In this
paper, we introduce a new, large-scale, open dataset of materials in the wild,
the Materials in Context Database (MINC), and combine this dataset with deep
learning to achieve material recognition and segmentation of images in the
wild.
  MINC is an order of magnitude larger than previous material databases, while
being more diverse and well-sampled across its 23 categories. Using MINC, we
train convolutional neural networks (CNNs) for two tasks: classifying materials
from patches, and simultaneous material recognition and segmentation in full
images. For patch-based classification on MINC we found that the best
performing CNN architectures can achieve 85.2% mean class accuracy. We convert
these trained CNN classifiers into an efficient fully convolutional framework
combined with a fully connected conditional random field (CRF) to predict the
material at every pixel in an image, achieving 73.1% mean class accuracy. Our
experiments demonstrate that having a large, well-sampled dataset such as MINC
is crucial for real-world material recognition and segmentation.
Sean Bell Paul Upchurch Noah Snavely Kavita Bala