of ground truth data that illustrate the different properties of
our two end-objectives.
The number of hidden layers in the ﬁrst and second fully
connected layers of both branches are 1024 and512, respectively. Both branches optimize the sum of squared difference loss over the (binary or continuous) labels. At test-ing time, the scalar outputs computed from these two subnetworks are averaged to produce a ﬁnal score indicative of
the probability that the candidate point is a contour. Visualization of this architecture is presented in Fig. 5.
In order to train our sub-network, we generate patch examples and labels using training images with ground truth
annotations from multiple human labelers. To generate the
binary labels, we ﬁrst sample 40;000positive examples that
were marked as contours by at least one of the labelers. To
generate negative examples we consider the points that were
selected as candidate contour points by the Canny edge detector but that have not been marked as contours by any of
the human labelers. These are essentially false positives.
For training, we use a random subset of 40;000 of such
points in order to have equal proportion of negative and
positive examples. These 80;000examples are then used
to train our classiﬁcation sub-network.
Contour detection has been a fundamental component in many image segmentation
and object detection systems. Most previous work utilizes low-level features
such as texture or saliency to detect contours and then use them as cues for a
higher-level task such as object detection. However, we claim that recognizing
objects and predicting contours are two mutually related tasks. Contrary to
traditional approaches, we show that we can invert the commonly established
pipeline: instead of detecting contours with low-level cues for a higher-level
recognition task, we exploit object-related features as high-level cues for
contour detection.
  We achieve this goal by means of a multi-scale deep network that consists of
five convolutional layers and a bifurcated fully-connected sub-network. The
section from the input layer to the fifth convolutional layer is fixed and
directly lifted from a pre-trained network optimized over a large-scale object
classification task. This section of the network is applied to four different
scales of the image input. These four parallel and identical streams are then
attached to a bifurcated sub-network consisting of two independently-trained
branches. One branch learns to predict the contour likelihood (with a
classification objective) whereas the other branch is trained to learn the
fraction of human labelers agreeing about the contour presence at a given point
(with a regression criterion).
  We show that without any feature engineering our multi-scale deep learning
approach achieves state-of-the-art results in contour detection.
Gedas Bertasius Jianbo Shi Lorenzo Torresani