categories. While such features have been optimized for the
task of object class recognition, they have been shown to
be highly effective for other image-analysis tasks, includ-ing object detection [10], attribute prediction [34], and image style recognition [15]. The network was trained on 1:2
million images and it includes more than 60million parameters. Its architecture consists of 5convolutional layers and
3fully connected layers. As we intend to use the KNet as
a feature extractor for boundary detection, we utilize only
the ﬁrst 5convolutional layers, which preserve explicit location information before the “spatial scrambling” of the
fully connection layers (note that a spatially variant representation is crucially necessary to predict the presence of
contours at individual pixels). The ﬁrst two KNet convolutional layers learn low-level information. As we move
into the deeper layers, however, we observe that the network
learns higher-level object information. The second convolutional layer seems to encode coherent edge structures. The
third convolutional layer ﬁres at locations corresponding to
prototypical object shapes. The fourth layer appears to generate high responses for full shapes of the object, whereas
the ﬁfth layer ﬁres on the speciﬁc object parts (See Fig. 3).
Contour detection has been a fundamental component in many image segmentation
and object detection systems. Most previous work utilizes low-level features
such as texture or saliency to detect contours and then use them as cues for a
higher-level task such as object detection. However, we claim that recognizing
objects and predicting contours are two mutually related tasks. Contrary to
traditional approaches, we show that we can invert the commonly established
pipeline: instead of detecting contours with low-level cues for a higher-level
recognition task, we exploit object-related features as high-level cues for
contour detection.
  We achieve this goal by means of a multi-scale deep network that consists of
five convolutional layers and a bifurcated fully-connected sub-network. The
section from the input layer to the fifth convolutional layer is fixed and
directly lifted from a pre-trained network optimized over a large-scale object
classification task. This section of the network is applied to four different
scales of the image input. These four parallel and identical streams are then
attached to a bifurcated sub-network consisting of two independently-trained
branches. One branch learns to predict the contour likelihood (with a
classification objective) whereas the other branch is trained to learn the
fraction of human labelers agreeing about the contour presence at a given point
(with a regression criterion).
  We show that without any feature engineering our multi-scale deep learning
approach achieves state-of-the-art results in contour detection.
Gedas Bertasius Jianbo Shi Lorenzo Torresani