RGB and depth channels at the input. HHA is the depth embedding of [14] as horizontal disparity, height above ground, and
the angle of the local surface normal with the inferred gravity
direction. RGB-HHA is the jointly trained late fusion model
that sums RGB and HHA predictions.
pixel
acc.mean
acc.mean
IUf.w.
IU
Gupta et al. [14] 60.3 - 28.6 47.0
FCN-32s RGB 60.0 42.2 29.2 43.9
FCN-32s RGBD 61.5 42.4 30.5 45.5
FCN-32s HHA 57.1 35.2 24.2 40.4
FCN-32s RGB-HHA 64.3 44.9 32.8 48.0
FCN-16s RGB-HHA 65.4 46.1 34.0 49.5
Microsoft Kinect. It has 1449 RGB-D images, with pixelwise labels that have been coalesced into a 40 class semantic segmentation task by Gupta et al. [13]. We report results
on the standard split of 795 training images and 654 testing
images. (Note: all model selection is performed on PASCAL 2011 val.) Table 4 gives the performance of our model
Convolutional networks are powerful visual models that yield hierarchies of
features. We show that convolutional networks by themselves, trained
end-to-end, pixels-to-pixels, exceed the state-of-the-art in semantic
segmentation. Our key insight is to build "fully convolutional" networks that
take input of arbitrary size and produce correspondingly-sized output with
efficient inference and learning. We define and detail the space of fully
convolutional networks, explain their application to spatially dense prediction
tasks, and draw connections to prior models. We adapt contemporary
classification networks (AlexNet, the VGG net, and GoogLeNet) into fully
convolutional networks and transfer their learned representations by
fine-tuning to the segmentation task. We then define a novel architecture that
combines semantic information from a deep, coarse layer with appearance
information from a shallow, fine layer to produce accurate and detailed
segmentations. Our fully convolutional network achieves state-of-the-art
segmentation of PASCAL VOC (20% relative improvement to 62.2% mean IU on 2012),
NYUDv2, and SIFT Flow, while inference takes one third of a second for a
typical image.
Jonathan Long Evan Shelhamer Trevor Darrell