[3] by convolutional feature masking and segment pursuit with the
VGG net. O2Pis the second order pooling method [1] as reported
in the errata of [26]. The 59 class task includes the 59 most frequent classes while the 33 class task consists of an easier subset
identiﬁed by [26].
59 classpixel
acc.mean
acc.mean
IUf.w.
IU
O2P - - 18.1 CFM - - 31.5 FCN-32s 63.8 42.7 31.8 48.3
FCN-16s 65.7 46.2 34.8 50.7
FCN-8s 65.9 46.5 35.1 51.0
33 class
O2P - - 29.2 CFM - - 46.1 FCN-32s 69.8 65.1 50.4 54.9
FCN-16s 71.8 68.0 53.4 57.5
FCN-8s 71.8 67.6 53.5 57.7
v2Add Appendix A giving upper bounds on mean IU and
Appendix B with PASCAL-Context results. Correct PASCAL validation numbers (previously, some val images were
included in train), SIFT Flow mean IU (which used an inappropriately strict metric), and an error in the frequency
Convolutional networks are powerful visual models that yield hierarchies of
features. We show that convolutional networks by themselves, trained
end-to-end, pixels-to-pixels, exceed the state-of-the-art in semantic
segmentation. Our key insight is to build "fully convolutional" networks that
take input of arbitrary size and produce correspondingly-sized output with
efficient inference and learning. We define and detail the space of fully
convolutional networks, explain their application to spatially dense prediction
tasks, and draw connections to prior models. We adapt contemporary
classification networks (AlexNet, the VGG net, and GoogLeNet) into fully
convolutional networks and transfer their learned representations by
fine-tuning to the segmentation task. We then define a novel architecture that
combines semantic information from a deep, coarse layer with appearance
information from a shallow, fine layer to produce accurate and detailed
segmentations. Our fully convolutional network achieves state-of-the-art
segmentation of PASCAL VOC (20% relative improvement to 62.2% mean IU on 2012),
NYUDv2, and SIFT Flow, while inference takes one third of a second for a
typical image.
Jonathan Long Evan Shelhamer Trevor Darrell