allowing us to optimise them via backpropagation, and also with respect to the Q(X)
values enabling us to optimise previous layers in the network.
Convergence of parallel mean ﬁeld updates Mean ﬁeld with parallel updates, as
proposed in [17] for speed, does not have any convergence guarantees in the general case.
However, we usually empirically observed convergence with higher order potentials,
without damping the mean ﬁeld update as described in [7, 42]. This may be explained by
the fact that the unaries from the initial pixelwise-prediction part of our network provide
a good initialisation. In cases where the mean ﬁeld energy did not converge, we still
empirically observed good ﬁnal segmentations.
6 Experiments
We evaluate our new CRF formulation on two different datasets using the CRF-RNN
network [10] as the main baseline, since we are essentially enriching the CRF model
of [10]. We then present ablation studies on our models.
6.1 Experimental set-up and results
Our deep network consists of two conceptually different, but jointly trained stages. The
ﬁrst, “unary” part of our network is formed by the FCN-8s architecture [3]. It is initialised
We address the problem of semantic segmentation using deep learning. Most
segmentation systems include a Conditional Random Field (CRF) to produce a
structured output that is consistent with the image's visual features. Recent
deep learning approaches have incorporated CRFs into Convolutional Neural
Networks (CNNs), with some even training the CRF end-to-end with the rest of
the network. However, these approaches have not employed higher order
potentials, which have previously been shown to significantly improve
segmentation performance. In this paper, we demonstrate that two types of
higher order potential, based on object detections and superpixels, can be
included in a CRF embedded within a deep network. We design these higher order
potentials to allow inference with the differentiable mean field algorithm. As
a result, all the parameters of our richer CRF model can be learned end-to-end
with our pixelwise CNN classifier. We achieve state-of-the-art segmentation
performance on the PASCAL VOC benchmark with these trainable higher order
potentials.
Anurag Arnab Sadeep Jayasumana Shuai Zheng Philip Torr