is composed out of two subsequent stacks of convolution, rectiﬁed linear (ReLU), pooling and local
response normalization units. Those are followed by three convolution–ReLU function pairs. Afterwards pooling is applied before two fully-connected–ReLU–dropout combinations are employed to
yield the input into a fully connected layer which ﬁnally computes the unary potentials. We employ
pairwise potentials similar to Eq. (7) which now fully model the correlations between any pair of
8
Accepted as a workshop contribution at ICLR 2015
output variables. This amounts to a total of 57;182;408parameters arising from the convolutional
units, fully connected units and corresponding biases as well as the pairwise weights.
We use a momentum based sub-gradient method for training with a mini-batch size of 300, a step
size of 0:0001 , a momentum of 0:95and set= 1 andcr= 18r. We initialized the deep-net
parameters using a model pre-trained on ImageNet (Deng et al., 2009). Our error metric is the
classiﬁcation error ( i.e., Hamming loss).
Joint training helps: The mean error for unary only potentials (‘Unary only’), piecewise training
Many problems in real-world applications involve predicting several random
variables which are statistically related. Markov random fields (MRFs) are a
great mathematical tool to encode such relationships. The goal of this paper is
to combine MRFs with deep learning algorithms to estimate complex
representations while taking into account the dependencies between the output
random variables. Towards this goal, we propose a training algorithm that is
able to learn structured models jointly with deep features that form the MRF
potentials. Our approach is efficient as it blends learning and inference and
makes use of GPU acceleration. We demonstrate the effectiveness of our
algorithm in the tasks of predicting words from noisy images, as well as
multi-class classification of Flickr photographs. We show that joint learning
of the deep features and the MRF parameters results in significant performance
gains.
Liang-Chieh Chen Alexander G. Schwing Alan L. Yuille Raquel Urtasun