Iteration  
JointTrain
PwTrain
PreTrainJoint
Unary weights distance-1 edges distance-2 edges Neg. Log-Likelihood
Figure 4: (left) Subset of the learned unary weights. Pairwise weights (middle two panels), the
darker, the larger the weight. (right) Negative log-likelihood for different learning approaches.
3 E XPERIMENTAL EVALUATION
We demonstrate the performance of our model on two tasks: word recognition and image classiﬁcation. We investigate four strategies to learn the model parameters. ‘ Unary only ’ denotes training
only unary classiﬁers while ignoring the structure of the graphical model, i.e., pairwise weights are
equal to 0. ‘JointTrain ’ initializes all weights at random and trains them jointly. ‘ PwTrain ’ uses
piecewise training by ﬁrst training the unary potentials and then keeping them ﬁxed when learning the pairwise potentials. ‘ PreTrainJoint ’ pre-trains the unaries but jointly optimizes pairwise
weights as well as unary weights in a second step.
3.1 W ORD RECOGNITION : W ORD50
Our ﬁrst task consists of word recognition from noisy images. Towards this goal, we created a
Many problems in real-world applications involve predicting several random
variables which are statistically related. Markov random fields (MRFs) are a
great mathematical tool to encode such relationships. The goal of this paper is
to combine MRFs with deep learning algorithms to estimate complex
representations while taking into account the dependencies between the output
random variables. Towards this goal, we propose a training algorithm that is
able to learn structured models jointly with deep features that form the MRF
potentials. Our approach is efficient as it blends learning and inference and
makes use of GPU acceleration. We demonstrate the effectiveness of our
algorithm in the tasks of predicting words from noisy images, as well as
multi-class classification of Flickr photographs. We show that joint learning
of the deep features and the MRF parameters results in significant performance
gains.
Liang-Chieh Chen Alexander G. Schwing Alan L. Yuille Raquel Urtasun