(Collobert et al., 2011; Peng et al., 2009; Ma et al., 2012; Do & Artieres, 2010; Prabhavalkar &
Fosler-Lussier, 2010; Morris & Fosler-Lussier, 2008) have extended the linear unary potential in
MRFs to incorporate non-linearities. However, they assume that exact inference can be performed
either via a forward-backward pass within the graphical model or dynamic programming. In contrast, in this paper we present learning algorithms for general graphical models, where inference is
hard. Moreover, all the previous works (except Do & Artieres (2010)) do not consider max-margin
loss during training which is incorporated into our framework by choosing = 0. More recently, Li
& Zemel (2014) use a hinge loss to learn the unary term deﬁned as a neural net, but keep the pairwise
potentials ﬁxed ( i.e., no joint training). Domke (2013) considers non-linear structured prediction and
decomposes the learning problem into a subset of logistic regressors, which require the parameter
updates to be run till convergence before updating the messages. Tompson et al. (2014) also jointly
train convolutional neural networks and a graphical model for pose estimation. However, the MRF
Many problems in real-world applications involve predicting several random
variables which are statistically related. Markov random fields (MRFs) are a
great mathematical tool to encode such relationships. The goal of this paper is
to combine MRFs with deep learning algorithms to estimate complex
representations while taking into account the dependencies between the output
random variables. Towards this goal, we propose a training algorithm that is
able to learn structured models jointly with deep features that form the MRF
potentials. Our approach is efficient as it blends learning and inference and
makes use of GPU acceleration. We demonstrate the effectiveness of our
algorithm in the tasks of predicting words from noisy images, as well as
multi-class classification of Flickr photographs. We show that joint learning
of the deep features and the MRF parameters results in significant performance
gains.
Liang-Chieh Chen Alexander G. Schwing Alan L. Yuille Raquel Urtasun