Efﬁciency of Blending: To illustrate that blending is indeed beneﬁcial we compare the negative
log-likelihood and the training error as a function of run-time in Fig. 7(b). The standard approach
is limited to 20 iterations of message passing to avoid time-consuming, repeated computation of a
stopping criterion involving both the approximated log-partition function and its dual. As show in
Fig. 7(b) blending learning and inference speeds up parameter estimation signiﬁcantly. For larger
graphical models, we expect the differences to be even more signiﬁcant.
4 D ISCUSSION & C ONCLUSION
Joint training of neural networks and graphical models: Neural Networks have been incorporated as unary potentials in graphical models. One of the earliest works by Bridle (1990) jointly optimizes a system consisting of multilayer perceptrons and hidden Markov models for speech recognition. For document processing systems, Bottou et al. (1997) propose Graph Transformer Networks
to jointly optimize sub-tasks, such as word segmentation and character recognition. Several works
(Collobert et al., 2011; Peng et al., 2009; Ma et al., 2012; Do & Artieres, 2010; Prabhavalkar &
Many problems in real-world applications involve predicting several random
variables which are statistically related. Markov random fields (MRFs) are a
great mathematical tool to encode such relationships. The goal of this paper is
to combine MRFs with deep learning algorithms to estimate complex
representations while taking into account the dependencies between the output
random variables. Towards this goal, we propose a training algorithm that is
able to learn structured models jointly with deep features that form the MRF
potentials. Our approach is efficient as it blends learning and inference and
makes use of GPU acceleration. We demonstrate the effectiveness of our
algorithm in the tasks of predicting words from noisy images, as well as
multi-class classification of Flickr photographs. We show that joint learning
of the deep features and the MRF parameters results in significant performance
gains.
Liang-Chieh Chen Alexander G. Schwing Alan L. Yuille Raquel Urtasun