Time [s]Neg. Log−Likelihood
  
w/o blend
w blend
0 5000 100000200040006000800010000
Time [s]Training error
  
w/o blend
w blend
(a) (b)
Figure 7: (a) Correlation matrix ( i.e., pairwise potentials) learned on the Flickr dataset. (b) Blending
learning and inference speeds-up training signiﬁcantly.
Deep helps: We tested our models using one layer and two-layer perceptrons with both shortrange and long-range connections in the MRF. For the two-layer MLP, the number of hidden units
in the ﬁrst layer is ﬁxed to H1= 512 , and we varied the number of hidden units H2in the second
layer. As shown in Table 1 we observe that the deeper and the more structured the model, the better
the performance we achieve. As expected, performance also grows with the number of hidden units.
Efﬁciency: Using GPUs, it takes on average 0.064s per iteration for the 1st order Markov model
and 0.104s for the 2nd order Markov model. The time employed for training the one layer vs. the
Many problems in real-world applications involve predicting several random
variables which are statistically related. Markov random fields (MRFs) are a
great mathematical tool to encode such relationships. The goal of this paper is
to combine MRFs with deep learning algorithms to estimate complex
representations while taking into account the dependencies between the output
random variables. Towards this goal, we propose a training algorithm that is
able to learn structured models jointly with deep features that form the MRF
potentials. Our approach is efficient as it blends learning and inference and
makes use of GPU acceleration. We demonstrate the effectiveness of our
algorithm in the tasks of predicting words from noisy images, as well as
multi-class classification of Flickr photographs. We show that joint learning
of the deep features and the MRF parameters results in significant performance
gains.
Liang-Chieh Chen Alexander G. Schwing Alan L. Yuille Raquel Urtasun