the linear pairwise function in Eq. (7) by a one-layer MLP, while keeping the other settings identical.
For this experiment we utilize a 1st order Markov model. As shown in Fig. 5, our model attains best
performance when using a non-linear pairwise function. We found 16 to 64 hidden units for the
non-linear pairwise function to be sufﬁcient for modeling the bi-gram combinations in this dataset.
In this case the largest model has 974,846 weights and training takes on average 0.068s per iteration.
3.2 I MAGE CLASSIFICATION : FLICKR
We next evaluate the importance of blending learning and inference. Towards this goal, we make
use of the Flickr dataset, which consists of 10;000training and 10;000test images from Flickr. The
task is to predict which of 38 possible tags should be assigned to each image. Fig. 6 shows some
examples. The graphical model has 38 binary random variables, each denoting the presence/absence
of a particular tag. We deﬁne the non-linear unaries fr(x;yi;wu)using the 8-layer deep-net architecture from Krizhevsky et al. (2013) followed by a 76-dimensional top layer. Hence the function
Many problems in real-world applications involve predicting several random
variables which are statistically related. Markov random fields (MRFs) are a
great mathematical tool to encode such relationships. The goal of this paper is
to combine MRFs with deep learning algorithms to estimate complex
representations while taking into account the dependencies between the output
random variables. Towards this goal, we propose a training algorithm that is
able to learn structured models jointly with deep features that form the MRF
potentials. Our approach is efficient as it blends learning and inference and
makes use of GPU acceleration. We demonstrate the effectiveness of our
algorithm in the tasks of predicting words from noisy images, as well as
multi-class classification of Flickr photographs. We show that joint learning
of the deep features and the MRF parameters results in significant performance
gains.
Liang-Chieh Chen Alexander G. Schwing Alan L. Yuille Raquel Urtasun