2014).
Deep neural networks can, however, be even more powerful when combined with graphical models
in order to capture the statistical dependencies between the variables of interest. For example, Deng
et al. (2014) exploit mutual exclusion, overlapping and subsumption properties of class labels in order to better predict in large scale classiﬁcation tasks. In pose estimation, more accurate predictions
can be obtained when encoding the spatial relationships between joint locations (Tompson et al.,
2014).
It is, however, an open problem how to develop scalable deep learning algorithms that can learn
higher-order knowledge taking into account the output variable’s dependencies. Existing approaches
often rely on a two-step process (Nowozin et al., 2011; Xu et al., 2014) where a non-linear classiﬁer
that employs deep features is trained ﬁrst, and its output is used to generate potentials for the structured predictor. This piece-wise training is, however, suboptimal as the deep features are learned
The ﬁrst two authors contributed equally to this work.
1arXiv:1407.2538v3  [cs.LG]  27 Apr 2015
Accepted as a workshop contribution at ICLR 2015
Many problems in real-world applications involve predicting several random
variables which are statistically related. Markov random fields (MRFs) are a
great mathematical tool to encode such relationships. The goal of this paper is
to combine MRFs with deep learning algorithms to estimate complex
representations while taking into account the dependencies between the output
random variables. Towards this goal, we propose a training algorithm that is
able to learn structured models jointly with deep features that form the MRF
potentials. Our approach is efficient as it blends learning and inference and
makes use of GPU acceleration. We demonstrate the effectiveness of our
algorithm in the tasks of predicting words from noisy images, as well as
multi-class classification of Flickr photographs. We show that joint learning
of the deep features and the MRF parameters results in significant performance
gains.
Liang-Chieh Chen Alexander G. Schwing Alan L. Yuille Raquel Urtasun