1arXiv:1407.2538v3  [cs.LG]  27 Apr 2015
Accepted as a workshop contribution at ICLR 2015
while ignoring the dependencies between the variables of interest, e.g., independently learned segmentation and detection features (Hariharan et al., 2014) might be focusing on predicting the same
examples correctly. But when learned jointly they can improve their predictive power by exploiting
complementary information to ﬁx additional mistakes.
In this paper we extend deep learning algorithms to learn complex representations taking into account the dependencies between the output random variables. Towards this goal, we propose a
learning algorithm that is able to learn structured models with arbitrary graphs jointly with deep
features that form Markov random ﬁeld (MRF) potentials. Our approach is efﬁcient as it blends
learning and inference resulting in a single loop algorithm which makes use of GPU acceleration.
We demonstrate the effectiveness of our method in the tasks of predicting words from noisy images,
and multi-class classiﬁcation of Flickr photographs. We show that joint learning of deep features
and MRF parameters results in big performance gains.
2 L EARNING DEEPSTRUCTURED MODELS
Many problems in real-world applications involve predicting several random
variables which are statistically related. Markov random fields (MRFs) are a
great mathematical tool to encode such relationships. The goal of this paper is
to combine MRFs with deep learning algorithms to estimate complex
representations while taking into account the dependencies between the output
random variables. Towards this goal, we propose a training algorithm that is
able to learn structured models jointly with deep features that form the MRF
potentials. Our approach is efficient as it blends learning and inference and
makes use of GPU acceleration. We demonstrate the effectiveness of our
algorithm in the tasks of predicting words from noisy images, as well as
multi-class classification of Flickr photographs. We show that joint learning
of the deep features and the MRF parameters results in significant performance
gains.
Liang-Chieh Chen Alexander G. Schwing Alan L. Yuille Raquel Urtasun