and MRF parameters results in big performance gains.
2 L EARNING DEEPSTRUCTURED MODELS
In this section we investigate how to learn ‘deep features’ that take into account the dependencies between the output variables. Let y2Y be the set of random variables y= (y1;:::;yN)
that we are interested in predicting. In this work we assume the space of valid conﬁgurations to
be a product space, i.e.,Y=QN
i=1Yi, and the domain of each individual variable yito be discrete, i.e.,Yi=f1;:::;jYijg. Given input data x2X and parameters w2RAof the function
F(x;y;w) :XY RA!R, inference amounts to ﬁnding the highest scoring conﬁguration
y= arg max yF(x;y;w):Note that ifFis a deep network and there are no connections between
the output variables to be predicted, inference corresponds to a forward pass to evaluate the function,
followed by independently ﬁnding the largest response for each variable. This can be interpreted as
inference in a graphical model with only unary potentials. However, for arbitrary graphical models
Many problems in real-world applications involve predicting several random
variables which are statistically related. Markov random fields (MRFs) are a
great mathematical tool to encode such relationships. The goal of this paper is
to combine MRFs with deep learning algorithms to estimate complex
representations while taking into account the dependencies between the output
random variables. Towards this goal, we propose a training algorithm that is
able to learn structured models jointly with deep features that form the MRF
potentials. Our approach is efficient as it blends learning and inference and
makes use of GPU acceleration. We demonstrate the effectiveness of our
algorithm in the tasks of predicting words from noisy images, as well as
multi-class classification of Flickr photographs. We show that joint learning
of the deep features and the MRF parameters results in significant performance
gains.
Liang-Chieh Chen Alexander G. Schwing Alan L. Yuille Raquel Urtasun