3.1 W ORD RECOGNITION : W ORD50
Our ﬁrst task consists of word recognition from noisy images. Towards this goal, we created a
challenging dataset by randomly selecting 50 words, each consisting of ﬁve characters. We then
generated writing variations of each word as follows: we took the lower case characters from the
Chars74K dataset (de Campos et al., 2009), and inserted them in random background image patches
(similar to Larochelle et al. (2007)) by alpha matting, i.e., characters have transparency. To increase
the difﬁculty, we perturbed each character image of size 2828by scaling, rotation and translation.
As shown in Fig. 3 the task is very challenging, some characters are fairly difﬁcult to recognize even
for humans. We denote the resulting dataset ‘ Word50 .’ The training, validation and test sets have
10;000,2;000and2;000variations of words respectively.
We experimented with graphical models composed of unary and pairwise regions deﬁned over ﬁve
Many problems in real-world applications involve predicting several random
variables which are statistically related. Markov random fields (MRFs) are a
great mathematical tool to encode such relationships. The goal of this paper is
to combine MRFs with deep learning algorithms to estimate complex
representations while taking into account the dependencies between the output
random variables. Towards this goal, we propose a training algorithm that is
able to learn structured models jointly with deep features that form the MRF
potentials. Our approach is efficient as it blends learning and inference and
makes use of GPU acceleration. We demonstrate the effectiveness of our
algorithm in the tasks of predicting words from noisy images, as well as
multi-class classification of Flickr photographs. We show that joint learning
of the deep features and the MRF parameters results in significant performance
gains.
Liang-Chieh Chen Alexander G. Schwing Alan L. Yuille Raquel Urtasun