Gated Recurrent Convolution Neural Network for OCR
Optical Character Recognition (OCR) aims to recognize text in natural images. Inspired by a recently proposed model for general image classification, Recurrent Convolution Neural Network (RCNN), we propose a new architecture named Gated RCNN (GRCNN) for solving this problem. Its critical component, Gated Recurrent Convolution Layer (GRCL), is constructed by adding a gate to the Recurrent Convolution Layer (RCL), the critical component of RCNN. The gate controls the context modulation in RCL and balances the feed-forward information and the recurrent information. In addition, an efficient Bidirectional Long ShortTerm Memory (BLSTM) is built for sequence modeling. The GRCNN is combined with BLSTM to recognize text in natural images. The entire GRCNN-BLSTM model can be trained end-to-end. Experiments show that the proposed model outperforms existing methods on several benchmark datasets including the IIIT-5K, Street View Text (SVT) and ICDAR.
: Illustration of using RCL with T = 2 for OCR.
A Recurrent Convolution Neural Network (RCNN) is proposed for general image classification [22], which simulates an anatomical fact that recurrent connections are ubiquitously existent in the neocortex. It is believed that the recurrent synapses that exist in the neocortex play an important role in context modulation during visual recognition. A feed-forward model can only capture the context in higher layers where units have larger receptive fields, but this information cannot modulate the units in lower layers which is responsible for recognizing smaller objects. Hence, using recurrent connections within a convolutional layer (called Recurrent Convolution Layer or RCL) can bring context information to all units in this layer. This implements a nonclassical receptive field [12,18] of a biological neuron as the effective receptive field is larger than that determined by feedforward connections. However, with increasing iterations, the size of the effective receptive field will increase unboundedly, which contradicts the biological fact. One needs a mechanism to constrain the growth of the effective receptive field.
In addition, from the viewpoint of performance enhancing, one also needs to control the context modulation of neurons in RCNN. For example, in Figure 1, it is seen that for recognizing a character, not all of the context are useful. When the network recognizes the character "h", the recurrent kernel which covers the other parts of the character is beneficial. However, when the recurrent kernel is enlarged to the parts of other characters, such as "p", the context carried by the kernel is unnecessary. Therefore, we need to weaken the signal that comes from unrelated context and combine the feed-forward information with recurrent information in a flexible way. To achieve the above goals, we introduce a gate to control the context modulation in each RCL layer, which leads to the Gated Recurrent Convolution Neural Network (GRCNN).
In addition, the recurrent neural network is adopted in the recognition of words in natural images since it is good at sequence modeling. In this work, we choose the Long Short Term Memory (LSTM) as the top layer of the proposed model, which is trained in an end-to-end fashion.
Authors added gating mechanism in Recurrent Convolutional NN that allow controlling the amount of context / recursion. In other words, it is similar to improvement from RNN to GRU but for Recurrent CNNs. Authors benchmark their model on text reading task. 
Positives aspects:
- authors improved upon the state of the art results on very well research benchmark
- the proposed model combines few proved ideas into a single model and made it work. 
Areas to improve:
- Lines 195-196: you state that choose 10 best models per architecture that perform well on each dataset and state the results as the median of them. It is ok when you compare different iterations of you model to make sure you found good hyperparameters. But this cannot be at the final table with the comparison to other works. You should clearly state what is your pool of models (is it coming from different hyperparameters? what parameters and what values did you choose). How many models did you try during this search? You should provide single hyper parameter + model setup with the results that works the best across all the problems in the final table. Additionally, you may provide variance of the results with the clear explanation of hyper parameter search you performed. 
- did authors try to use attention not bidirectional RNN + CTC loss? In few recent works (https://arxiv.org/pdf/1603.03101.pdf, https://arxiv.org/pdf/1704.03549.pdf, https://arxiv.org/pdf/1603.03915.pdf). This should still allow for improving the results, or if didn't work, this is worth stating. 
I vote for top 50% of papers as improving upon the previous state of the art results on such a benchmark is a difficult job. Authors proposed a new model for extracting image features that is potentially beneficial to other visual tasks. This is conditional on fixing the results number (to present numbers from a single model, not median) and open-sourcing the code and models to the community for reproducibility.