Cortical-Inspired Open-Bigram Representation for Handwritten Word Recognition
Recent research in the cognitive process of reading hypothesized that we do not read words by sequentially recognizing letters, but rather by identifing open-bigrams, i.e. couple of letters that are not necessarily next to each other. In this paper, we evaluate an handwritten word recognition method based on original open-bigrams representation. We trained Long Short-Term Memory Recurrent Neural Networks (LSTM-RNNs) to predict open-bigrams rather than characters, and we show that such models are able to learn the long-range, complicated and intertwined dependencies in the input signal, necessary to the prediction. For decoding, we decomposed each word of a large vocabulary into the set of constituent bigrams, and apply a simple cosine similarity measure between this representation and the bagged RNN prediction to retrieve the vocabulary word. We compare this method to standard word recognition techniques based on sequential character recognition. Experiments are carried out on two public databases of handwritten words (Rimes and IAM), an the results with our bigram decoder are comparable to more conventional decoding methods based on sequences of letters.
This submission investigates the usability of cortical-inspired distant bigram representations for handwritten word recognition. Instead of generating neural network based posterior features for character (optionally in local context), sets posterior for character bigrams of different length are used to represent words. The aim here is to investigate the viability of this approach and to compare to the standard approach. Overall, the submission is well written, although information is missing w.r.t. to the comparison between the proposed approach and the standard approach, see below. It would be desirable to see the model complexity of all the different models used here, i.e. the number of parameters used. Language models are not used here. Since the different models utilize different levels of context, language models can be expected to have a different effect on the different approaches. Therefore I suggest to include the use of language models into the evaluation. For your comparative experiments you use only 70% of the data by choosing longer words only. On the other hand, it is well known that the shorter words are more prone to result in misrecognitions. The question remains, if this choice is advantageous for one of the tasks, or not - corresponding quantitative results should be provided to be able to better evaluate the effect of using this constrained corpus. Without clarification of this I would not readily agree that the error rates are competitive or better than the standard approach, as stated at the end of Sec. 5. I do see the motivation for introducing open-bigrams in an unordered way due to the corresponding evidence from cognitive research. However, decision theoretically I wonder, why the order should be given up, if the underlying sequential classification problem clearly is of a monotonous nature. It would be interesting to see an experiment, where only the use of the order is varied, to differentiate the effect of the order from the effect of other aspects of the approach. End of page 1: *whole language method* - please explain what is meant by this. Page 6: define your notation for rnn_d(x,t). The number of target for the RNNs modeling order 0 (unigrams effectively) and the RNNs modeling order 1 and larger are very much different. Therefore the precision and recall numbers in Table 2 do not seem to be readily comparable between order 0 and orders >=1. At least, the column for order 0 should be visually separated to highlight this. Minor comments: a spell check is recommended p. 2: state-of-art -> state-of-the-art p. 2: predict character sequence -> predict a character sequence p. 3, top: Their approach include -> Their approach includes p. 3, top: an handwritten -> a handwritten p. 3, bottom: consituent -> constituent p. 4, top: in classical approach -> in the classical approach p. 4, top: transformed in a vector -> transformed into a vector p. 5: were build -> were built References: first authors name written wrongly: Thodore Bluche -> Theodore Bluche
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Cortical-Inspired Open-Bigram Representation for Handwritten Word Recognition
Recent research in the cognitive process of reading hypothesized that we do not read words by sequentially recognizing letters, but rather by identifing open-bigrams, i.e. couple of letters that are not necessarily next to each other. In this paper, we evaluate an handwritten word recognition method based on original open-bigrams representation. We trained Long Short-Term Memory Recurrent Neural Networks (LSTM-RNNs) to predict open-bigrams rather than characters, and we show that such models are able to learn the long-range, complicated and intertwined dependencies in the input signal, necessary to the prediction. For decoding, we decomposed each word of a large vocabulary into the set of constituent bigrams, and apply a simple cosine similarity measure between this representation and the bagged RNN prediction to retrieve the vocabulary word. We compare this method to standard word recognition techniques based on sequential character recognition. Experiments are carried out on two public databases of handwritten words (Rimes and IAM), an the results with our bigram decoder are comparable to more conventional decoding methods based on sequences of letters.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This submission investigates the usability of cortical-inspired distant bigram representations for handwritten word recognition. Instead of generating neural network based posterior features for character (optionally in local context), sets posterior for character bigrams of different length are used to represent words. The aim here is to investigate the viability of this approach and to compare to the standard approach. Overall, the submission is well written, although information is missing w.r.t. to the comparison between the proposed approach and the standard approach, see below. It would be desirable to see the model complexity of all the different models used here, i.e. the number of parameters used. Language models are not used here. Since the different models utilize different levels of context, language models can be expected to have a different effect on the different approaches. Therefore I suggest to include the use of language models into the evaluation. For your comparative experiments you use only 70% of the data by choosing longer words only. On the other hand, it is well known that the shorter words are more prone to result in misrecognitions. The question remains, if this choice is advantageous for one of the tasks, or not - corresponding quantitative results should be provided to be able to better evaluate the effect of using this constrained corpus. Without clarification of this I would not readily agree that the error rates are competitive or better than the standard approach, as stated at the end of Sec. 5. I do see the motivation for introducing open-bigrams in an unordered way due to the corresponding evidence from cognitive research. However, decision theoretically I wonder, why the order should be given up, if the underlying sequential classification problem clearly is of a monotonous nature. It would be interesting to see an experiment, where only the use of the order is varied, to differentiate the effect of the order from the effect of other aspects of the approach. End of page 1: *whole language method* - please explain what is meant by this. Page 6: define your notation for rnn_d(x,t). The number of target for the RNNs modeling order 0 (unigrams effectively) and the RNNs modeling order 1 and larger are very much different. Therefore the precision and recall numbers in Table 2 do not seem to be readily comparable between order 0 and orders >=1. At least, the column for order 0 should be visually separated to highlight this. Minor comments: a spell check is recommended p. 2: state-of-art -> state-of-the-art p. 2: predict character sequence -> predict a character sequence p. 3, top: Their approach include -> Their approach includes p. 3, top: an handwritten -> a handwritten p. 3, bottom: consituent -> constituent p. 4, top: in classical approach -> in the classical approach p. 4, top: transformed in a vector -> transformed into a vector p. 5: were build -> were built References: first authors name written wrongly: Thodore Bluche -> Theodore Bluche
assistant
This submission investigates the usability of cortical-inspired distant bigram representations for handwritten word recognition. Instead of generating neural network based posterior features for character (optionally in local context), sets posterior for character bigrams of different length are used to represent words. The aim here is to investigate the viability of this approach and to compare to the standard approach. Overall, the submission is well written, although information is missing w.r.t. to the comparison between the proposed approach and the standard approach, see below. It would be desirable to see the model complexity of all the different models used here, i.e. the number of parameters used. Language models are not used here. Since the different models utilize different levels of context, language models can be expected to have a different effect on the different approaches. Therefore I suggest to include the use of language models into the evaluation. For your comparative experiments you use only 70% of the data by choosing longer words only. On the other hand, it is well known that the shorter words are more prone to result in misrecognitions. The question remains, if this choice is advantageous for one of the tasks, or not - corresponding quantitative results should be provided to be able to better evaluate the effect of using this constrained corpus. Without clarification of this I would not readily agree that the error rates are competitive or better than the standard approach, as stated at the end of Sec. 5. I do see the motivation for introducing open-bigrams in an unordered way due to the corresponding evidence from cognitive research. However, decision theoretically I wonder, why the order should be given up, if the underlying sequential classification problem clearly is of a monotonous nature. It would be interesting to see an experiment, where only the use of the order is varied, to differentiate the effect of the order from the effect of other aspects of the approach. End of page 1: *whole language method* - please explain what is meant by this. Page 6: define your notation for rnn_d(x,t). The number of target for the RNNs modeling order 0 (unigrams effectively) and the RNNs modeling order 1 and larger are very much different. Therefore the precision and recall numbers in Table 2 do not seem to be readily comparable between order 0 and orders >=1. At least, the column for order 0 should be visually separated to highlight this. Minor comments: a spell check is recommended p. 2: state-of-art -> state-of-the-art p. 2: predict character sequence -> predict a character sequence p. 3, top: Their approach include -> Their approach includes p. 3, top: an handwritten -> a handwritten p. 3, bottom: consituent -> constituent p. 4, top: in classical approach -> in the classical approach p. 4, top: transformed in a vector -> transformed into a vector p. 5: were build -> were built References: first authors name written wrongly: Thodore Bluche -> Theodore Bluche