Amharic language is an official language of the federal government of the Federal Democratic Republic of Ethiopia. Accordingly, there is a bulk of handwritten Amharic documents available in libraries, information centres, museums, and offices. Digitization of these documents enables to harness already available language technologies to local information needs and developments. Converting these documents will have a lot of advantages including (i) to preserve and transfer history of the country (ii) to save storage space (ii) proper handling of documents (iv) enhance retrieval of information through internet and other applications. Handwritten Amharic character recognition system becomes a challenging task due to inconsistency of a writer, variability in writing styles of different writers, relatively large number of characters of the script, high interclass similarity, structural complexity and degradation of documents due to different reasons. In order to recognize handwritten Amharic character a novel method based on deep neural networks is used which has recently shown exceptional performance in various pattern recognition and machine learning applications, but has not been endeavoured for Ethiopic script. The CNN model is trained and tested our database that contains 132,500 datasets of handwritten Amharic characters. Common machine learning methods usually apply a combination of feature extractor and trainable classifier. The use of CNN leads to significant improvements across different machinelearning classification algorithms. Our proposed CNN model is giving an accuracy of 91.83% on training data and 90.47% on validation data.
This paper considers the problem of character recognition applied to handwritten Amharic text. The authors collect a dataset of handwritten Amharic characters and apply a CNN model for character recognition. The task is contextualized within an extensive description of related work both in Amharic and other languages. The dataset is novel and would be of interest to the character recognition community, and I would encourage the authors to present it in its own right along with technical details about its collection. Unfortunately, due to methodological issues (detailed below), I do not think that the machine learning results in this paper are ready for publication at ICLR. The machine learning techniques applied are of limited novelty, however, the dataset is certainly a novel contribution.
Methodological issues:
The paper describes dividing the dataset in an 80-20 train-validation split. To the best of my understanding, there is no held out test set, and so we cannot know the performance of the model on unseen data. It appears that validation accuracy was taken into account when selecting hyperparameters, and so, validation accuracy also does not represent the model's performance on unseen data.
I would recommend that the authors introduce a test set into their dataset split, or designate the validation part of the dataset as a test part use cross validation for hyperparameter tuning.
The authors present the train and validation numbers on their dataset, but it is difficult to know the impact of the result without comparing to a baseline of some kind. It is challenging to compare to prior work since, according to the authors, prior work on Amharic character recognition has been focused on printed text.  However, a simple non-neural baseline would be illuminating.
Recommendations:
1. The authors should report results on an unseen test dataset, rather than train and validation sets.
2. The authors spend a lot of time motivating the use of deep learning in the introduction, and later on describing convolutional neural networks, ReLU, fully connected layers, etc. in great details. I believe that this space in the paper could be better utilized describing the things that are unique to the work presented.
3. It is really important to be able to see (a) the references and (b) the example characters, as it helps readers to situate the work and to understand the specific challenges addressed. I urge the authors to prioritize these technical details in submissions of future versions of this work.
4. Please consider separating out the description of related work into its own section. While it is useful to describe state of the art systems applied to related datasets, it is not necessary to go into great technical detail, especially if the methods applied are quite different from those attempted in this paper.
5. Based on the challenging nature of Amharic character recognition, it would be extremely helpful to see F1 numbers or a confusion matrix. While Figure 3 shows that different characters can be written similarly, it would be great to provide a quantitative measure of this phenomenon.
6. For a figure such as Figure 3, please provide an indication (e.g. the character number from 0-265, or a printed version of the character) of what the drawn character is supposed to look like, to help readers who cannot read Amharic script.
Questions about dataset
1. Please clarify what you mean by "the data collected are of two types". Does this refer to the train/validation split?
2. It would be great to know more about how many individuals were selected (and the choices made about their demographics) for writing the example characters, and whether there were any interesting variations observed in the dataset based on attributes highlighted in the paper (e.g. age range)
3. Please explain the meaning of 'Form A' and 'Form B' in Figure 2.
4. In the text, features such as a mark of palatalization are noted. Are these addressed in the data collection? Please give further details about how the dataset addresses the special features of the Amharic script.
5. At the top of page 6, it says that the data was labelled. Could you say more about the annotators (multiple?), and report inter annotator agreement? Or, did the writers produce each character based on a specific prompt, such that the labels are known to be correct?
6. Section 3 mentions data augmentation. Could you describe this in more detail?
7. Are the images in Figure 3 from your dataset? Please show some examples from the new dataset you have collected!