Researches shows that attempts done to apply existing dependency parser on morphological rich languages including Amharic shows a poor performance. In this study, a dependency parser for Amharic language is implemented using arc-eager transition system and LSTM network. The study introduced another way of building labeled dependency structure by using a separate network model to predict dependency relation. This helps the number of classes to decrease from 2n+2 into n, where n is the number of relationship types in the language and increases the number of examples for each class in the data set. Evaluation of the parser model results 91.54 and 81.4 unlabeled and labeled attachment score respectively. The major challenge in this study was the decrease of the accuracy of labeled attachment score. This is mainly due to the size and quality of the tree-bank available for Amharic language. Improving the tree-bank by increasing the size and by adding morphological information can make the performance of parser better.
The paper claims to build a transition-based dependency parser for Amharic. To produce a parse tree, the paper takes a two-stage approach, first producing the structure of the tree, followed by annotating each individual arcs on the tree. The paper also explores attention and finds it useful for improving the performance.
I am giving a score 1, because the paper lacks precision and the contribution is not significant.
The major problem of the paper is precision. There is a lack of formal descriptions of the parser, e.g., the representation of the input and output, what each individual block in the figures are doing, how the arc-eager transition parser works. The text description is certainly helpful, but without formal descriptions, it is impossible to reproduce the results in the paper. Besides, without setting up a precise language/notation, the paper is not easy to follow.
In terms of contribution, the paper begins with the motivation that Amharic has certain language characteristics that are absent in languages, such as English, where most research effort has been spent. However, the paper still uses a parser for English, and does not utilize the special features of Amharic. Generic architectural modification, such as adding attention, has been explored, but this has little contribution on its own and has little to do with Amharic.
In addition to the above, other minor weaknesses of the paper includes typos, poor experimental practices, missing citations and related work, and other wording issues.
In particular, choosing the split of the data set that achieves the best result, as quoted below, is concerning.
"We experimented on 60/40, 70/30 and 80/20 train-test splitting ratios ... Therefore, we select 70/30 train-test splitting ratio throughout the experiments."