Show HN: I trained a neural network to learn Arabic morphology
Nice work! If I'm not mistaken, the root requires morphological disambiguation, which may change depending on the context/phrase in which the word is observed.<p>This is an active area of research in Morphologically Rich Languages (MRLs), since this problem also appears in other semitic languages like Hebrew, as well as Turkish. There's a nice body of work to learn from, both with and without neural nets. For example, this paper from 2017 (<a href="http://aclweb.org/anthology/D17-1073" rel="nofollow">http://aclweb.org/anthology/D17-1073</a>) uses a neural model for morphological disambiguation. You can see a nice comparison of tools in the recent 2018 Universal Dependencies Shared Task results: <a href="http://universaldependencies.org/conll18/results-lemmas.html" rel="nofollow">http://universaldependencies.org/conll18/results-lemmas.html</a> (look for ar_padt).<p>If you're looking for training data, the Arabic treebanks in <a href="http://universaldependencies.org" rel="nofollow">http://universaldependencies.org</a> could help. I think some of them contain surface tokens with lemmas. I'm quite sure they also have roots.<p>Also, you might want to take a look at the SIGMORPHON CONLL shared task (2017 <a href="https://sites.google.com/view/conll-sigmorphon2017/" rel="nofollow">https://sites.google.com/view/conll-sigmorphon2017/</a> and 2018 <a href="https://sigmorphon.github.io/sharedtasks/2018/" rel="nofollow">https://sigmorphon.github.io/sharedtasks/2018/</a>) on morphological reinflection, which IIRC is a similar task - taking an inflected form and reinflecting it with other morphological properties. They also have a nice data set to train on.