The Arabic language is a morphological rich language, posing many challenges for information extraction (IE) tasks, including Named Entity Recognition (NER), Part-of-Speech tagging (POS), Argument Role Labeling (ARL)  and Relation Extraction (RE). A few multilingual pre-trained models have been proposed and show good performance for Arabic, however, most experiment results are reported on language understanding tasks, such as natural language inference, question answering and sentiment analysis. Their performance on the IE tasks is less known, in particular, the cross-lingual transfer capability from English to Arabic. In this work, we pre-train a Gigaword-based bilingual language model (GigaBERT) to study these two distant languages as well as zero-short transfer learning on the information extraction tasks. Our GigaBERT model can outperform mBERT and XLM-R-base on NER, POS and ARL tasks, with regarding to the per-language and /or zero-transfer performance.We make our pre-trained models publicly available at URL to facilitate the research of this field.
Arabic is a morphological rich language, posing many challenges for information extraction (IE) tasks, including Named Entity Recognition (NER), Part-of-Speech tagging (POS), Argument Role Labeling (ARL) , and Relation Extraction (RE). A few multilingual pre-trained models have been proposed and show good performance for Arabic, however, most experiment results are reported on language understanding tasks, such as natural language inference, question answering and sentiment analysis. Their performance on the IE tasks is less known, in particular, the cross-lingual transfer capability from English to Arabic. In this work, we pre-train a Gigaword-based bilingual language model (GigaBERT) to study these two distant languages as well as zero-short transfer learning on various IE tasks. Our GigaBERT outperforms multilingual BERT and and monolingual AraBERT on these tasks, in both supervised and zero-shot learning settings.We have made our pre-trained models publicly available at URL 
R
The Arabic language
Arabic
clarity
clarity clarity clarity
A
,
fluency
fluency fluency fluency
R
the information extraction
various IE
clarity
clarity clarity clarity
R
model can outperform mBERT and XLM-R-base on NER, POS and ARL tasks, with regarding to the per-language and /or zero-transfer performance.We make
outperforms multilingual BERT and and monolingual AraBERT on these tasks, in both supervised and zero-shot learning settings.
clarity
meaning-changed clarity clarity
A
We have made
clarity
coherence clarity clarity
D
to facilitate the research of this field.
clarity
clarity clarity clarity