readme: ---
  language: 
  - ar
  - en
  license: apache-2.0
  datasets:
  - 4Dialects
  - MADAR
  - CSCS
  thumbnail: https://www.informatik.hu-berlin.de/en/forschung-en/gebiete/ml-en/resolveuid/a6f82e0d7fa446a59c902cac4cafa9cb/@@images/image/preview
  tags:
  - flair
  - token-classification
  - sequence-tagger-model
  - Dialectal Arabic
  - Code-Switching
  - Code-Mixing
  metrics:
  - f1
  widget:
  - text: "طلعوا جماعة الممانعة بالسياسة ما بيعرفوا ولا بالصحة بيعرفوا ولا حتى بالدين"
  - text: "أعلم أن هذا يبدو غير عادل ، لكن لا يمكن أن يكون هناك ظلم"
  - text: "أنا عارف أن الموضوع ده شكله مش عادل ، بس لا يمكن أن يكون فيه ظلم"
  ---
  
  
  # Arabic Flair + fastText Part-of-Speech tagging Model (Egyptian and Levant)
  Pretrained Part-of-Speech tagging model built on a joint corpus written in Egyptian and Levantine (Jordanian, Lebanese, Palestinian, Syrian) dialects with code-switching of Egyptian Arabic and English. The model is trained using [Flair](https://aclanthology.org/C18-1139/) (forward+backward)and [fastText](https://fasttext.cc) embeddings.
  
  
  
  # Pretraining Corpora:
  This sequence labeling model was pretrained on three corpora jointly:
  1. [4 Dialects](https://huggingface.co/datasets/viewer/?dataset=arabic_pos_dialect)
  A Dialectal Arabic Datasets containing four dialects of Arabic, Egyptian (EGY), Levantine (LEV), Gulf (GLF), and Maghrebi (MGR). Each dataset consists of a set of 350 manually segmented and PoS tagged tweets.
  2. [UD South Levantine Arabic MADAR](https://universaldependencies.org/treebanks/ajp_madar/index.html)
  A Dataset with 100 manually-annotated sentences taken from the [MADAR](https://camel.abudhabi.nyu.edu/madar/) (Multi-Arabic Dialect Applications and Resources) project by [Shorouq Zahra](mailto:shorouqjzahra@gmail.com).
  3. Parts of the Cairo Students Code-Switch (CSCS) corpus developed for ["Collection and Analysis of Code-switch Egyptian Arabic-English Speech Corpus"](https://aclanthology.org/L18-1601.pdf) by Hamed et al.
  
  # Usage
  ```python
  from flair.data import Sentence
  from flair.models import SequenceTagger
    
  tagger = SequenceTagger.load("megantosh/flair-arabic-dialects-codeswitch-egy-lev")
  sentence = Sentence('عمرو عادلي أستاذ للاقتصاد السياسي المساعد في الجامعة الأمريكية  بالقاهرة .')
  tagger.predict(sentence)
  for entity in sentence.get_spans('pos'):
      print(entity)
  ```
  
  Due to the right-to-left in left-to-right context, some formatting errors might occur. and your code might appear like [this](https://ibb.co/ky20Lnq), (link accessed on 2020-10-27) 
  
  <!--# Example
  
  # Tagset-->
  
  # Scores & Tagset
  <details> 
  
  | |precision |   recall | f1-score |  support|
  |--|-----------|------|-------------|--------------|
  |INTJ |    0.8182   | 0.9000    |0.8571    |    10|
  |OUN   |  0.9009   | 0.9402    |0.9201      | 435|
  |NUM    | 0.9524   | 0.8333   | 0.8889       | 24|
  |ADJ     |0.8762   | 0.7603  |  0.8142      | 121|
  |ADP     |0.9903    |0.9623 |   0.9761       |106|
  | CCONJ |    0.9600   | 0.9730 |   0.9664 |       74|
  |PROPN |    0.9333   | 0.9333  |  0.9333  |      15|
  | ADV  |   0.9135   | 0.8051  |  0.8559   |    118|
  |VERB   |  0.8852    | 0.9231 |   0.9038   |    117|
  |PRON    | 0.9620    | 0.9465 |   0.9542    |   187|
  |SCONJ |    0.8571   | 0.9474  |  0.9000      |  19|
  |PART  |   0.9350   | 0.9791   | 0.9565       | 191|
  | DET   |  0.9348    | 0.9149  |  0.9247 |       47|
  |PUNCT    | 1.0000    | 1.0000  |  1.0000  |      35|
  | AUX  |   0.9286    | 0.9811  |  0.9541   |     53|
  |MENTION   |  0.9231   |  1.0000  |  0.9600    |    12|
  |     V    | 0.8571   | 0.8780    | 0.8675     |   82|
  | FUT-PART+V+PREP+PRON     |1.0000   | 0.0000   | 0.0000       |  1|
  |  PROG-PART+V+PRON+PREP+PRON |     0.0000  |  1.0000  |  0.0000       |  0|
  |ADJ+NSUFF |    0.6111   | 0.8462   | 0.7097 |       26|
  |NOUN+NSUFF  |   0.8182   | 0.8438   | 0.8308  |      64|
  |PREP+PRON   |  0.9565   | 0.9565   | 0.9565   |     23|
  |                   PUNC    | 0.9941   | 1.0000   | 0.9971    |   169|
  |                    EOS     |1.0000   | 1.0000   | 1.0000    |   70|
  |             NOUN+PRON   |  0.6986   | 0.8500   | 0.7669      |  60|
  |                V+PRON    | 0.7258   | 0.8036   | 0.7627       | 56|
  |            PART+PRON    | 1.0000   | 0.9474   | 0.9730    |    19|
  |          PROG-PART+V    | 0.8333   | 0.9302   | 0.8791 |       43|
  |            DET+NOUN    | 0.9625   | 1.0000   | 0.9809  |      77|
  |     NOUN+NSUFF+PRON    | 0.9091   | 0.7143   | 0.8000   |     14|
  |     PROG-PART+V+PRON    | 0.7083   | 0.9444   | 0.8095    |    18|
  |      PREP+NOUN+NSUFF    | 0.6667   | 0.4000   | 0.5000         5|
  |     NOUN+NSUFF+NSUFF    | 1.0000   | 0.0000   | 0.0000 |        3|
  |                CONJ    | 0.9722   | 1.0000   | 0.9859  |      35|
  |        V+PRON+PRON    | 0.6364   | 0.5833   | 0.6087   |     12|
  |           FOREIGN    | 0.6667   | 0.6667   | 0.6667    |     3|
  |        PREP+NOUN    | 0.6316   | 0.7500  |  0.6857 |       16|
  |  DET+NOUN+NSUFF    | 0.9000   | 0.9310  |  0.9153  |      29|
  |  DET+ADJ+NSUFF    | 1.0000   | 0.5714  |  0.7273   |      7