readme: ---
  language: ar
  tags:
  - pytorch
  - tf
  - QARiB
  - qarib
  datasets:
  - arabic_billion_words
  - open_subtitles
  - twitter
  - Farasa
  metrics:
  - f1
  widget:
   - text: "و+قام ال+مدير [MASK]"
  ---
  # QARiB: QCRI Arabic and Dialectal BERT
  ## About QARiB Farasa
  QCRI Arabic and Dialectal BERT  (QARiB) model, was trained on a collection of ~ 420 Million tweets and ~ 180 Million sentences of text.
  For the tweets, the data was collected using twitter API and using language filter. `lang:ar`. For the text data, it was a combination from
  [Arabic GigaWord](url), [Abulkhair Arabic Corpus]() and [OPUS](http://opus.nlpl.eu/).
  QARiB: Is the Arabic name for "Boat".
  ## Model and Parameters:
  - Data size: 14B tokens 
  - Vocabulary: 64k
  - Iterations:  10M
  - Number of Layers: 12
  ## Training QARiB
  See details in [Training QARiB](https://github.com/qcri/QARIB/Training_QARiB.md)
  ## Using QARiB
  You can use the raw model for either masked language modeling or next sentence prediction, but it's mostly intended to be fine-tuned on a downstream task. See the model hub to look for fine-tuned versions on a task that interests you. For more details, see [Using QARiB](https://github.com/qcri/QARIB/Using_QARiB.md)
  
  This model expects the data to be segmented. You may use [Farasa Segmenter](https://farasa-api.qcri.org/segmentation/) API.
  
  ### How to use
  You can use this model directly with a pipeline for masked language modeling:
  ```python
  >>>from transformers import pipeline
  >>>fill_mask = pipeline("fill-mask", model="./models/bert-base-qarib_far")
  >>> fill_mask("و+قام ال+مدير [MASK]")
  
  >>> fill_mask("و+قام+ت ال+مدير+ة [MASK]")
  
  >>> fill_mask("قللي وشفيييك يرحم [MASK]")
  
  ```
  ## Evaluations:
  
  
  ## Model Weights and Vocab Download
  From Huggingface site: https://huggingface.co/qarib/bert-base-qarib_far
  ## Contacts
  Ahmed Abdelali, Sabit Hassan, Hamdy Mubarak, Kareem Darwish and Younes Samih
  ## Reference
  ```
  @article{abdelali2021pretraining,
      title={Pre-Training BERT on Arabic Tweets: Practical Considerations},
      author={Ahmed Abdelali and Sabit Hassan and Hamdy Mubarak and Kareem Darwish and Younes Samih},
      year={2021},
      eprint={2102.10684},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
  }
  ```