readme: ---
  license: apache-2.0
  language: fa
  widget:
   - text: "از هر دستی بگیری از همون [MASK] میدی"
   - text: "این آخرین باره بهت [MASK] میگم"
   - text: 'چرا آن جوان بیچاره را به سخره [MASK]'
   - text: 'آخه محسن [MASK] هم شد خواننده؟'
   - text: 'پسر عجب [MASK] زد'
  tags:
  - bert-fa
  - bert-persian
  model-index:
  - name: dal-bert
    results: []
  ---
  
  
  DAL-BERT: Another pre-trained language model for Persian
  ---
  
  DAL-BERT is a transformer-based model trained on more than 80 gigabytes of Persian text including both formal and informal (conversational) contexts. The architecture of this model follows the original BERT [[Devlin et al.](https://arxiv.org/abs/1810.04805)].
  
  How to use the Model
  ---
  ```python
  from transformers import BertForMaskedLM, BertTokenizer, pipeline
  model = BertForMaskedLM.from_pretrained('sharif-dal/dal-bert')
  tokenizer = BertTokenizer.from_pretrained('sharif-dal/dal-bert')
  fill_sentence = pipeline('fill-mask', model=model, tokenizer=tokenizer)
  fill_sentence('اینجا جمله مورد نظر خود را بنویسید و کلمه موردنظر را [MASK] کنید')
  ```
  
  The Training Data
  ---
  The abovementioned model was trained on a bunch of newspapers, news agencies' websites, technology-related sources, people's comments, magazines, literary criticism, and some blogs.
  
  Evaluation
  ---
  
  | Training Loss | Epoch | Step  |
  |:-------------:|:-----:|:-----:|
  | 2.1855        | 13   | 7649486 |
  
  Contributors
  ---
  - Arman Malekzadeh [[Github](https://github.com/arm-on)]
  - Amirhossein Ramazani, Master's Student in AI @ Sharif University of Technology [[Linkedin](https://www.linkedin.com/in/amirhossein-ramazani/)] [[Github](https://github.com/amirhossein1376)]
  