readme: ---
  language: nl
  license: mit
  datasets:
  - dbrd
  model-index:
  - name: robbertje-merged-dutch-sentiment
    results:
    - task:
        type: text-classification
        name: Text Classification
      dataset:
        name: dbrd
        type: sentiment-analysis
        split: test
      metrics:
      - name: Accuracy
        type: accuracy
        value: 0.9294064748201439
  widget:
  - text: "Ik erken dat dit een boek is, daarmee is alles gezegd."
  - text: "Prachtig verhaal, heel mooi verteld en een verrassend einde... Een topper!"
  thumbnail: "https://github.com/iPieter/robbertje/raw/master/images/robbertje_logo_with_name.png"
  tags:
  - Dutch
  - Flemish
  - RoBERTa
  - RobBERT
  ---
  
  <p align="center"> 
      <img src="https://github.com/iPieter/robbertje/raw/master/images/robbertje_logo_with_name.png" alt="RobBERTje: A collection of distilled Dutch models" width="75%">
   </p>
  
  # RobBERTje finetuned for sentiment analysis on DBRD
  
  This is a finetuned model based on [RobBERTje (merged)](https://huggingface.co/DTAI-KULeuven/robbertje-1-gb-non-shuffled). We used [DBRD](https://huggingface.co/datasets/dbrd), which consists of book reviews from [hebban.nl](hebban.nl). Hence our example sentences about books. We did some limited experiments to test if this also works for other domains, but this was not exactly amazing. 
  
  We released a distilled model and a `base`-sized model. Both models perform quite well, so there is only a slight performance tradeoff:
  
  
  | Model          | Identifier                                                             | Layers | #Params.  | Accuracy  |
  |----------------|------------------------------------------------------------------------|--------|-----------|-----------|
  | RobBERT (v2)   | [`DTAI-KULeuven/robbert-v2-dutch-sentiment`](https://huggingface.co/DTAI-KULeuven/robbert-v2-dutch-sentiment)    | 12     | 116 M     |93.3*      | 
  | RobBERTje - Merged (p=0.5)| [`DTAI-KULeuven/robbertje-merged-dutch-sentiment`](https://huggingface.co/DTAI-KULeuven/robbertje-merged-dutch-sentiment) | 6 | 74 M      |92.9       |
  
  *The results of RobBERT are of a different run than the one reported in the paper.
  
  # Training data and setup
  We used the [Dutch Book Reviews Dataset (DBRD)](https://huggingface.co/datasets/dbrd) from van der Burgh et al. (2019).
  Originally, these reviews got a five-star rating, but this has been converted to positive (⭐️⭐️⭐️⭐️ and ⭐️⭐️⭐️⭐️⭐️), neutral (⭐️⭐️⭐️) and negative (⭐️ and ⭐️⭐️). 
  We used 19.5k reviews for the training set, 528 reviews for the validation set and 2224 to calculate the final accuracy.
  
  The validation set was used to evaluate a random hyperparameter search over the learning rate, weight decay and gradient accumulation steps. 
  The full training details are available in [`training_args.bin`](https://huggingface.co/DTAI-KULeuven/robbert-v2-dutch-sentiment/blob/main/training_args.bin) as a binary PyTorch file. 
  
  # Limitations and biases
  - The domain of the reviews is limited to book reviews.
  - Most authors of the book reviews were women, which could have caused [a difference in performance for reviews written by men and women](https://www.aclweb.org/anthology/2020.findings-emnlp.292). 
  
  ## Credits and citation
  
  This project is created by [Pieter Delobelle](https://people.cs.kuleuven.be/~pieter.delobelle), [Thomas Winters](https://thomaswinters.be) and [Bettina Berendt](https://people.cs.kuleuven.be/~bettina.berendt/).
  If you would like to cite our paper or models, you can use the following BibTeX:
  
  ```
  @article{Delobelle_Winters_Berendt_2021,
  	title        = {RobBERTje: A Distilled Dutch BERT Model},
  	author       = {Delobelle, Pieter and Winters, Thomas and Berendt, Bettina},
  	year         = 2021,
  	month        = {Dec.},
  	journal      = {Computational Linguistics in the Netherlands Journal},
  	volume       = 11,
  	pages        = {125–140},
  	url          = {https://www.clinjournal.org/clinj/article/view/131}
  }
  
  
  @inproceedings{delobelle2020robbert,
      title = "{R}ob{BERT}: a {D}utch {R}o{BERT}a-based {L}anguage {M}odel",
      author = "Delobelle, Pieter  and
        Winters, Thomas  and
        Berendt, Bettina",
      booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2020",
      month = nov,
      year = "2020",
      address = "Online",
      publisher = "Association for Computational Linguistics",
      url = "https://www.aclweb.org/anthology/2020.findings-emnlp.292",
      doi = "10.18653/v1/2020.findings-emnlp.292",
      pages = "3255--3265"
  }
  ```