readme: ---
  license: cc-by-4.0
  ---
  
  ## Readability benchmark (ES): bertin-es-paragraphs-2class
  
  This project is part of a series of models from the paper "A Benchmark for Neural Readability Assessment of Texts in Spanish". 
  You can find more details about the project in our [GitHub](https://github.com/lmvasque/readability-es-benchmark).
  
  ## Models
  
  Our models were fine-tuned in multiple settings, including readability assessment in 2-class (simple/complex) and 3-class (basic/intermediate/advanced) for sentences and paragraph datasets.
  You can find more details in our [paper](https://drive.google.com/file/d/1KdwvqrjX8MWYRDGBKeHmiR1NCzDcVizo/view?usp=share_link). 
  These are the available models you can use (current model page in bold):
  
  | Model                                                                                                     | Granularity    | # classes |
  |-----------------------------------------------------------------------------------------------------------|----------------|:---------:|
  | **[BERTIN (ES)](https://huggingface.co/lmvasque/readability-es-benchmark-bertin-es-paragraphs-2class)**   | **paragraphs**     |     **2**     |
  | [BERTIN (ES)](https://huggingface.co/lmvasque/readability-es-benchmark-bertin-es-paragraphs-3class)       | paragraphs     |     3     |
  | [mBERT (ES)](https://huggingface.co/lmvasque/readability-es-benchmark-mbert-es-paragraphs-2class)         | paragraphs     |     2     |
  | [mBERT (ES)](https://huggingface.co/lmvasque/readability-es-benchmark-mbert-es-paragraphs-3class)         | paragraphs     |     3     |
  | [mBERT (EN+ES)](https://huggingface.co/lmvasque/readability-es-benchmark-mbert-en-es-paragraphs-3class) | paragraphs |     3     |
  | [BERTIN (ES)](https://huggingface.co/lmvasque/readability-es-benchmark-bertin-es-sentences-2class)        | sentences      |     2     |
  | [BERTIN (ES)](https://huggingface.co/lmvasque/readability-es-benchmark-bertin-es-sentences-3class)        | sentences      |     3     |
  | [mBERT (ES)](https://huggingface.co/lmvasque/readability-es-benchmark-mbert-es-sentences-2class)          | sentences      |     2     |
  | [mBERT (ES)](https://huggingface.co/lmvasque/readability-es-benchmark-mbert-es-sentences-3class)          | sentences      |     3     |
  | [mBERT (EN+ES)](https://huggingface.co/lmvasque/readability-es-benchmark-mbert-en-es-sentences-3class) | sentences  |   3   |
  
  For the zero-shot setting, we used the original models [BERTIN](bertin-project/bertin-roberta-base-spanish) and [mBERT](https://huggingface.co/bert-base-multilingual-uncased) with no further training.
  ## Results
  
  These are our results for all the readability models in different settings. Please select your model based on the desired performance:
  
  | Granularity | Model         | F1 Score (2-class)  |  Precision (2-class)  |  Recall (2-class)  |  F1 Score (3-class)  |  Precision (3-class)  |  Recall (3-class)  |
  |-------------|---------------|:-------------------:|:---------------------:|:------------------:|:--------------------:|:---------------------:|:------------------:|
  | Paragraph   | Baseline (TF-IDF+LR) |        0.829        |         0.832         |       0.827        |        0.556         |         0.563         |       0.550        |
  | Paragraph   | BERTIN (Zero) |        0.308        |         0.222         |       0.500        |        0.227         |         0.284         |       0.338        |
  | Paragraph   | BERTIN (ES)   |        0.924        |         0.923         |       0.925        |        0.772         |         0.776         |       0.768        |
  | Paragraph   | mBERT (Zero)  |        0.308        |         0.222         |       0.500        |        0.253         |         0.312         |       0.368        |
  | Paragraph   | mBERT (EN)    |          -          |           -           |         -          |        0.505         |         0.560         |       0.552        |
  | Paragraph   | mBERT (ES)    |      **0.933**      |       **0.932**       |     **0.936**      |        0.776         |         0.777         |       0.778        |
  | Paragraph   | mBERT (EN+ES) |          -          |           -           |         -          |      **0.779**       |       **0.783**       |     **0.779**      |
  | Sentence | Baseline (TF-IDF+LR) |        0.811        |         0.814         |       0.808        |        0.525         |         0.531         |       0.521        |
  | Sentence | BERTIN (Zero) |        0.367        |         0.290         |       0.500        |        0.188         |         0.232         |       0.335        |
  | Sentence | BERTIN (ES)   |      **0.900**      |       **0.900**       |     **0.900**      |      **0.699**       |       **0.701**       |     **0.698**      |
  | Sentence | mBERT (Zero)  |        0.367        |         0.290         |       0.500        |        0.278         |         0.329         |       0.351        |
  | Sentence | mBERT (EN)    |          -          |           -           |       