eval samples | 1000             | 1000              |  Note that the amount of training data is limited to a fraction of the total dataset sizes, therefore the scores below can only be used to compare the 'transfer-learning' strength. The fine-tuned checkpoints for this evaluation  are not saved, since they were trained for comparison of pre-trained models only.  The numbers for summarization are the Rouge scores on 1000 documents from the test split.  |                         |   [t5-base-dutch](https://huggingface.co/yhavinga/t5-base-dutch) |   [t5-v1.1-base-dutch-uncased](https://huggingface.co/yhavinga/t5-v1.1-base-dutch-uncased) |   [t5-v1.1-base-dutch-cased](https://huggingface.co/yhavinga/t5-v1.1-base-dutch-cased) |   [t5-v1_1-base-dutch-english-cased](https://huggingface.co/yhavinga/t5-v1_1-base-dutch-english-cased) |   [t5-v1_1-base-dutch-english-cased-1024](https://huggingface.co/yhavinga/t5-v1_1-base-dutch-english-cased-1024) |   [t5-small-24L-dutch-english](https://huggingface.co/yhavinga/t5-small-24L-dutch-english) |   [t5-xl-4L-dutch-english-cased](https://huggingface.co/yhavinga/t5-xl-4L-dutch-english-cased) |   [t5-base-36L-dutch-english-cased](https://huggingface.co/yhavinga/t5-base-36L-dutch-english-cased) |   [t5-eff-large-8l-dutch-english-cased](https://huggingface.co/yhavinga/t5-eff-large-8l-dutch-english-cased) |   mt5-base | |:------------------------|----------------:|-----------------------------:|---------------------------:|-----------------------------------:|----------------------------------------:|-----------------------------:|-------------------------------:|----------------------------------:|--------------------------------------:|-----------:| | *rouge1* |           33.38 |                        33.97 |                      34.39 |                              33.38 |                                   34.97 |                        34.38 |                          30.35 |                             **35.04** |                                 34.04 |      33.25 | | *rouge2* |           13.32 |                        13.85 |                      13.98 |                              13.47 |                                   14.01 |                        13.89 |                          11.57 |                             **14.23** |                                 13.76 |      12.74 | | *rougeL* |           24.22 |                        24.72 |                      25.1  |                              24.34 |                                   24.99 |                        **25.25** |                          22.69 |                             25.05 |                                 24.75 |      23.5  | | *rougeLsum* |           30.23 |                        30.9  |                      31.44 |                              30.51 |                                   32.01 |                        31.38 |                          27.5  |                             **32.12** |                                 31.12 |      30.15 | | *samples_per_second* |            3.18 |                         3.02 |                       2.99 |                               3.22 |                                    2.97 |                         1.57 |                           2.8  |                              0.61 |                                  **3.27** |       1.22 |  The models below have been evaluated for English to Dutch translation. Note that the first four models are pre-trained on Dutch only. That they still perform adequate is probably because the translation direction is English to Dutch. The numbers reported are the Bleu scores on 1000 documents from the test split.  |                                |   [t5-base-dutch](https://huggingface.co/yhavinga/t5-base-dutch) |   [t5-v1.1-base-dutch-uncased](https://huggingface.co/yhavinga/t5-v1.1-base-dutch-uncased) |   [t5-v1.1-base-dutch-cased](https://huggingface.co/yhavinga/t5-v1.1-base-dutch-cased) |   [t5-v1.1-large-dutch-cased](https://huggingface.co/yhavinga/t5-v1.1-large-dutch-cased) |   [t5-v1_1-base-dutch-english-cased](https://huggingface.co/yhavinga/t5-v1_1-base-dutch-english-cased) |   [t5-v1_1-base-dutch-english-cased-1024](https://huggingface.co/yhavinga/t5-v1_1-base-dutch-english-cased-1024) |   [t5-small-24L-dutch-english](https://huggingface.co/yhavinga/t5-small-24L-dutch-english) |   [t5-xl-4L-dutch-english-cased](https://huggingface.co/yhavinga/t5-xl-4L-dutch-english-cased) |   [t5-base-36L-dutch-english-cased](https://huggingface.co/yhavinga/t5-base-36L-dutch-english-cased) |   [t5-eff-large-8l-dutch-english-cased](https://huggingface.co/yhavinga/t5-eff-large-8l-dutch-english-cased) |   mt5-base | |:-------------------------------|----------------:|-----------------------------:|---------------------------:|----------------------------:|-----------------------------------:|----------------------------------------:|-----------------------------:|-------------------------------:|----------------------------------:|--------------------------------------:|-----------:| | *precision_ng1* |           74.17 |                        78.09 |                      77.08 |                       72.12 |                              77.19 |                                   78.76 |                        78.59 |                          77.3  |                             **79.75** |                                 78.88 |      73.47 | | *precision_ng2* |           52.42 |                        57.52 |                      55.31 |                       48.7  |                              55.39 |                                   58.01 |                        57.83 |                          55.27 |                             **59.89** |                                 58.27 |      50.12 | | *precision_ng3* |           39.55 |                        45.2  |                      42.54 |                       35.54 |                              42.25 |                                   45.13 |                        45.02 |                          42.06 |                             **47.4**  |                                 45.95 |      36.59 | | *precision_ng4* |           30.23 |                        36.04 |                      33.26 |                       26.27 |                              32.74 |                                   35.72 |                        35.41 |                          32.61 |                             **38.1**  |                                 36.91 |      27.26 | | *bp* |            0.99 |                         0.98 |                       0.97 |                        0.98 |                               0.98 |                                    0.98 |                         0.98 |                           0.97 |                              0.98 |                                  0.98 |       0.98 | | *score* |           45.88 |                        51.21 |                      48.31 |                       41.59 |                              48.17 |                                   51.31 |                        50.82 |                          47.83 |                             **53**    |                                 51.79 |      42.74 | | *samples_per_second* |           **45.19** |                        45.05 |                      38.67 |                       10.12 |                              42.19 |                                   42.61 |                        12.85 |                          33.74 |                              9.07 |                                 37.86 |       9.03 |   