transformers tensorboard safetensors led text2text-generation summarization generated_from_trainer autotrain_compatible endpoints_compatible region:us
---
tags:
- summarization
- generated_from_trainer
model-index:
- name: finetune-led-35000
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# finetune-led-35000

This model was trained from scratch on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.8474
- Rouge1 Precision: 0.2576
- Rouge1 Recall: 0.3438
- Rouge1 Fmeasure: 0.2911

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 2
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rouge1 Fmeasure | Rouge1 Precision | Rouge1 Recall |
|:-------------:|:-----:|:----:|:---------------:|:---------------:|:----------------:|:-------------:|
| 2.0001        | 0.01  | 10   | 2.8444          | 0.2732          | 0.2484           | 0.3213        |
| 1.7409        | 0.01  | 20   | 2.6895          | 0.2682          | 0.2375           | 0.3203        |
| 1.6096        | 0.02  | 30   | 2.6335          | 0.2797          | 0.246            | 0.3362        |
| 1.7452        | 0.03  | 40   | 2.5817          | 0.285           | 0.251            | 0.3417        |
| 1.6212        | 0.03  | 50   | 2.5765          | 0.2899          | 0.253            | 0.3515        |
| 1.6743        | 0.04  | 60   | 2.4425          | 0.2705          | 0.241            | 0.3188        |
| 1.5229        | 0.04  | 70   | 2.5146          | 0.2802          | 0.2501           | 0.3294        |
| 1.5681        | 0.05  | 80   | 2.4785          | 0.2665          | 0.2302           | 0.3273        |
| 1.5811        | 0.06  | 90   | 2.3622          | 0.287           | 0.2577           | 0.3349        |
| 1.5642        | 0.06  | 100  | 2.4053          | 0.2779          | 0.2449           | 0.3322        |
| 1.5175        | 0.07  | 110  | 2.3818          | 0.2862          | 0.2577           | 0.3324        |
| 1.5073        | 0.08  | 120  | 2.3930          | 0.2905          | 0.25             | 0.3587        |
| 1.5336        | 0.12  | 130  | 2.3619          | 0.2755          | 0.2456           | 0.3242        |
| 1.5155        | 0.13  | 140  | 2.3696          | 0.2752          | 0.243            | 0.3284        |
| 1.5124        | 0.14  | 150  | 2.3240          | 0.2788          | 0.2478           | 0.3296        |
| 1.5187        | 0.15  | 160  | 2.2883          | 0.2739          | 0.2438           | 0.3225        |
| 1.4642        | 0.16  | 170  | 2.3006          | 0.2769          | 0.2442           | 0.3307        |
| 1.6535        | 0.16  | 180  | 2.2766          | 0.2716          | 0.2435           | 0.3165        |
| 1.4924        | 0.17  | 190  | 2.3077          | 0.2782          | 0.2463           | 0.3303        |
| 1.5221        | 0.18  | 200  | 2.2725          | 0.2829          | 0.2502           | 0.3357        |
| 1.388         | 0.19  | 210  | 2.2555          | 0.2811          | 0.2502           | 0.3305        |
| 1.5172        | 0.2   | 220  | 2.2519          | 0.2758          | 0.2458           | 0.3243        |
| 1.498         | 0.21  | 230  | 2.2585          | 0.2809          | 0.2428           | 0.3437        |
| 1.447         | 0.22  | 240  | 2.2110          | 0.2745          | 0.2457           | 0.3206        |
| 1.4637        | 0.23  | 250  | 2.2511          | 0.2854          | 0.252            | 0.3395        |
| 1.5084        | 0.24  | 260  | 2.2070          | 0.2876          | 0.2547           | 0.341         |
| 1.4347        | 0.25  | 270  | 2.2558          | 0.2852          | 0.2493           | 0.344         |
| 1.3108        | 0.26  | 280  | 2.2498          | 0.2815          | 0.2488           | 0.3349        |
| 1.3894        | 0.27  | 290  | 2.2302          | 0.2794          | 0.2454           | 0.3347        |
| 1.4392        | 0.27  | 300  | 2.2021          | 0.276           | 0.2412           | 0.3327        |
| 1.3774        | 0.28  | 310  | 2.1958          | 0.2763          | 0.244            | 0.3286        |
| 1.3696        | 0.29  | 320  | 2.1657          | 0.2811          | 0.2512           | 0.3292        |
| 1.4186        | 0.3   | 330  | 2.1533          | 0.2807          | 0.251            | 0.3288        |
| 1.3073        | 0.31  | 340  | 2.1890          | 0.2805          | 0.2486           | 0.3324        |
| 1.3533        | 0.32  | 350  | 2.1639          | 0.2836          | 0.2546           | 0.3302        |
| 1.3691        | 0.33  | 360  | 2.1255          | 0.2765          | 0.25             | 0.3188        |
| 1.3716        | 0.34  | 370  | 2.1436          | 0.2818          | 0.2512           | 0.3315        |
| 1.3985        | 0.35  | 380  | 2.1160          | 0.2859          | 0.2555           | 0.3351        |
| 1.2993        | 0.36  | 390  | 2.1417          | 0.2756          | 0.2442           | 0.3262        |
| 1.3453        | 0.37  | 400  | 2.1459          | 0.2875          | 0.256            | 0.3386        |
| 1.4036        | 0.37  | 410  | 2.1200          | 0.2811          | 0.2506           | 0.33          |
| 1.3784        | 0.38  | 420  | 2.0878          | 0.2885          | 0.2592           | 0.335         |
| 1.2923        | 0.39  | 430  | 2.1342          | 0.2873          | 0.2535           | 0.3423        |
| 1.3629        | 0.4   | 440  | 2.0887          | 0.2848          | 0.2546           | 0.3328        |
| 1.4151        | 0.41  | 450  | 2.0905          | 0.2857          | 0.254            | 0.3365        |
| 1.3679        | 0.42  | 460  | 2.0829          | 0.285           | 0.2525           | 0.3375        |
| 1.339         | 0.43  | 470  | 2.0786          | 0.2791          | 0.2452           | 0.334         |
| 1.4258        | 0.44  | 480  | 2.0726          | 0.2877          | 0.2603           | 0.3317        |
| 1.4056        | 0.45  | 490  | 2.0995          | 0.2891          | 0.2556           | 0.3428        |
| 1.3548        | 0.46  | 500  | 2.0637          | 0.2757          | 0.2481           | 0.3199        |
| 1.3253        | 0.47  | 510  | 2.0638          | 0.2794          | 0.25             | 0.3266        |
| 1.264         | 0.48  | 520  | 2.0609          | 0.2861          | 0.2587           | 0.3296        |
| 1.3307        | 0.48  | 530  | 2.0396          | 0.2823          | 0.2559           | 0.3243        |
| 1.3536        | 0.49  | 540  | 2.0464          | 0.2824          | 0.2532           | 0.3288        |
| 1.2592        | 0.5   | 550  | 2.0480          | 0.2876          | 0.2592           | 0.3333        |
| 1.358         | 0.51  | 560  | 2.0432          | 0.2818          | 0.2528           | 0.3284        |
| 1.3227        | 0.52  | 570  | 2.0560          | 0.2831          | 0.2502           | 0.3365        |
| 1.3189        | 0.53  | 580  | 2.0311          | 0.2823          | 0.251            | 0.3321        |
| 1.3367        | 0.54  | 590  | 2.0498          | 0.285           | 0.2538           | 0.335         |
| 1.3473        | 0.55  | 600  | 2.0690          | 0.2773          | 0.2452           | 0.3292        |
| 1.2846        | 0.56  | 610  | 2.0555          | 0.2796          | 0.2473           | 0.3321        |
| 1.3066        | 0.57  | 620  | 2.0684          | 0.2799          | 0.245            | 0.3366        |
| 1.3193        | 0.58  | 630  | 2.0467          | 0.2852          | 0.2536           | 0.336         |
| 1.269         | 0.59  | 640  | 2.0381          | 0.2859          | 0.2561           | 0.3337        |
| 1.2906        | 0.59  | 650  | 2.0191          | 0.2831          | 0.2514           | 0.3338        |
| 1.2981        | 0.6   | 660  | 2.0184          | 0.2783          | 0.249            | 0.3251        |
| 1.2888        | 0.61  | 670  | 2.0295          | 0.2827          | 0.2515           | 0.3331        |
| 1.3179        | 0.62  | 680  | 2.0121          | 0.2885          | 0.2611           | 0.333         |
| 1.3313        | 0.63  | 690  | 2.0296          | 0.2739          | 0.2427           | 0.3245        |
| 1.1749        | 0.64  | 700  | 2.0419          | 0.2809          | 0.2507           | 0.3298        |
| 1.3023        | 0.65  | 710  | 2.0275          | 0.2838          | 0.2504           | 0.3379        |
| 1.262         | 0.66  | 720  | 1.9974          | 0.286           | 0.2539           | 0.3378        |
| 1.2906        | 0.67  | 730  | 1.9839          | 0.2839          | 0.252            | 0.3357        |
| 1.24          | 0.68  | 740  | 2.0041          | 0.286           | 0.2528           | 0.3401        |
| 1.239         | 0.69  | 750  | 2.0116          | 0.2789          | 0.2455           | 0.3326        |
| 1.1972        | 0.69  | 760  | 2.0293          | 0.2861          | 0.2536           | 0.3385        |
| 1.2114        | 0.7   | 770  | 2.0271          | 0.2738          | 0.2436           | 0.322         |
| 1.2711        | 0.71  | 780  | 2.0084          | 0.2881          | 0.2548           | 0.3417        |
| 1.262         | 0.72  | 790  | 1.9984          | 0.2806          | 0.2488           | 0.3322        |
| 1.2616        | 0.73  | 800  | 1.9715          | 0.2856          | 0.2541           | 0.3364        |
| 1.2765        | 0.74  | 810  | 1.9718          | 0.2825          | 0.2494           | 0.3356        |
| 1.2151        | 0.75  | 820  | 1.9947          | 0.2857          | 0.2513           | 0.341         |
| 1.3165        | 0.76  | 830  | 1.9854          | 0.2863          | 0.2524           | 0.3411        |
| 1.2704        | 0.77  | 840  | 1.9858          | 0.2903          | 0.2569           | 0.3443        |
| 1.3032        | 0.78  | 850  | 1.9774          | 0.2926          | 0.2583           | 0.3481        |
| 1.2461        | 0.79  | 860  | 1.9596          | 0.2847          | 0.2556           | 0.3314        |
| 1.2288        | 0.8   | 870  | 1.9873          | 0.2868          | 0.2547           | 0.339         |
| 1.2278        | 0.8   | 880  | 1.9712          | 0.289           | 0.2546           | 0.3455        |
| 1.2119        | 0.81  | 890  | 1.9862          | 0.2822          | 0.2478           | 0.338         |
| 1.3363        | 0.82  | 900  | 1.9555          | 0.2871          | 0.2576           | 0.3349        |
| 1.2324        | 0.83  | 910  | 1.9394          | 0.2878          | 0.2588           | 0.3339        |
| 1.2528        | 0.84  | 920  | 1.9593          | 0.2801          | 0.2498           | 0.3289        |
| 1.2572        | 0.85  | 930  | 1.9500          | 0.2825          | 0.2507           | 0.3337        |
| 1.2045        | 0.86  | 940  | 1.9586          | 0.2901          | 0.2589           | 0.3401        |
| 1.2173        | 0.87  | 950  | 1.9551          | 0.281           | 0.2487           | 0.3328        |
| 1.2315        | 0.88  | 960  | 1.9307          | 0.2842          | 0.2533           | 0.3337        |
| 1.2445        | 0.89  | 970  | 1.9362          | 0.2853          | 0.2537           | 0.336         |
| 1.2491        | 0.9   | 980  | 1.9614          | 0.2829          | 0.2482           | 0.3397        |
| 1.3081        | 0.91  | 990  | 1.9500          | 0.2857          | 0.2513           | 0.3411        |
| 1.1928        | 0.91  | 1000 | 1.9439          | 0.2826          | 0.2514           | 0.333         |
| 1.2243        | 0.92  | 1010 | 1.9074          | 0.2883          | 0.259            | 0.3346        |
| 1.2662        | 0.93  | 1020 | 1.9143          | 0.2912          | 0.2593           | 0.3422        |
| 1.2223        | 0.94  | 1030 | 1.9342          | 0.2899          | 0.2581           | 0.3408        |
| 1.2499        | 0.95  | 1040 | 1.9352          | 0.2835          | 0.2507           | 0.3366        |
| 1.3395        | 0.96  | 1050 | 1.9284          | 0.2864          | 0.2548           | 0.3375        |
| 1.1908        | 0.97  | 1060 | 1.9471          | 0.2853          | 0.2528           | 0.3376        |
| 1.2473        | 0.98  | 1070 | 1.9462          | 0.2941          | 0.2613           | 0.3472        |
| 1.2139        | 0.99  | 1080 | 1.9317          | 0.2859          | 0.2534           | 0.338         |
| 1.2534        | 1.0   | 1090 | 1.9278          | 0.2938          | 0.2594           | 0.3488        |
| 1.2204        | 1.01  | 1100 | 1.9177          | 0.2912          | 0.2596           | 0.341         |
| 1.2399        | 1.01  | 1110 | 1.9236          | 0.2903          | 0.2568           | 0.3443        |
| 1.1541        | 1.02  | 1120 | 1.9441          | 0.2889          | 0.2548           | 0.3431        |
| 1.1038        | 1.03  | 1130 | 1.9223          | 0.2925          | 0.2626           | 0.3399        |
| 1.1177        | 1.04  | 1140 | 1.9244          | 0.2881          | 0.2565           | 0.338         |
| 1.1224        | 1.05  | 1150 | 1.9324          | 0.2884          | 0.2547           | 0.3428        |
| 1.104         | 1.06  | 1160 | 1.9188          | 0.2798          | 0.2482           | 0.3304        |
| 1.175         | 1.07  | 1170 | 1.9042          | 0.2915          | 0.2618           | 0.3388        |
| 1.102         | 1.08  | 1180 | 1.9325          | 0.2853          | 0.253            | 0.3372        |
| 1.0829        | 1.09  | 1190 | 1.9503          | 0.2819          | 0.2478           | 0.3371        |
| 1.1842        | 1.1   | 1200 | 1.9360          | 0.2784          | 0.2438           | 0.3346        |
| 1.1552        | 1.11  | 1210 | 1.9055          | 0.286           | 0.254            | 0.3369        |
| 1.1266        | 1.12  | 1220 | 1.9106          | 0.286           | 0.2555           | 0.3345        |
| 1.1288        | 1.13  | 1230 | 1.9072          | 0.2865          | 0.2566           | 0.3336        |
| 1.1722        | 1.13  | 1240 | 1.9114          | 0.2856          | 0.2539           | 0.3364        |
| 1.1514        | 1.14  | 1250 | 1.9180          | 0.2906          | 0.2561           | 0.3461        |
| 1.1642        | 1.15  | 1260 | 1.9226          | 0.2918          | 0.2571           | 0.3475        |
| 1.1464        | 1.16  | 1270 | 1.9004          | 0.2819          | 0.2525           | 0.3283        |
| 1.1829        | 1.17  | 1280 | 1.9181          | 0.2935          | 0.2568           | 0.3524        |
| 1.17          | 1.18  | 1290 | 1.9031          | 0.2848          | 0.2523           | 0.3369        |
| 1.0751        | 1.19  | 1300 | 1.9334          | 0.2875          | 0.2531           | 0.3428        |
| 1.1327        | 1.2   | 1310 | 1.8966          | 0.2891          | 0.2568           | 0.3407        |
| 1.1319        | 1.21  | 1320 | 1.9076          | 0.2902          | 0.2575           | 0.3422        |
| 1.106         | 1.22  | 1330 | 1.8941          | 0.2908          | 0.259            | 0.3413        |
| 1.1721        | 1.23  | 1340 | 1.8956          | 0.2945          | 0.2609           | 0.3479        |
| 1.1964        | 1.23  | 1350 | 1.9140          | 0.2851          | 0.2513           | 0.3389        |
| 1.1195        | 1.24  | 1360 | 1.9168          | 0.2917          | 0.2561           | 0.3483        |
| 1.1352        | 1.25  | 1370 | 1.8962          | 0.286           | 0.253            | 0.3389        |
| 1.1164        | 1.26  | 1380 | 1.9050          | 0.2916          | 0.258            | 0.3453        |
| 1.1219        | 1.27  | 1390 | 1.9054          | 0.2872          | 0.2551           | 0.3386        |
| 1.1571        | 1.28  | 1400 | 1.8845          | 0.2896          | 0.2574           | 0.3402        |
| 1.2033        | 1.29  | 1410 | 1.8985          | 0.2852          | 0.2532           | 0.3362        |
| 1.1114        | 1.3   | 1420 | 1.8956          | 0.2882          | 0.2559           | 0.3395        |
| 1.1268        | 1.31  | 1430 | 1.8955          | 0.2895          | 0.2563           | 0.3424        |
| 1.1347        | 1.32  | 1440 | 1.8883          | 0.2865          | 0.2524           | 0.3412        |
| 1.0345        | 1.33  | 1450 | 1.8960          | 0.2895          | 0.2571           | 0.3412        |
| 1.1231        | 1.34  | 1460 | 1.8873          | 0.29            | 0.2575           | 0.3415        |
| 1.236         | 1.34  | 1470 | 1.8744          | 0.2898          | 0.2578           | 0.34          |
| 1.1054        | 1.35  | 1480 | 1.8867          | 0.2884          | 0.2546           | 0.3425        |
| 1.1393        | 1.36  | 1490 | 1.8907          | 0.2927          | 0.2605           | 0.344         |
| 1.1004        | 1.37  | 1500 | 1.8953          | 0.288           | 0.2543           | 0.3416        |
| 1.1482        | 1.38  | 1510 | 1.8731          | 0.288           | 0.2568           | 0.3377        |
| 1.1701        | 1.39  | 1520 | 1.8868          | 0.2866          | 0.2525           | 0.3411        |
| 1.1233        | 1.4   | 1530 | 1.8803          | 0.2882          | 0.2562           | 0.3385        |
| 1.0685        | 1.41  | 1540 | 1.8843          | 0.2935          | 0.262            | 0.3433        |
| 1.0657        | 1.42  | 1550 | 1.8748          | 0.2892          | 0.2553           | 0.3437        |
| 1.1275        | 1.43  | 1560 | 1.8804          | 0.2881          | 0.2553           | 0.3405        |
| 1.0883        | 1.44  | 1570 | 1.8803          | 0.2868          | 0.2527           | 0.3412        |
| 1.1096        | 1.45  | 1580 | 1.8862          | 0.2927          | 0.2586           | 0.3472        |
| 1.1521        | 1.45  | 1590 | 1.8724          | 0.288           | 0.2564           | 0.3379        |
| 1.142         | 1.46  | 1600 | 1.8788          | 0.2926          | 0.2593           | 0.3454        |
| 1.0451        | 1.47  | 1610 | 1.8684          | 0.2863          | 0.2571           | 0.3324        |
| 1.1294        | 1.48  | 1620 | 1.8704          | 0.2902          | 0.2569           | 0.3427        |
| 1.1671        | 1.49  | 1630 | 1.8756          | 0.2909          | 0.259            | 0.3413        |
| 1.2252        | 1.5   | 1640 | 1.8618          | 0.2937          | 0.2599           | 0.347         |
| 1.0834        | 1.51  | 1650 | 1.8776          | 0.2909          | 0.2589           | 0.3416        |
| 1.0417        | 1.52  | 1660 | 1.8658          | 0.2911          | 0.2592           | 0.342         |
| 1.1036        | 1.53  | 1670 | 1.8789          | 0.289           | 0.2553           | 0.343         |
| 1.1575        | 1.54  | 1680 | 1.8608          | 0.2927          | 0.2597           | 0.3452        |
| 1.058         | 1.55  | 1690 | 1.8804          | 0.2921          | 0.2585           | 0.3455        |
| 1.1251        | 1.55  | 1700 | 1.8682          | 0.2973          | 0.2637           | 0.3503        |
| 1.0818        | 1.56  | 1710 | 1.8800          | 0.2887          | 0.2544           | 0.3432        |
| 1.1346        | 1.57  | 1720 | 1.8577          | 0.289           | 0.2564           | 0.3404        |
| 1.1024        | 1.58  | 1730 | 1.8681          | 0.2946          | 0.2608           | 0.3482        |
| 1.0823        | 1.59  | 1740 | 1.8603          | 0.2908          | 0.2584           | 0.342         |
| 1.0562        | 1.6   | 1750 | 1.8670          | 0.2931          | 0.2584           | 0.3484        |
| 1.1128        | 1.61  | 1760 | 1.8576          | 0.2926          | 0.2603           | 0.3439        |
| 1.0769        | 1.62  | 1770 | 1.8709          | 0.2902          | 0.2568           | 0.3434        |
| 1.0422        | 1.63  | 1780 | 1.8597          | 0.2911          | 0.2587           | 0.3425        |
| 1.1912        | 1.64  | 1790 | 1.8648          | 0.2911          | 0.2574           | 0.3448        |
| 1.1349        | 1.65  | 1800 | 1.8667          | 0.2933          | 0.2606           | 0.3453        |
| 1.1195        | 1.66  | 1810 | 1.8684          | 0.2899          | 0.2568           | 0.3422        |
| 1.1186        | 1.66  | 1820 | 1.8581          | 0.2908          | 0.2579           | 0.3434        |
| 1.0795        | 1.67  | 1830 | 1.8639          | 0.2907          | 0.2561           | 0.3462        |
| 1.1394        | 1.68  | 1840 | 1.8467          | 0.2929          | 0.2602           | 0.3446        |
| 1.0743        | 1.69  | 1850 | 1.8682          | 0.291           | 0.2585           | 0.3428        |
| 1.0954        | 1.7   | 1860 | 1.8504          | 0.2928          | 0.2603           | 0.3445        |
| 1.0938        | 1.71  | 1870 | 1.8604          | 0.2916          | 0.2589           | 0.3436        |
| 1.1093        | 1.72  | 1880 | 1.8427          | 0.2897          | 0.2581           | 0.3398        |
| 1.1399        | 1.73  | 1890 | 1.8715          | 0.2891          | 0.2561           | 0.3422        |
| 1.1574        | 1.74  | 1900 | 1.8448          | 0.2893          | 0.2568           | 0.3409        |
| 1.1244        | 1.75  | 1910 | 1.8594          | 0.2927          | 0.2597           | 0.3453        |
| 1.1205        | 1.76  | 1920 | 1.8492          | 0.2922          | 0.2606           | 0.3425        |
| 1.1218        | 1.77  | 1930 | 1.8547          | 0.2906          | 0.2591           | 0.3401        |
| 1.1208        | 1.77  | 1940 | 1.8605          | 0.2924          | 0.2588           | 0.3459        |
| 1.0983        | 1.78  | 1950 | 1.8425          | 0.2933          | 0.2611           | 0.3442        |
| 1.1992        | 1.79  | 1960 | 1.8587          | 0.2907          | 0.2565           | 0.3455        |
| 1.1724        | 1.8   | 1970 | 1.8413          | 0.2909          | 0.2576           | 0.3435        |
| 1.1344        | 1.81  | 1980 | 1.8494          | 0.2904          | 0.2583           | 0.3413        |
| 1.1469        | 1.82  | 1990 | 1.8463          | 0.2911          | 0.2581           | 0.3437        |
| 1.1491        | 1.83  | 2000 | 1.8530          | 0.2905          | 0.2568           | 0.3441        |
| 1.0913        | 1.84  | 2010 | 1.8493          | 0.2913          | 0.258            | 0.3443        |
| 1.1298        | 1.85  | 2020 | 1.8465          | 0.2905          | 0.2573           | 0.3433        |
| 1.1202        | 1.86  | 2030 | 1.8488          | 0.2892          | 0.256            | 0.3419        |
| 1.1439        | 1.87  | 2040 | 1.8494          | 0.2911          | 0.2584           | 0.3428        |
| 1.0328        | 1.87  | 2050 | 1.8469          | 0.2907          | 0.2582           | 0.3423        |
| 1.1347        | 1.88  | 2060 | 1.8426          | 0.29            | 0.2576           | 0.341         |
| 1.094         | 1.89  | 2070 | 1.8480          | 0.2905          | 0.2577           | 0.3425        |
| 1.1201        | 1.9   | 2080 | 1.8542          | 0.2896          | 0.2568           | 0.3415        |
| 1.1475        | 1.91  | 2090 | 1.8520          | 0.29            | 0.2574           | 0.3416        |
| 1.0793        | 1.92  | 2100 | 1.8506          | 0.2897          | 0.2569           | 0.3414        |
| 1.0669        | 1.93  | 2110 | 1.8484          | 0.2907          | 0.2577           | 0.3426        |
| 1.1276        | 1.94  | 2120 | 1.8487          | 0.2904          | 0.2573           | 0.3427        |
| 1.0902        | 1.95  | 2130 | 1.8487          | 0.2904          | 0.2575           | 0.3423        |
| 1.1449        | 1.96  | 2140 | 1.8490          | 0.2898          | 0.2569           | 0.3419        |
| 1.1142        | 1.97  | 2150 | 1.8505          | 0.29            | 0.2569           | 0.3424        |
| 1.1475        | 1.98  | 2160 | 1.8501          | 0.2895          | 0.2561           | 0.3424        |
| 1.0663        | 1.98  | 2170 | 1.8485          | 0.2906          | 0.2571           | 0.3434        |
| 1.1454        | 1.99  | 2180 | 1.8475          | 0.2907          | 0.2573           | 0.3435        |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.1+cu121
- Datasets 2.14.5
- Tokenizers 0.15.1
