transformers tensorboard safetensors led text2text-generation summarization generated_from_trainer autotrain_compatible endpoints_compatible region:us
---
tags:
- summarization
- generated_from_trainer
model-index:
- name: finetune-led2
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# finetune-led2

This model was trained from scratch on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.9753
- Rouge1 Precision: 0.2523
- Rouge1 Recall: 0.3403
- Rouge1 Fmeasure: 0.2864

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 2
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rouge1 Fmeasure | Rouge1 Precision | Rouge1 Recall |
|:-------------:|:-----:|:----:|:---------------:|:---------------:|:----------------:|:-------------:|
| 2.0588        | 0.06  | 10   | 2.6812          | 0.2738          | 0.2434           | 0.3302        |
| 1.89          | 0.13  | 20   | 2.5273          | 0.2786          | 0.2527           | 0.3216        |
| 1.7108        | 0.19  | 30   | 2.4943          | 0.281           | 0.2471           | 0.337         |
| 1.6632        | 0.26  | 40   | 2.4320          | 0.2805          | 0.2456           | 0.3387        |
| 1.6764        | 0.32  | 50   | 2.3851          | 0.2806          | 0.2473           | 0.3352        |
| 1.6218        | 0.38  | 60   | 2.3718          | 0.2741          | 0.2395           | 0.3306        |
| 1.6105        | 0.11  | 70   | 2.3655          | 0.2851          | 0.2486           | 0.3446        |
| 1.6344        | 0.13  | 80   | 2.3529          | 0.2798          | 0.2485           | 0.3301        |
| 1.6359        | 0.14  | 90   | 2.3933          | 0.2845          | 0.2505           | 0.3394        |
| 1.5291        | 0.16  | 100  | 2.4088          | 0.2802          | 0.2453           | 0.3375        |
| 1.483         | 0.18  | 110  | 2.3258          | 0.2891          | 0.2575           | 0.3397        |
| 1.5317        | 0.19  | 120  | 2.2498          | 0.2785          | 0.2508           | 0.3223        |
| 1.5924        | 0.21  | 130  | 2.2690          | 0.2797          | 0.2471           | 0.3326        |
| 1.5086        | 0.22  | 140  | 2.3032          | 0.2768          | 0.2439           | 0.33          |
| 1.4654        | 0.24  | 150  | 2.2767          | 0.2792          | 0.2511           | 0.324         |
| 1.4575        | 0.26  | 160  | 2.2510          | 0.2825          | 0.2493           | 0.3356        |
| 1.5274        | 0.27  | 170  | 2.2635          | 0.2787          | 0.2446           | 0.3338        |
| 1.4993        | 0.29  | 180  | 2.2363          | 0.2845          | 0.2496           | 0.3406        |
| 1.5557        | 0.3   | 190  | 2.2319          | 0.282           | 0.2479           | 0.3364        |
| 1.476         | 0.32  | 200  | 2.2196          | 0.2735          | 0.2437           | 0.3216        |
| 1.4695        | 0.34  | 210  | 2.2604          | 0.2828          | 0.2477           | 0.3394        |
| 1.4819        | 0.35  | 220  | 2.2487          | 0.2806          | 0.2481           | 0.3329        |
| 1.4132        | 0.37  | 230  | 2.2176          | 0.2903          | 0.2543           | 0.348         |
| 1.4574        | 0.38  | 240  | 2.1912          | 0.2847          | 0.2513           | 0.3378        |
| 1.5263        | 0.4   | 250  | 2.1772          | 0.2761          | 0.2463           | 0.3234        |
| 1.4955        | 0.42  | 260  | 2.2065          | 0.2859          | 0.2483           | 0.3467        |
| 1.4698        | 0.43  | 270  | 2.1837          | 0.2831          | 0.2495           | 0.3368        |
| 1.5245        | 0.45  | 280  | 2.1769          | 0.2846          | 0.2513           | 0.3377        |
| 1.4691        | 0.46  | 290  | 2.1495          | 0.2832          | 0.252            | 0.3326        |
| 1.4203        | 0.48  | 300  | 2.1742          | 0.2791          | 0.2483           | 0.328         |
| 1.4827        | 0.5   | 310  | 2.1789          | 0.2793          | 0.2445           | 0.335         |
| 1.4478        | 0.51  | 320  | 2.1456          | 0.279           | 0.2483           | 0.328         |
| 1.4188        | 0.53  | 330  | 2.1637          | 0.2795          | 0.2499           | 0.3268        |
| 1.4096        | 0.54  | 340  | 2.1568          | 0.2863          | 0.2522           | 0.3407        |
| 1.4355        | 0.56  | 350  | 2.1613          | 0.2818          | 0.2467           | 0.3384        |
| 1.4587        | 0.58  | 360  | 2.1394          | 0.278           | 0.2441           | 0.3319        |
| 1.4674        | 0.59  | 370  | 2.1284          | 0.2818          | 0.2503           | 0.3317        |
| 1.3842        | 0.61  | 380  | 2.1286          | 0.2828          | 0.2521           | 0.3311        |
| 1.456         | 0.62  | 390  | 2.1013          | 0.2806          | 0.2513           | 0.3268        |
| 1.3512        | 0.64  | 400  | 2.1385          | 0.2817          | 0.2463           | 0.3384        |
| 1.3697        | 0.66  | 410  | 2.1282          | 0.2779          | 0.2451           | 0.3303        |
| 1.4404        | 0.67  | 420  | 2.1087          | 0.2837          | 0.2498           | 0.3376        |
| 1.4956        | 0.69  | 430  | 2.1034          | 0.2773          | 0.2446           | 0.3293        |
| 1.5532        | 0.7   | 440  | 2.1103          | 0.2834          | 0.2471           | 0.342         |
| 1.3254        | 0.72  | 450  | 2.1337          | 0.2811          | 0.2466           | 0.3363        |
| 1.4318        | 0.74  | 460  | 2.0690          | 0.287           | 0.2548           | 0.3382        |
| 1.3766        | 0.75  | 470  | 2.1392          | 0.2837          | 0.2438           | 0.3487        |
| 1.318         | 0.77  | 480  | 2.1353          | 0.2864          | 0.2492           | 0.3465        |
| 1.343         | 0.78  | 490  | 2.1262          | 0.282           | 0.2478           | 0.3375        |
| 1.3887        | 0.8   | 500  | 2.1246          | 0.2904          | 0.2552           | 0.3471        |
| 1.4047        | 0.82  | 510  | 2.0871          | 0.2835          | 0.2552           | 0.3285        |
| 1.459         | 0.83  | 520  | 2.1219          | 0.2824          | 0.247            | 0.3391        |
| 1.5071        | 0.85  | 530  | 2.0846          | 0.2799          | 0.2472           | 0.3321        |
| 1.3899        | 0.86  | 540  | 2.1019          | 0.2857          | 0.2505           | 0.3416        |
| 1.4277        | 0.88  | 550  | 2.0739          | 0.2831          | 0.2503           | 0.335         |
| 1.3651        | 0.9   | 560  | 2.0838          | 0.2788          | 0.2458           | 0.3312        |
| 1.4135        | 0.91  | 570  | 2.0783          | 0.2878          | 0.2547           | 0.3403        |
| 1.4165        | 0.93  | 580  | 2.0414          | 0.2848          | 0.2554           | 0.3311        |
| 1.3862        | 0.94  | 590  | 2.0631          | 0.2824          | 0.2497           | 0.3343        |
| 1.3839        | 0.96  | 600  | 2.0496          | 0.2778          | 0.2477           | 0.3252        |
| 1.3881        | 0.98  | 610  | 2.0682          | 0.2836          | 0.2492           | 0.3387        |
| 1.4462        | 0.99  | 620  | 2.0703          | 0.2824          | 0.2492           | 0.3355        |
| 1.3299        | 1.01  | 630  | 2.0345          | 0.2932          | 0.2607           | 0.3444        |
| 1.2978        | 1.02  | 640  | 2.0593          | 0.2868          | 0.2524           | 0.3416        |
| 1.2671        | 1.04  | 650  | 2.0719          | 0.285           | 0.2519           | 0.3382        |
| 1.2823        | 1.06  | 660  | 2.0529          | 0.2838          | 0.2542           | 0.3307        |
| 1.2797        | 1.07  | 670  | 2.0745          | 0.2867          | 0.2518           | 0.3424        |
| 1.3185        | 1.09  | 680  | 2.0725          | 0.2795          | 0.2465           | 0.3324        |
| 1.3278        | 1.1   | 690  | 2.0492          | 0.2881          | 0.2541           | 0.3422        |
| 1.3344        | 1.12  | 700  | 2.0541          | 0.2869          | 0.251            | 0.3442        |
| 1.2929        | 1.14  | 710  | 2.0588          | 0.2834          | 0.2487           | 0.339         |
| 1.2152        | 1.15  | 720  | 2.0515          | 0.2884          | 0.254            | 0.3427        |
| 1.2662        | 1.17  | 730  | 2.0552          | 0.2777          | 0.2452           | 0.3295        |
| 1.2999        | 1.18  | 740  | 2.0337          | 0.2823          | 0.2512           | 0.3314        |
| 1.327         | 1.2   | 750  | 2.0363          | 0.2843          | 0.2486           | 0.3416        |
| 1.2588        | 1.22  | 760  | 2.0550          | 0.2889          | 0.2549           | 0.3437        |
| 1.2676        | 1.23  | 770  | 2.0146          | 0.2819          | 0.2483           | 0.3351        |
| 1.3498        | 1.25  | 780  | 2.0291          | 0.2812          | 0.2475           | 0.3348        |
| 1.3004        | 1.26  | 790  | 2.0501          | 0.2836          | 0.2484           | 0.3398        |
| 1.2663        | 1.28  | 800  | 2.0205          | 0.2838          | 0.2505           | 0.3364        |
| 1.2419        | 1.3   | 810  | 2.0274          | 0.2837          | 0.2511           | 0.3353        |
| 1.3022        | 1.31  | 820  | 2.0277          | 0.2838          | 0.2496           | 0.3376        |
| 1.3404        | 1.33  | 830  | 2.0417          | 0.2835          | 0.2491           | 0.3384        |
| 1.2684        | 1.34  | 840  | 2.0474          | 0.2841          | 0.2485           | 0.3412        |
| 1.3272        | 1.36  | 850  | 2.0520          | 0.2889          | 0.254            | 0.3444        |
| 1.2133        | 1.38  | 860  | 2.0172          | 0.2848          | 0.2521           | 0.3367        |
| 1.2698        | 1.39  | 870  | 2.0031          | 0.2864          | 0.2536           | 0.3382        |
| 1.1931        | 1.41  | 880  | 2.0211          | 0.2832          | 0.2507           | 0.3349        |
| 1.2514        | 1.42  | 890  | 2.0176          | 0.2904          | 0.2551           | 0.3464        |
| 1.3456        | 1.44  | 900  | 1.9953          | 0.2769          | 0.243            | 0.3308        |
| 1.2857        | 1.46  | 910  | 1.9979          | 0.2891          | 0.2542           | 0.3447        |
| 1.2462        | 1.47  | 920  | 2.0191          | 0.2842          | 0.2485           | 0.3414        |
| 1.2398        | 1.49  | 930  | 1.9891          | 0.2854          | 0.253            | 0.3368        |
| 1.2982        | 1.5   | 940  | 1.9947          | 0.2894          | 0.2566           | 0.3416        |
| 1.2334        | 1.52  | 950  | 1.9975          | 0.287           | 0.2512           | 0.3444        |
| 1.2572        | 1.54  | 960  | 1.9977          | 0.288           | 0.2539           | 0.3424        |
| 1.1719        | 1.55  | 970  | 2.0149          | 0.2857          | 0.2514           | 0.3404        |
| 1.1536        | 1.57  | 980  | 1.9899          | 0.2905          | 0.2558           | 0.3456        |
| 1.2731        | 1.58  | 990  | 1.9915          | 0.2864          | 0.2528           | 0.3401        |
| 1.2694        | 1.6   | 1000 | 2.0039          | 0.2872          | 0.252            | 0.3434        |
| 1.2481        | 1.62  | 1010 | 2.0071          | 0.2806          | 0.2467           | 0.335         |
| 1.1825        | 1.63  | 1020 | 1.9943          | 0.2862          | 0.2514           | 0.3417        |
| 1.2626        | 1.65  | 1030 | 1.9953          | 0.2837          | 0.2489           | 0.3396        |
| 1.2497        | 1.66  | 1040 | 2.0078          | 0.287           | 0.2511           | 0.3444        |
| 1.2401        | 1.68  | 1050 | 1.9765          | 0.2853          | 0.2501           | 0.3422        |
| 1.2074        | 1.7   | 1060 | 1.9970          | 0.2863          | 0.252            | 0.3412        |
| 1.1557        | 1.71  | 1070 | 1.9860          | 0.2841          | 0.2492           | 0.34          |
| 1.1298        | 1.73  | 1080 | 1.9895          | 0.2862          | 0.2525           | 0.3397        |
| 1.1531        | 1.74  | 1090 | 1.9813          | 0.2892          | 0.2546           | 0.3442        |
| 1.2627        | 1.76  | 1100 | 1.9906          | 0.2863          | 0.2522           | 0.3407        |
| 1.2191        | 1.78  | 1110 | 1.9853          | 0.2879          | 0.253            | 0.3435        |
| 1.2415        | 1.79  | 1120 | 1.9927          | 0.288           | 0.2541           | 0.342         |
| 1.2879        | 1.81  | 1130 | 1.9839          | 0.2831          | 0.2491           | 0.3378        |
| 1.2328        | 1.82  | 1140 | 1.9698          | 0.2852          | 0.2526           | 0.337         |
| 1.2562        | 1.84  | 1150 | 1.9953          | 0.2866          | 0.2504           | 0.3444        |
| 1.3071        | 1.86  | 1160 | 1.9698          | 0.2847          | 0.2507           | 0.3389        |
| 1.2276        | 1.87  | 1170 | 1.9787          | 0.2863          | 0.2517           | 0.3415        |
| 1.1608        | 1.89  | 1180 | 1.9944          | 0.289           | 0.2528           | 0.3469        |
| 1.3046        | 1.9   | 1190 | 1.9750          | 0.2863          | 0.2516           | 0.3414        |
| 1.2468        | 1.92  | 1200 | 1.9742          | 0.2841          | 0.2503           | 0.338         |
| 1.2839        | 1.94  | 1210 | 1.9812          | 0.2841          | 0.2497           | 0.3392        |
| 1.2117        | 1.95  | 1220 | 1.9755          | 0.2847          | 0.2507           | 0.3384        |
| 1.2055        | 1.97  | 1230 | 1.9759          | 0.2855          | 0.2516           | 0.3393        |
| 1.2356        | 1.98  | 1240 | 1.9747          | 0.286           | 0.2522           | 0.3399        |
| 1.1865        | 2.0   | 1250 | 1.9753          | 0.2863          | 0.2523           | 0.3404        |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.1+cu121
- Datasets 2.14.5
- Tokenizers 0.15.1
