---
base_model: google/pegasus-large
tags:
- generated_from_trainer
metrics:
- rouge
- bleu
model-index:
- name: PhysicalSciencePegasusLargeModel
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# PhysicalSciencePegasusLargeModel

This model is a fine-tuned version of [google/pegasus-large](https://huggingface.co/google/pegasus-large) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 5.4857
- Rouge1: 45.4096
- Rouge2: 14.1811
- Rougel: 31.2204
- Rougelsum: 40.9948
- Bertscore Precision: 78.8464
- Bertscore Recall: 82.0345
- Bertscore F1: 80.4009
- Bleu: 0.0934
- Gen Len: 192.7149

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 16
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 1

### Training results

| Training Loss | Epoch  | Step | Validation Loss | Rouge1  | Rouge2  | Rougel  | Rougelsum | Bertscore Precision | Bertscore Recall | Bertscore F1 | Bleu   | Gen Len  |
|:-------------:|:------:|:----:|:---------------:|:-------:|:-------:|:-------:|:---------:|:-------------------:|:----------------:|:------------:|:------:|:--------:|
| 6.8224        | 0.0620 | 100  | 6.4676          | 35.0969 | 8.6135  | 23.5193 | 31.2379   | 75.6321             | 79.8224          | 77.6611      | 0.0532 | 192.7149 |
| 6.3975        | 0.1239 | 200  | 6.1277          | 38.4317 | 11.0631 | 27.2821 | 34.6261   | 76.5444             | 80.6289          | 78.5221      | 0.0704 | 192.7149 |
| 6.1059        | 0.1859 | 300  | 6.0085          | 39.425  | 11.6567 | 27.8193 | 35.5195   | 76.8342             | 80.9269          | 78.8164      | 0.0749 | 192.7149 |
| 6.1215        | 0.2478 | 400  | 5.9209          | 40.7923 | 12.1231 | 28.4792 | 36.7207   | 77.1433             | 81.1253          | 79.0738      | 0.0786 | 192.7149 |
| 6.0244        | 0.3098 | 500  | 5.8054          | 41.3668 | 12.6049 | 28.8963 | 37.3873   | 77.1495             | 81.267           | 79.1445      | 0.0830 | 192.7149 |
| 5.9814        | 0.3717 | 600  | 5.7414          | 43.1237 | 13.2946 | 29.7215 | 38.645    | 77.6801             | 81.5336          | 79.5507      | 0.0875 | 192.7149 |
| 5.9421        | 0.4337 | 700  | 5.6719          | 43.8248 | 13.4876 | 30.0523 | 39.3496   | 77.8832             | 81.6572          | 79.7164      | 0.0887 | 192.7149 |
| 5.7744        | 0.4957 | 800  | 5.6277          | 44.5131 | 13.6831 | 30.4494 | 39.9332   | 78.1354             | 81.7356          | 79.8859      | 0.0896 | 192.7149 |
| 5.7227        | 0.5576 | 900  | 5.5916          | 44.8391 | 13.8057 | 30.6822 | 40.4155   | 78.4064             | 81.8258          | 80.0711      | 0.0902 | 192.7149 |
| 5.7532        | 0.6196 | 1000 | 5.5600          | 44.5759 | 13.8594 | 30.7325 | 40.1139   | 78.3708             | 81.8401          | 80.0593      | 0.0909 | 192.7149 |
| 5.7156        | 0.6815 | 1100 | 5.5421          | 45.2882 | 13.9686 | 31.0389 | 40.7447   | 78.6902             | 81.9492          | 80.2784      | 0.0916 | 192.7149 |
| 5.6819        | 0.7435 | 1200 | 5.5229          | 45.3689 | 14.1424 | 31.0474 | 40.781    | 78.6599             | 81.9628          | 80.269       | 0.0925 | 192.7149 |
| 5.7394        | 0.8055 | 1300 | 5.5060          | 45.4608 | 14.1555 | 31.1182 | 40.9486   | 78.7697             | 82.0031          | 80.3458      | 0.0930 | 192.7149 |
| 5.7374        | 0.8674 | 1400 | 5.4912          | 44.9845 | 14.0026 | 31.0462 | 40.6411   | 78.6797             | 81.9501          | 80.2733      | 0.0919 | 192.7149 |
| 5.6422        | 0.9294 | 1500 | 5.4874          | 45.5215 | 14.1925 | 31.2402 | 41.0589   | 78.8731             | 82.0393          | 80.4171      | 0.0934 | 192.7149 |
| 5.6744        | 0.9913 | 1600 | 5.4857          | 45.4096 | 14.1811 | 31.2204 | 40.9948   | 78.8464             | 82.0345          | 80.4009      | 0.0934 | 192.7149 |


### Framework versions

- Transformers 4.41.2
- Pytorch 2.3.1+cu121
- Datasets 2.19.2
- Tokenizers 0.19.1