transformers pytorch pegasus text2text-generation generated_from_trainer dataset:yawnick/QMSum autotrain_compatible endpoints_compatible region:us
---
datasets:
- yawnick/QMSum
metrics:
- rouge
tags:
- generated_from_trainer
model-index:
- name: pegasus-qmsum-meeting-summarization
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# pegasus-qmsum-meeting-summarization

This model is a fine-tuned version of [google/pegasus-xsum](https://huggingface.co/google/pegasus-xsum) on the QMSum dataset.
It achieves the following results on the evaluation set:
- Loss: 4.2331
- Rouge1: 32.7156
- Rouge2: 10.5699
- Rougel: 23.2759
- Rougelsum: 29.7903
- Gen Len: 61.65

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-06
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 300
- label_smoothing_factor: 0.1

### Training results

| Training Loss | Epoch  | Step  | Validation Loss | Rouge1  | Rouge2  | Rougel  | Rougelsum | Gen Len |
|:-------------:|:------:|:-----:|:---------------:|:-------:|:-------:|:-------:|:---------:|:-------:|
| 5.5746        | 1.09   | 100   | 5.1739          | 9.4941  | 1.7868  | 7.2455  | 8.4302    | 29.825  |
| 5.5784        | 2.17   | 200   | 5.0939          | 9.113   | 1.7887  | 6.9741  | 8.0457    | 26.85   |
| 5.3777        | 3.26   | 300   | 4.9723          | 9.6387  | 1.9301  | 7.349   | 8.7941    | 25.325  |
| 5.1884        | 4.35   | 400   | 4.8423          | 10.6045 | 2.4008  | 7.8423  | 9.4593    | 22.625  |
| 5.0795        | 5.43   | 500   | 4.7313          | 13.7621 | 3.1231  | 9.6944  | 12.2204   | 32.175  |
| 4.9369        | 6.52   | 600   | 4.6555          | 19.5696 | 4.9121  | 14.2603 | 16.9622   | 46.45   |
| 4.8926        | 7.61   | 700   | 4.6038          | 22.8411 | 5.9791  | 17.2227 | 20.1173   | 51.825  |
| 4.7502        | 8.7    | 800   | 4.5659          | 24.0555 | 6.1971  | 18.967  | 20.9143   | 54.25   |
| 4.6876        | 9.78   | 900   | 4.5379          | 24.7066 | 6.0317  | 19.542  | 21.5774   | 57.575  |
| 4.6266        | 10.87  | 1000  | 4.5160          | 26.128  | 6.5089  | 20.5573 | 22.5338   | 58.0    |
| 4.6303        | 11.96  | 1100  | 4.4983          | 26.6639 | 7.1208  | 20.5222 | 23.5783   | 57.925  |
| 4.6263        | 13.04  | 1200  | 4.4815          | 26.8262 | 7.1029  | 20.5172 | 23.6216   | 57.575  |
| 4.577         | 14.13  | 1300  | 4.4667          | 27.7952 | 7.8331  | 21.1111 | 24.6086   | 56.95   |
| 4.5797        | 15.22  | 1400  | 4.4559          | 27.728  | 7.8144  | 21.1519 | 24.4858   | 56.6    |
| 4.4923        | 16.3   | 1500  | 4.4448          | 28.0998 | 8.1346  | 21.4004 | 25.3769   | 55.975  |
| 4.4583        | 17.39  | 1600  | 4.4335          | 28.9003 | 8.6135  | 22.0139 | 26.0409   | 56.55   |
| 4.5036        | 18.48  | 1700  | 4.4246          | 29.2187 | 8.8301  | 22.3569 | 26.1964   | 58.125  |
| 4.4383        | 19.57  | 1800  | 4.4144          | 28.8424 | 8.9131  | 22.0398 | 25.9214   | 56.75   |
| 4.4797        | 20.65  | 1900  | 4.4054          | 28.9285 | 8.9298  | 22.222  | 26.0316   | 56.225  |
| 4.4264        | 21.74  | 2000  | 4.3989          | 29.7184 | 9.0477  | 22.2885 | 26.7439   | 56.225  |
| 4.3615        | 22.83  | 2100  | 4.3902          | 29.1538 | 8.9529  | 22.0076 | 26.4925   | 57.175  |
| 4.329         | 23.91  | 2200  | 4.3839          | 29.5186 | 9.2777  | 21.9025 | 26.3141   | 55.5    |
| 4.3578        | 25.0   | 2300  | 4.3766          | 28.4309 | 8.9423  | 21.0945 | 25.8191   | 53.975  |
| 4.3748        | 26.09  | 2400  | 4.3707          | 28.3    | 9.0625  | 21.4946 | 25.1966   | 53.0    |
| 4.3233        | 27.17  | 2500  | 4.3639          | 28.2325 | 8.9889  | 21.6226 | 25.3677   | 54.6    |
| 4.339         | 28.26  | 2600  | 4.3578          | 28.0744 | 8.774   | 21.2509 | 25.2901   | 54.1    |
| 4.2798        | 29.35  | 2700  | 4.3532          | 27.772  | 8.7096  | 21.1687 | 25.3345   | 54.025  |
| 4.2964        | 30.43  | 2800  | 4.3465          | 27.7827 | 8.1597  | 20.8139 | 25.0152   | 54.45   |
| 4.3365        | 31.52  | 2900  | 4.3423          | 28.2039 | 8.4661  | 21.3546 | 25.6381   | 55.5    |
| 4.2385        | 32.61  | 3000  | 4.3380          | 28.1098 | 8.6483  | 21.5279 | 25.2009   | 53.95   |
| 4.2451        | 33.7   | 3100  | 4.3331          | 28.2745 | 8.5024  | 21.4456 | 25.3363   | 52.6    |
| 4.2393        | 34.78  | 3200  | 4.3289          | 28.7597 | 9.0881  | 21.6532 | 25.8954   | 52.65   |
| 4.2116        | 35.87  | 3300  | 4.3252          | 29.0463 | 9.1218  | 21.8026 | 26.2037   | 53.65   |
| 4.2175        | 36.96  | 3400  | 4.3210          | 28.8009 | 9.0188  | 21.8368 | 25.8678   | 52.85   |
| 4.2071        | 38.04  | 3500  | 4.3169          | 28.9313 | 8.9787  | 21.3554 | 26.0628   | 54.325  |
| 4.1775        | 39.13  | 3600  | 4.3132          | 28.837  | 8.9621  | 21.6342 | 26.0569   | 54.025  |
| 4.1962        | 40.22  | 3700  | 4.3086          | 28.9265 | 9.0701  | 21.588  | 26.0702   | 53.075  |
| 4.1452        | 41.3   | 3800  | 4.3060          | 29.7968 | 9.366   | 22.1712 | 26.8461   | 54.925  |
| 4.1912        | 42.39  | 3900  | 4.3018          | 29.1488 | 9.1631  | 21.6566 | 26.1476   | 54.25   |
| 4.1356        | 43.48  | 4000  | 4.2984          | 30.0138 | 9.2456  | 22.2547 | 27.2714   | 54.85   |
| 4.1272        | 44.57  | 4100  | 4.2949          | 29.8858 | 9.1498  | 22.1221 | 27.0798   | 55.65   |
| 4.1174        | 45.65  | 4200  | 4.2895          | 30.0427 | 9.2297  | 22.2602 | 27.4219   | 56.175  |
| 4.1029        | 46.74  | 4300  | 4.2885          | 29.9443 | 9.4293  | 22.1229 | 27.3496   | 56.45   |
| 4.157         | 47.83  | 4400  | 4.2851          | 30.3693 | 9.406   | 22.471  | 27.7511   | 56.775  |
| 4.1105        | 48.91  | 4500  | 4.2827          | 30.6193 | 9.7082  | 22.6169 | 27.8044   | 57.225  |
| 4.083         | 50.0   | 4600  | 4.2796          | 30.8083 | 9.9211  | 22.5228 | 28.1236   | 57.575  |
| 4.0891        | 51.09  | 4700  | 4.2764          | 30.4201 | 9.6192  | 22.4747 | 27.7514   | 57.475  |
| 4.0603        | 52.17  | 4800  | 4.2741          | 30.7777 | 9.7432  | 22.6705 | 27.5956   | 57.1    |
| 4.0472        | 53.26  | 4900  | 4.2731          | 30.8093 | 9.7916  | 22.5533 | 27.7858   | 56.15   |
| 4.0712        | 54.35  | 5000  | 4.2703          | 29.9667 | 9.5645  | 22.113  | 26.647    | 56.525  |
| 4.0658        | 55.43  | 5100  | 4.2674          | 29.5415 | 9.4291  | 21.6862 | 26.7816   | 56.55   |
| 4.059         | 56.52  | 5200  | 4.2659          | 30.2032 | 9.8875  | 22.2539 | 27.1801   | 56.925  |
| 4.0257        | 57.61  | 5300  | 4.2629          | 30.3181 | 9.8187  | 22.4266 | 27.4318   | 56.925  |
| 4.0002        | 58.7   | 5400  | 4.2608          | 29.6641 | 9.9252  | 22.1725 | 27.0764   | 56.6    |
| 4.0978        | 59.78  | 5500  | 4.2591          | 30.653  | 10.087  | 22.6956 | 27.7481   | 56.25   |
| 3.9978        | 60.87  | 5600  | 4.2568          | 29.5473 | 9.5653  | 21.6367 | 26.391    | 55.825  |
| 3.9832        | 61.96  | 5700  | 4.2552          | 30.6368 | 10.1624 | 22.7204 | 27.5866   | 57.425  |
| 3.9841        | 63.04  | 5800  | 4.2525          | 30.3045 | 9.7966  | 22.2939 | 27.0978   | 57.725  |
| 4.002         | 64.13  | 5900  | 4.2507          | 30.4468 | 9.9323  | 22.6572 | 27.0761   | 57.5    |
| 3.9705        | 65.22  | 6000  | 4.2491          | 30.1218 | 9.6921  | 22.465  | 26.3835   | 57.55   |
| 3.9863        | 66.3   | 6100  | 4.2477          | 31.3982 | 9.9901  | 22.8762 | 27.6169   | 58.975  |
| 3.9308        | 67.39  | 6200  | 4.2454          | 30.2673 | 9.5804  | 22.4474 | 26.6111   | 59.2    |
| 3.9794        | 68.48  | 6300  | 4.2449          | 30.8612 | 9.8254  | 22.8444 | 27.4979   | 58.075  |
| 3.9499        | 69.57  | 6400  | 4.2412          | 30.8366 | 9.7     | 22.4469 | 27.1621   | 59.025  |
| 3.9722        | 70.65  | 6500  | 4.2414          | 30.9625 | 9.8251  | 22.4089 | 27.4342   | 59.1    |
| 3.9125        | 71.74  | 6600  | 4.2394          | 30.5777 | 9.5514  | 22.1581 | 26.8665   | 58.75   |
| 3.9184        | 72.83  | 6700  | 4.2396          | 30.8306 | 9.5469  | 22.6571 | 27.4302   | 59.725  |
| 3.9337        | 73.91  | 6800  | 4.2377          | 30.8688 | 9.6733  | 22.3073 | 27.2943   | 58.975  |
| 3.9145        | 75.0   | 6900  | 4.2358          | 30.467  | 9.6393  | 22.225  | 27.0127   | 58.45   |
| 3.9038        | 76.09  | 7000  | 4.2353          | 30.6344 | 9.3676  | 22.1945 | 27.1871   | 59.275  |
| 3.893         | 77.17  | 7100  | 4.2335          | 31.4486 | 9.8839  | 22.735  | 27.7854   | 59.025  |
| 3.885         | 78.26  | 7200  | 4.2318          | 30.7118 | 9.8568  | 22.2546 | 27.3983   | 58.5    |
| 3.9266        | 79.35  | 7300  | 4.2304          | 31.6171 | 9.8817  | 22.6145 | 27.6888   | 59.25   |
| 3.8826        | 80.43  | 7400  | 4.2299          | 31.0976 | 9.4662  | 22.2285 | 27.817    | 58.95   |
| 3.8775        | 81.52  | 7500  | 4.2286          | 31.1379 | 10.0975 | 22.5686 | 27.883    | 59.8    |
| 3.8455        | 82.61  | 7600  | 4.2292          | 32.076  | 10.0214 | 22.8866 | 28.3828   | 59.15   |
| 3.8838        | 83.7   | 7700  | 4.2269          | 31.5696 | 9.7812  | 22.7619 | 28.2236   | 58.6    |
| 3.8425        | 84.78  | 7800  | 4.2266          | 31.1731 | 9.97    | 22.4203 | 27.4956   | 59.1    |
| 3.8766        | 85.87  | 7900  | 4.2260          | 32.3221 | 10.6243 | 23.079  | 28.9008   | 58.45   |
| 3.8217        | 86.96  | 8000  | 4.2258          | 31.9956 | 10.4201 | 23.083  | 28.4945   | 58.5    |
| 3.8319        | 88.04  | 8100  | 4.2245          | 32.0272 | 10.4673 | 23.3471 | 28.9845   | 58.35   |
| 3.8283        | 89.13  | 8200  | 4.2231          | 32.2943 | 10.2594 | 23.1819 | 29.1345   | 60.5    |
| 3.8394        | 90.22  | 8300  | 4.2221          | 31.3976 | 10.3085 | 22.6581 | 28.2494   | 59.25   |
| 3.8258        | 91.3   | 8400  | 4.2203          | 31.4433 | 10.1184 | 22.672  | 28.1236   | 58.85   |
| 3.7981        | 92.39  | 8500  | 4.2205          | 31.1313 | 10.0056 | 22.677  | 27.7409   | 59.075  |
| 3.8349        | 93.48  | 8600  | 4.2215          | 31.5779 | 10.0303 | 22.6155 | 28.0566   | 59.2    |
| 3.8225        | 94.57  | 8700  | 4.2201          | 31.9646 | 10.0643 | 22.7808 | 28.67     | 58.925  |
| 3.8145        | 95.65  | 8800  | 4.2193          | 32.0347 | 10.5103 | 23.095  | 28.6056   | 57.225  |
| 3.7771        | 96.74  | 8900  | 4.2180          | 30.8138 | 9.602   | 22.2649 | 27.7948   | 57.875  |
| 3.823         | 97.83  | 9000  | 4.2168          | 31.3785 | 9.7046  | 22.3877 | 28.2578   | 58.675  |
| 3.7701        | 98.91  | 9100  | 4.2169          | 31.4511 | 9.9183  | 22.6645 | 28.1932   | 59.0    |
| 3.773         | 100.0  | 9200  | 4.2169          | 31.7392 | 9.9669  | 22.5894 | 28.218    | 58.15   |
| 3.7661        | 101.09 | 9300  | 4.2161          | 31.5507 | 9.8992  | 22.4602 | 28.3357   | 58.375  |
| 3.7875        | 102.17 | 9400  | 4.2163          | 31.5145 | 9.5173  | 22.321  | 27.8613   | 58.375  |
| 3.7659        | 103.26 | 9500  | 4.2152          | 31.2967 | 9.8797  | 22.6247 | 28.0317   | 57.925  |
| 3.7576        | 104.35 | 9600  | 4.2139          | 31.5739 | 9.8376  | 22.7561 | 28.2318   | 58.4    |
| 3.7784        | 105.43 | 9700  | 4.2144          | 32.2269 | 10.2299 | 22.6582 | 28.6249   | 58.425  |
| 3.7356        | 106.52 | 9800  | 4.2139          | 32.3031 | 10.1505 | 22.7079 | 28.9052   | 58.475  |
| 3.7799        | 107.61 | 9900  | 4.2124          | 31.1334 | 9.1481  | 22.1297 | 27.5951   | 58.6    |
| 3.7269        | 108.7  | 10000 | 4.2122          | 31.6957 | 9.2874  | 22.4867 | 28.225    | 58.4    |
| 3.719         | 109.78 | 10100 | 4.2108          | 31.477  | 10.0245 | 22.4703 | 28.1316   | 58.075  |
| 3.7411        | 110.87 | 10200 | 4.2112          | 31.4165 | 9.9791  | 22.4396 | 28.3068   | 58.275  |
| 3.7135        | 111.96 | 10300 | 4.2122          | 31.4924 | 9.9864  | 22.496  | 28.2414   | 57.8    |
| 3.7317        | 113.04 | 10400 | 4.2120          | 31.6599 | 10.1605 | 22.5322 | 28.3045   | 59.075  |
| 3.7113        | 114.13 | 10500 | 4.2127          | 31.6814 | 10.106  | 22.4311 | 28.5808   | 59.5    |
| 3.7063        | 115.22 | 10600 | 4.2132          | 31.2448 | 10.0006 | 22.5549 | 28.4686   | 57.775  |
| 3.681         | 116.3  | 10700 | 4.2123          | 31.1739 | 10.0533 | 22.2954 | 28.0822   | 58.35   |
| 3.7369        | 117.39 | 10800 | 4.2118          | 31.8541 | 10.1452 | 22.7607 | 28.9501   | 58.8    |
| 3.6645        | 118.48 | 10900 | 4.2122          | 31.7128 | 9.8554  | 22.4464 | 28.5888   | 58.375  |
| 3.6766        | 119.57 | 11000 | 4.2118          | 31.1492 | 9.8058  | 22.0978 | 28.1827   | 58.725  |
| 3.6915        | 120.65 | 11100 | 4.2110          | 31.1679 | 9.5755  | 22.1391 | 28.0886   | 58.375  |
| 3.6702        | 121.74 | 11200 | 4.2129          | 31.0682 | 9.7375  | 22.0118 | 28.2189   | 59.15   |
| 3.6946        | 122.83 | 11300 | 4.2118          | 31.6134 | 9.5918  | 22.2506 | 28.5343   | 59.175  |
| 3.6713        | 123.91 | 11400 | 4.2110          | 31.3585 | 9.4211  | 22.1884 | 27.8744   | 59.05   |
| 3.6694        | 125.0  | 11500 | 4.2126          | 32.0058 | 9.6453  | 22.3911 | 28.6928   | 59.55   |
| 3.6585        | 126.09 | 11600 | 4.2123          | 31.7679 | 9.7101  | 22.2378 | 28.4985   | 59.2    |
| 3.6857        | 127.17 | 11700 | 4.2118          | 31.7766 | 10.0375 | 22.5097 | 28.8104   | 59.6    |
| 3.6338        | 128.26 | 11800 | 4.2126          | 32.2508 | 10.2617 | 22.6745 | 29.0714   | 59.075  |
| 3.6412        | 129.35 | 11900 | 4.2135          | 32.0515 | 10.0905 | 22.7015 | 29.0028   | 58.9    |
| 3.6594        | 130.43 | 12000 | 4.2122          | 32.7784 | 10.351  | 23.0969 | 29.6672   | 59.525  |
| 3.6571        | 131.52 | 12100 | 4.2120          | 32.3165 | 10.329  | 22.8445 | 29.2886   | 59.5    |
| 3.6002        | 132.61 | 12200 | 4.2120          | 32.5553 | 10.0875 | 22.6064 | 29.1046   | 59.425  |
| 3.6621        | 133.7  | 12300 | 4.2126          | 31.7637 | 9.9785  | 22.5716 | 28.7173   | 59.275  |
| 3.6651        | 134.78 | 12400 | 4.2122          | 31.7568 | 9.7503  | 22.3876 | 28.6015   | 59.6    |
| 3.6127        | 135.87 | 12500 | 4.2123          | 31.5708 | 9.5203  | 21.9951 | 28.2082   | 58.75   |
| 3.6544        | 136.96 | 12600 | 4.2124          | 32.0767 | 9.8955  | 22.2724 | 28.4755   | 59.5    |
| 3.5994        | 138.04 | 12700 | 4.2125          | 31.8523 | 9.9159  | 22.2978 | 28.8159   | 59.175  |
| 3.6174        | 139.13 | 12800 | 4.2114          | 32.2165 | 9.784   | 22.4377 | 28.5603   | 59.1    |
| 3.6122        | 140.22 | 12900 | 4.2115          | 32.0247 | 9.6881  | 22.3116 | 28.61     | 58.9    |
| 3.6174        | 141.3  | 13000 | 4.2116          | 31.9549 | 9.5924  | 22.3997 | 28.9145   | 59.15   |
| 3.5965        | 142.39 | 13100 | 4.2113          | 32.6173 | 10.4241 | 22.8644 | 29.3928   | 60.9    |
| 3.6076        | 143.48 | 13200 | 4.2112          | 33.0058 | 10.6417 | 23.0297 | 29.8375   | 61.0    |
| 3.6013        | 144.57 | 13300 | 4.2105          | 33.005  | 10.5398 | 22.9758 | 29.7266   | 60.325  |
| 3.6181        | 145.65 | 13400 | 4.2117          | 31.0558 | 9.4714  | 21.9025 | 27.9627   | 60.025  |
| 3.6288        | 146.74 | 13500 | 4.2107          | 32.7196 | 10.4991 | 22.9182 | 29.6586   | 60.25   |
| 3.5879        | 147.83 | 13600 | 4.2091          | 32.6755 | 10.3936 | 22.9559 | 29.5314   | 60.425  |
| 3.591         | 148.91 | 13700 | 4.2101          | 33.2956 | 10.6616 | 22.8509 | 29.5237   | 60.4    |
| 3.5658        | 150.0  | 13800 | 4.2116          | 33.4712 | 10.3725 | 23.1449 | 30.0987   | 60.2    |
| 3.574         | 151.09 | 13900 | 4.2115          | 33.5427 | 10.5852 | 22.9671 | 29.8456   | 60.175  |
| 3.5795        | 152.17 | 14000 | 4.2115          | 33.4387 | 10.5744 | 23.4785 | 30.0494   | 60.15   |
| 3.5728        | 153.26 | 14100 | 4.2119          | 33.1244 | 10.0308 | 22.8377 | 29.7725   | 60.775  |
| 3.5441        | 154.35 | 14200 | 4.2121          | 32.9226 | 9.9625  | 22.9013 | 29.6004   | 59.7    |
| 3.5236        | 155.43 | 14300 | 4.2114          | 32.3717 | 9.9122  | 22.78   | 28.8305   | 59.725  |
| 3.5679        | 156.52 | 14400 | 4.2120          | 33.6347 | 10.7457 | 23.5191 | 30.1966   | 60.65   |
| 3.5574        | 157.61 | 14500 | 4.2119          | 33.4821 | 10.986  | 23.3567 | 30.1972   | 60.1    |
| 3.5935        | 158.7  | 14600 | 4.2115          | 32.7255 | 10.2639 | 23.1617 | 29.8065   | 60.35   |
| 3.5316        | 159.78 | 14700 | 4.2118          | 32.8033 | 10.0216 | 22.7099 | 29.3968   | 60.525  |
| 3.5618        | 160.87 | 14800 | 4.2118          | 32.6244 | 10.7228 | 22.8601 | 29.3613   | 60.8    |
| 3.545         | 161.96 | 14900 | 4.2132          | 32.6231 | 10.0711 | 22.4686 | 29.5341   | 59.675  |
| 3.5466        | 163.04 | 15000 | 4.2129          | 32.7601 | 10.3376 | 22.2373 | 29.3588   | 59.4    |
| 3.5594        | 164.13 | 15100 | 4.2127          | 32.4645 | 10.5106 | 22.6804 | 29.6229   | 60.375  |
| 3.4839        | 165.22 | 15200 | 4.2130          | 32.1799 | 10.0462 | 22.5474 | 29.1419   | 59.75   |
| 3.5492        | 166.3  | 15300 | 4.2133          | 32.6831 | 10.5307 | 22.8539 | 29.6406   | 59.875  |
| 3.5053        | 167.39 | 15400 | 4.2133          | 32.8614 | 10.0344 | 23.0577 | 29.5848   | 60.975  |
| 3.5427        | 168.48 | 15500 | 4.2140          | 32.7897 | 10.178  | 22.6287 | 29.4839   | 60.1    |
| 3.5495        | 169.57 | 15600 | 4.2126          | 33.1428 | 10.2866 | 22.9377 | 29.6883   | 60.525  |
| 3.5245        | 170.65 | 15700 | 4.2116          | 32.9892 | 10.1082 | 23.1528 | 29.576    | 60.675  |
| 3.5121        | 171.74 | 15800 | 4.2131          | 33.2677 | 10.5916 | 23.3002 | 29.8222   | 59.975  |
| 3.5559        | 172.83 | 15900 | 4.2126          | 32.5155 | 9.9557  | 22.6846 | 29.1171   | 60.85   |
| 3.4758        | 173.91 | 16000 | 4.2133          | 32.374  | 9.9127  | 22.4816 | 29.2839   | 60.9    |
| 3.5148        | 175.0  | 16100 | 4.2125          | 32.5611 | 9.8266  | 22.5993 | 28.9821   | 61.1    |
| 3.5093        | 176.09 | 16200 | 4.2132          | 32.1092 | 9.6761  | 22.3612 | 28.7771   | 60.05   |
| 3.5248        | 177.17 | 16300 | 4.2143          | 32.2696 | 9.6471  | 22.2791 | 28.9759   | 60.925  |
| 3.4807        | 178.26 | 16400 | 4.2139          | 31.9593 | 9.3878  | 22.0643 | 28.5392   | 61.3    |
| 3.5138        | 179.35 | 16500 | 4.2144          | 32.0284 | 9.8303  | 22.5724 | 29.0168   | 59.95   |
| 3.4834        | 180.43 | 16600 | 4.2153          | 32.3203 | 9.5741  | 22.4998 | 28.8014   | 60.5    |
| 3.4701        | 181.52 | 16700 | 4.2156          | 31.7243 | 9.544   | 22.1355 | 28.2238   | 61.275  |
| 3.5501        | 182.61 | 16800 | 4.2152          | 32.519  | 9.9372  | 22.3881 | 28.8347   | 61.45   |
| 3.4789        | 183.7  | 16900 | 4.2148          | 32.3324 | 9.7556  | 22.2474 | 28.7559   | 61.575  |
| 3.5172        | 184.78 | 17000 | 4.2156          | 32.161  | 9.4847  | 22.2358 | 28.8895   | 60.95   |
| 3.4681        | 185.87 | 17100 | 4.2167          | 32.6524 | 9.7116  | 22.8415 | 29.0798   | 60.575  |
| 3.4936        | 186.96 | 17200 | 4.2173          | 32.533  | 9.9478  | 22.7379 | 29.1301   | 61.575  |
| 3.4664        | 188.04 | 17300 | 4.2165          | 32.4549 | 10.1094 | 22.7097 | 28.7992   | 61.4    |
| 3.4599        | 189.13 | 17400 | 4.2164          | 32.6665 | 10.3463 | 22.7678 | 29.308    | 61.575  |
| 3.4724        | 190.22 | 17500 | 4.2175          | 32.4146 | 10.1782 | 22.7414 | 29.3546   | 60.75   |
| 3.4923        | 191.3  | 17600 | 4.2163          | 32.3624 | 9.8306  | 22.7311 | 28.7497   | 59.825  |
| 3.4771        | 192.39 | 17700 | 4.2161          | 33.1427 | 10.429  | 23.462  | 29.6967   | 60.35   |
| 3.4737        | 193.48 | 17800 | 4.2168          | 31.6894 | 9.7073  | 22.527  | 28.3711   | 60.65   |
| 3.4307        | 194.57 | 17900 | 4.2182          | 32.4769 | 10.1673 | 22.8356 | 29.4565   | 60.75   |
| 3.4843        | 195.65 | 18000 | 4.2168          | 32.5461 | 10.2855 | 22.8587 | 29.1242   | 60.825  |
| 3.4479        | 196.74 | 18100 | 4.2170          | 32.9284 | 10.2293 | 23.2679 | 29.8067   | 61.075  |
| 3.489         | 197.83 | 18200 | 4.2180          | 32.9561 | 10.481  | 23.2807 | 29.5499   | 61.25   |
| 3.4596        | 198.91 | 18300 | 4.2179          | 33.1418 | 10.2768 | 22.8762 | 30.0241   | 61.2    |
| 3.4552        | 200.0  | 18400 | 4.2171          | 33.5524 | 10.5969 | 23.5734 | 30.1587   | 61.525  |
| 3.4699        | 201.09 | 18500 | 4.2176          | 33.1941 | 10.3296 | 23.1962 | 30.1624   | 61.45   |
| 3.4281        | 202.17 | 18600 | 4.2187          | 33.3715 | 10.1919 | 23.1843 | 30.3192   | 61.55   |
| 3.4561        | 203.26 | 18700 | 4.2186          | 32.5288 | 9.9299  | 22.6515 | 29.2853   | 61.575  |
| 3.446         | 204.35 | 18800 | 4.2188          | 33.4268 | 10.7152 | 23.6525 | 30.4668   | 61.575  |
| 3.4259        | 205.43 | 18900 | 4.2189          | 33.1715 | 10.198  | 22.9264 | 29.8387   | 61.25   |
| 3.4497        | 206.52 | 19000 | 4.2192          | 33.3472 | 10.5372 | 23.0833 | 30.2925   | 61.25   |
| 3.4674        | 207.61 | 19100 | 4.2192          | 32.7581 | 10.2502 | 23.0554 | 29.6639   | 61.175  |
| 3.4521        | 208.7  | 19200 | 4.2186          | 33.7883 | 10.8639 | 23.4038 | 30.6114   | 61.475  |
| 3.443         | 209.78 | 19300 | 4.2194          | 33.029  | 10.6622 | 22.9009 | 29.9762   | 61.675  |
| 3.4356        | 210.87 | 19400 | 4.2199          | 32.7229 | 9.9204  | 22.5445 | 29.5517   | 61.3    |
| 3.4198        | 211.96 | 19500 | 4.2208          | 33.5216 | 10.3836 | 22.9423 | 29.9006   | 61.625  |
| 3.4417        | 213.04 | 19600 | 4.2210          | 32.7772 | 10.3206 | 22.9031 | 29.3774   | 61.625  |
| 3.4348        | 214.13 | 19700 | 4.2214          | 31.9959 | 10.0821 | 22.2012 | 28.6722   | 61.375  |
| 3.4528        | 215.22 | 19800 | 4.2213          | 32.5434 | 10.2807 | 22.6512 | 29.1705   | 61.65   |
| 3.3955        | 216.3  | 19900 | 4.2220          | 32.9148 | 10.5869 | 22.8107 | 29.4975   | 61.675  |
| 3.4437        | 217.39 | 20000 | 4.2227          | 32.8879 | 10.4334 | 22.6863 | 29.6794   | 61.125  |
| 3.4374        | 218.48 | 20100 | 4.2225          | 32.1453 | 9.9115  | 22.2936 | 28.9428   | 61.1    |
| 3.429         | 219.57 | 20200 | 4.2230          | 33.0805 | 10.5792 | 22.9417 | 29.9572   | 61.55   |
| 3.4089        | 220.65 | 20300 | 4.2239          | 32.0499 | 10.1613 | 22.6264 | 28.9217   | 61.65   |
| 3.418         | 221.74 | 20400 | 4.2237          | 32.6069 | 10.5032 | 22.8024 | 29.5804   | 61.275  |
| 3.4274        | 222.83 | 20500 | 4.2235          | 31.8624 | 10.2513 | 22.2816 | 28.8234   | 61.2    |
| 3.4156        | 223.91 | 20600 | 4.2242          | 32.2666 | 10.4604 | 22.5607 | 29.0666   | 61.025  |
| 3.4135        | 225.0  | 20700 | 4.2247          | 31.3445 | 10.0898 | 22.0664 | 28.5988   | 60.5    |
| 3.4283        | 226.09 | 20800 | 4.2245          | 31.47   | 10.0171 | 21.9423 | 28.4329   | 61.175  |
| 3.4048        | 227.17 | 20900 | 4.2242          | 31.93   | 10.4874 | 22.5287 | 29.1292   | 60.7    |
| 3.3925        | 228.26 | 21000 | 4.2243          | 32.3618 | 10.0902 | 22.6176 | 29.2689   | 60.775  |
| 3.4371        | 229.35 | 21100 | 4.2245          | 32.174  | 10.0424 | 22.516  | 28.9855   | 60.775  |
| 3.3789        | 230.43 | 21200 | 4.2239          | 33.0237 | 10.8644 | 23.3016 | 29.916    | 61.275  |
| 3.4109        | 231.52 | 21300 | 4.2248          | 32.88   | 10.6969 | 22.8426 | 30.0468   | 60.8    |
| 3.4128        | 232.61 | 21400 | 4.2257          | 32.6551 | 10.6032 | 22.6787 | 29.5307   | 60.725  |
| 3.3941        | 233.7  | 21500 | 4.2266          | 31.9296 | 10.0718 | 22.5    | 28.9451   | 60.75   |
| 3.3734        | 234.78 | 21600 | 4.2266          | 32.4862 | 10.0754 | 22.9705 | 29.2087   | 61.225  |
| 3.4144        | 235.87 | 21700 | 4.2269          | 32.1757 | 10.1225 | 22.6842 | 29.1731   | 60.75   |
| 3.3986        | 236.96 | 21800 | 4.2273          | 32.3403 | 10.481  | 22.7186 | 29.3236   | 60.725  |
| 3.3898        | 238.04 | 21900 | 4.2275          | 32.4957 | 10.4595 | 22.8682 | 29.6414   | 60.8    |
| 3.4031        | 239.13 | 22000 | 4.2275          | 32.4625 | 10.3807 | 22.7121 | 29.5187   | 60.725  |
| 3.3836        | 240.22 | 22100 | 4.2274          | 31.8107 | 10.2075 | 22.4437 | 28.9719   | 60.725  |
| 3.4084        | 241.3  | 22200 | 4.2272          | 32.3374 | 10.1027 | 22.5784 | 29.2192   | 61.2    |
| 3.3805        | 242.39 | 22300 | 4.2276          | 32.2783 | 10.375  | 22.7825 | 29.3762   | 61.2    |
| 3.3815        | 243.48 | 22400 | 4.2277          | 32.3337 | 10.3561 | 22.8489 | 29.4485   | 61.15   |
| 3.418         | 244.57 | 22500 | 4.2273          | 32.333  | 10.2841 | 22.8481 | 29.403    | 61.125  |
| 3.369         | 245.65 | 22600 | 4.2277          | 32.038  | 10.3555 | 22.6939 | 29.242    | 60.7    |
| 3.4305        | 246.74 | 22700 | 4.2276          | 32.7594 | 10.6867 | 23.0632 | 29.5852   | 61.575  |
| 3.3928        | 247.83 | 22800 | 4.2282          | 32.4979 | 10.5013 | 22.7875 | 29.4793   | 61.55   |
| 3.3676        | 248.91 | 22900 | 4.2286          | 32.6014 | 10.5697 | 22.8526 | 29.7876   | 61.6    |
| 3.3918        | 250.0  | 23000 | 4.2288          | 32.4746 | 10.6321 | 22.586  | 29.6323   | 60.675  |
| 3.395         | 251.09 | 23100 | 4.2294          | 32.4704 | 10.5456 | 22.6785 | 29.5769   | 60.725  |
| 3.363         | 252.17 | 23200 | 4.2296          | 32.2721 | 10.2554 | 22.5303 | 29.4554   | 60.725  |
| 3.3884        | 253.26 | 23300 | 4.2298          | 32.2746 | 10.434  | 22.6686 | 29.4486   | 60.725  |
| 3.3891        | 254.35 | 23400 | 4.2296          | 32.5382 | 10.5112 | 23.0243 | 29.8106   | 61.125  |
| 3.3679        | 255.43 | 23500 | 4.2296          | 32.4656 | 10.5631 | 22.9952 | 29.6832   | 61.125  |
| 3.4078        | 256.52 | 23600 | 4.2297          | 32.3377 | 10.4791 | 22.8362 | 29.6212   | 60.7    |
| 3.3642        | 257.61 | 23700 | 4.2302          | 32.2519 | 10.5551 | 22.6957 | 29.3763   | 61.075  |
| 3.3745        | 258.7  | 23800 | 4.2300          | 31.9413 | 10.4752 | 22.7447 | 29.1      | 61.175  |
| 3.3844        | 259.78 | 23900 | 4.2305          | 32.237  | 10.5492 | 23.0342 | 29.4079   | 61.65   |
| 3.3501        | 260.87 | 24000 | 4.2302          | 31.9797 | 10.4631 | 22.9089 | 29.332    | 61.65   |
| 3.4259        | 261.96 | 24100 | 4.2304          | 31.7515 | 10.3564 | 22.5923 | 29.1275   | 61.175  |
| 3.3578        | 263.04 | 24200 | 4.2309          | 32.0462 | 10.3883 | 22.9083 | 29.3591   | 61.65   |
| 3.39          | 264.13 | 24300 | 4.2308          | 31.9307 | 10.3057 | 22.8501 | 29.2547   | 61.65   |
| 3.3805        | 265.22 | 24400 | 4.2312          | 32.1836 | 10.3577 | 23.1293 | 29.4325   | 61.65   |
| 3.3667        | 266.3  | 24500 | 4.2309          | 32.1545 | 10.301  | 23.0613 | 29.343    | 61.65   |
| 3.3977        | 267.39 | 24600 | 4.2313          | 31.9549 | 10.2824 | 23.0397 | 29.2684   | 61.65   |
| 3.3434        | 268.48 | 24700 | 4.2314          | 31.9432 | 10.167  | 23.098  | 29.2669   | 61.65   |
| 3.3577        | 269.57 | 24800 | 4.2316          | 31.9679 | 10.3075 | 23.0715 | 29.3077   | 61.65   |
| 3.3781        | 270.65 | 24900 | 4.2317          | 32.2292 | 10.2988 | 23.0879 | 29.4171   | 61.65   |
| 3.3514        | 271.74 | 25000 | 4.2321          | 32.1653 | 10.4198 | 23.0554 | 29.3574   | 61.65   |
| 3.3935        | 272.83 | 25100 | 4.2320          | 32.134  | 10.2884 | 22.9444 | 29.2272   | 61.65   |
| 3.3447        | 273.91 | 25200 | 4.2324          | 32.3498 | 10.4505 | 23.0734 | 29.4438   | 61.65   |
| 3.3872        | 275.0  | 25300 | 4.2323          | 32.1743 | 10.4152 | 22.9462 | 29.3187   | 61.65   |
| 3.3755        | 276.09 | 25400 | 4.2324          | 32.2311 | 10.372  | 22.9563 | 29.3285   | 61.65   |
| 3.3832        | 277.17 | 25500 | 4.2323          | 32.0289 | 10.2105 | 22.9636 | 29.1449   | 61.65   |
| 3.3367        | 278.26 | 25600 | 4.2321          | 32.3053 | 10.2512 | 23.0834 | 29.4111   | 61.65   |
| 3.3767        | 279.35 | 25700 | 4.2323          | 32.4099 | 10.2793 | 23.0137 | 29.4049   | 61.65   |
| 3.3989        | 280.43 | 25800 | 4.2324          | 32.3471 | 10.4356 | 23.0179 | 29.4453   | 61.65   |
| 3.3625        | 281.52 | 25900 | 4.2325          | 32.2213 | 10.4363 | 22.9573 | 29.2886   | 61.65   |
| 3.3352        | 282.61 | 26000 | 4.2328          | 32.713  | 10.7489 | 23.2367 | 29.8725   | 61.65   |
| 3.3899        | 283.7  | 26100 | 4.2328          | 32.2145 | 10.2347 | 22.7896 | 29.2107   | 61.65   |
| 3.359         | 284.78 | 26200 | 4.2327          | 32.2466 | 10.4236 | 22.916  | 29.4227   | 61.65   |
| 3.3866        | 285.87 | 26300 | 4.2327          | 32.2466 | 10.4236 | 22.916  | 29.4227   | 61.65   |
| 3.3845        | 286.96 | 26400 | 4.2328          | 32.2466 | 10.4236 | 22.916  | 29.4227   | 61.65   |
| 3.3486        | 288.04 | 26500 | 4.2328          | 32.595  | 10.5041 | 23.1214 | 29.69     | 61.65   |
| 3.3807        | 289.13 | 26600 | 4.2328          | 32.759  | 10.566  | 23.3108 | 29.8555   | 61.65   |
| 3.3676        | 290.22 | 26700 | 4.2330          | 32.759  | 10.566  | 23.3108 | 29.8555   | 61.65   |
| 3.3361        | 291.3  | 26800 | 4.2332          | 32.759  | 10.566  | 23.3108 | 29.8555   | 61.65   |
| 3.3897        | 292.39 | 26900 | 4.2331          | 32.7251 | 10.566  | 23.3108 | 29.7958   | 61.65   |
| 3.3579        | 293.48 | 27000 | 4.2331          | 32.759  | 10.566  | 23.3108 | 29.8555   | 61.65   |
| 3.3809        | 294.57 | 27100 | 4.2331          | 32.759  | 10.566  | 23.3108 | 29.8555   | 61.65   |
| 3.3885        | 295.65 | 27200 | 4.2331          | 32.759  | 10.566  | 23.3108 | 29.8555   | 61.65   |
| 3.3173        | 296.74 | 27300 | 4.2331          | 32.7156 | 10.5699 | 23.2759 | 29.7903   | 61.65   |
| 3.3648        | 297.83 | 27400 | 4.2331          | 32.7156 | 10.5699 | 23.2759 | 29.7903   | 61.65   |
| 3.3793        | 298.91 | 27500 | 4.2331          | 32.7156 | 10.5699 | 23.2759 | 29.7903   | 61.65   |
| 3.3604        | 300.0  | 27600 | 4.2331          | 32.7156 | 10.5699 | 23.2759 | 29.7903   | 61.65   |


### Framework versions

- Transformers 4.18.0
- Pytorch 1.11.0+cu113
- Datasets 2.1.0
- Tokenizers 0.12.1
