readme: ---
  license: apache-2.0
  tags:
  - generated_from_trainer
  model-index:
  - name: t5-small-mlm-pubmed-35
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # t5-small-mlm-pubmed-35
  
  This model is a fine-tuned version of [t5-small](https://huggingface.co/t5-small) on an unknown dataset.
  It achieves the following results on the evaluation set:
  - Loss: 1.1101
  - Rouge2 Precision: 0.4758
  - Rouge2 Recall: 0.3498
  - Rouge2 Fmeasure: 0.3927
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 2e-05
  - train_batch_size: 16
  - eval_batch_size: 16
  - seed: 42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - num_epochs: 40
  - mixed_precision_training: Native AMP
  
  ### Training results
  
  | Training Loss | Epoch | Step  | Validation Loss | Rouge2 Precision | Rouge2 Recall | Rouge2 Fmeasure |
  |:-------------:|:-----:|:-----:|:---------------:|:----------------:|:-------------:|:---------------:|
  | 1.8404        | 0.75  | 500   | 1.5005          | 0.4265           | 0.2786        | 0.3273          |
  | 1.6858        | 1.51  | 1000  | 1.4216          | 0.4318           | 0.2946        | 0.3404          |
  | 1.6071        | 2.26  | 1500  | 1.3777          | 0.4472           | 0.3148        | 0.3598          |
  | 1.5551        | 3.02  | 2000  | 1.3360          | 0.4406           | 0.3168        | 0.3586          |
  | 1.5116        | 3.77  | 2500  | 1.3128          | 0.4523           | 0.3234        | 0.3671          |
  | 1.4837        | 4.52  | 3000  | 1.2937          | 0.4477           | 0.3215        | 0.3645          |
  | 1.4513        | 5.28  | 3500  | 1.2766          | 0.4511           | 0.3262        | 0.3689          |
  | 1.4336        | 6.03  | 4000  | 1.2626          | 0.4548           | 0.3283        | 0.3718          |
  | 1.4149        | 6.79  | 4500  | 1.2449          | 0.4495           | 0.3274        | 0.3687          |
  | 1.3977        | 7.54  | 5000  | 1.2349          | 0.4507           | 0.3305        | 0.3712          |
  | 1.3763        | 8.3   | 5500  | 1.2239          | 0.4519           | 0.3266        | 0.3688          |
  | 1.371         | 9.05  | 6000  | 1.2171          | 0.4546           | 0.3305        | 0.3727          |
  | 1.3501        | 9.8   | 6500  | 1.2080          | 0.4575           | 0.3329        | 0.3755          |
  | 1.3443        | 10.56 | 7000  | 1.2017          | 0.4576           | 0.3314        | 0.3742          |
  | 1.326         | 11.31 | 7500  | 1.1926          | 0.4578           | 0.333         | 0.3757          |
  | 1.3231        | 12.07 | 8000  | 1.1866          | 0.4606           | 0.3357        | 0.3782          |
  | 1.3089        | 12.82 | 8500  | 1.1816          | 0.4591           | 0.3338        | 0.3765          |
  | 1.3007        | 13.57 | 9000  | 1.1764          | 0.4589           | 0.3361        | 0.3777          |
  | 1.2943        | 14.33 | 9500  | 1.1717          | 0.4641           | 0.3382        | 0.3811          |
  | 1.2854        | 15.08 | 10000 | 1.1655          | 0.4617           | 0.3378        | 0.38            |
  | 1.2777        | 15.84 | 10500 | 1.1612          | 0.464            | 0.3401        | 0.3823          |
  | 1.2684        | 16.59 | 11000 | 1.1581          | 0.4608           | 0.3367        | 0.3789          |
  | 1.2612        | 17.35 | 11500 | 1.1554          | 0.4623           | 0.3402        | 0.3818          |
  | 1.2625        | 18.1  | 12000 | 1.1497          | 0.4613           | 0.3381        | 0.3802          |
  | 1.2529        | 18.85 | 12500 | 1.1465          | 0.4671           | 0.3419        | 0.3848          |
  | 1.2461        | 19.61 | 13000 | 1.1431          | 0.4646           | 0.3399        | 0.3824          |
  | 1.2415        | 20.36 | 13500 | 1.1419          | 0.4659           | 0.341         | 0.3835          |
  | 1.2375        | 21.12 | 14000 | 1.1377          | 0.4693           | 0.3447        | 0.3873          |
  | 1.2315        | 21.87 | 14500 | 1.1353          | 0.4672           | 0.3433        | 0.3855          |
  | 1.2263        | 22.62 | 15000 | 1.1333          | 0.467            | 0.3433        | 0.3854          |
  | 1.2214        | 23.38 | 15500 | 1.1305          | 0.4682           | 0.3446        | 0.3869          |
  | 1.2202        | 24.13 | 16000 | 1.1291          | 0.4703           | 0.3465        | 0.3888          |
  | 1.2155        | 24.89 | 16500 | 1.1270          | 0.472            | 0.348         | 0.3903          |
  | 1.2064        | 25.64 | 17000 | 1.1261          | 0.4724           | 0.3479        | 0.3905          |
  | 1.2173        | 26.4  | 17500 | 1.1236          | 0.4734           | 0.3485        | 0.3912          |
  | 1.1994   