readme: ---
  license: apache-2.0
  tags:
  - generated_from_trainer
  model-index:
  - name: t5-small-mlm-pubmed-15
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # t5-small-mlm-pubmed-15
  
  This model is a fine-tuned version of [t5-small](https://huggingface.co/t5-small) on an unknown dataset.
  It achieves the following results on the evaluation set:
  - Loss: 0.5389
  - Rouge2 Precision: 0.7165
  - Rouge2 Recall: 0.5375
  - Rouge2 Fmeasure: 0.5981
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 2e-05
  - train_batch_size: 16
  - eval_batch_size: 16
  - seed: 42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - num_epochs: 40
  - mixed_precision_training: Native AMP
  
  ### Training results
  
  | Training Loss | Epoch | Step  | Validation Loss | Rouge2 Precision | Rouge2 Recall | Rouge2 Fmeasure |
  |:-------------:|:-----:|:-----:|:---------------:|:----------------:|:-------------:|:---------------:|
  | 1.1024        | 0.75  | 500   | 0.7890          | 0.6854           | 0.4813        | 0.5502          |
  | 0.8788        | 1.51  | 1000  | 0.7176          | 0.6906           | 0.4989        | 0.5638          |
  | 0.8086        | 2.26  | 1500  | 0.6830          | 0.6872           | 0.5052        | 0.5663          |
  | 0.7818        | 3.02  | 2000  | 0.6650          | 0.6912           | 0.5104        | 0.5711          |
  | 0.7466        | 3.77  | 2500  | 0.6458          | 0.6965           | 0.5167        | 0.5774          |
  | 0.731         | 4.52  | 3000  | 0.6355          | 0.6955           | 0.5161        | 0.5763          |
  | 0.7126        | 5.28  | 3500  | 0.6249          | 0.6924           | 0.517         | 0.576           |
  | 0.6998        | 6.03  | 4000  | 0.6166          | 0.6995           | 0.5207        | 0.5809          |
  | 0.6855        | 6.79  | 4500  | 0.6076          | 0.6981           | 0.5215        | 0.5813          |
  | 0.676         | 7.54  | 5000  | 0.6015          | 0.7003           | 0.5242        | 0.5836          |
  | 0.6688        | 8.3   | 5500  | 0.5962          | 0.7004           | 0.5235        | 0.583           |
  | 0.6569        | 9.05  | 6000  | 0.5900          | 0.6997           | 0.5234        | 0.5827          |
  | 0.6503        | 9.8   | 6500  | 0.5880          | 0.703            | 0.5257        | 0.5856          |
  | 0.6455        | 10.56 | 7000  | 0.5818          | 0.7008           | 0.5259        | 0.5849          |
  | 0.635         | 11.31 | 7500  | 0.5796          | 0.7017           | 0.5271        | 0.5861          |
  | 0.6323        | 12.07 | 8000  | 0.5769          | 0.7053           | 0.5276        | 0.5877          |
  | 0.6241        | 12.82 | 8500  | 0.5730          | 0.7011           | 0.5243        | 0.5838          |
  | 0.6224        | 13.57 | 9000  | 0.5696          | 0.7046           | 0.5286        | 0.5879          |
  | 0.6139        | 14.33 | 9500  | 0.5685          | 0.7047           | 0.5295        | 0.5886          |
  | 0.6118        | 15.08 | 10000 | 0.5653          | 0.704            | 0.5297        | 0.5886          |
  | 0.6089        | 15.84 | 10500 | 0.5633          | 0.703            | 0.5272        | 0.5865          |
  | 0.598         | 16.59 | 11000 | 0.5613          | 0.7059           | 0.5293        | 0.5889          |
  | 0.6003        | 17.35 | 11500 | 0.5602          | 0.7085           | 0.532         | 0.5918          |
  | 0.5981        | 18.1  | 12000 | 0.5587          | 0.7106           | 0.5339        | 0.5938          |
  | 0.5919        | 18.85 | 12500 | 0.5556          | 0.708            | 0.5319        | 0.5914          |
  | 0.5897        | 19.61 | 13000 | 0.5556          | 0.7106           | 0.5327        | 0.5931          |
  | 0.5899        | 20.36 | 13500 | 0.5526          | 0.7114           | 0.534         | 0.5939          |
  | 0.5804        | 21.12 | 14000 | 0.5521          | 0.7105           | 0.5328        | 0.5928          |
  | 0.5764        | 21.87 | 14500 | 0.5520          | 0.715            | 0.537         | 0.5976          |
  | 0.5793        | 22.62 | 15000 | 0.5506          | 0.713            | 0.5346        | 0.5951          |
  | 0.5796        | 23.38 | 15500 | 0.5492          | 0.7124           | 0.5352        | 0.5952          |
  | 0.5672        | 24.13 | 16000 | 0.5482          | 0.7124           | 0.5346        | 0.5948          |
  | 0.5737        | 24.89 | 16500 | 0.5470          | 0.7134           | 0.5352        | 0.5956          |
  | 0.5685        | 25.64 | 17000 | 0.5463          | 0.7117           | 0.5346        | 0.5946          |
  | 0.5658        | 26.4  | 17500 | 0.5457          | 0.7145           | 0.5359        | 0.5965          |
  | 0.5657   