readme: ---
  license: apache-2.0
  tags:
  - generated_from_trainer
  metrics:
  - rouge
  model-index:
  - name: t5-small-finetuned-qa-google-en_v1
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # t5-small-finetuned-qa-google-en_v1
  
  This model is a fine-tuned version of [t5-small](https://huggingface.co/t5-small) on an unknown dataset.
  It achieves the following results on the evaluation set:
  - Loss: 2.1413
  - Rouge1: 40.2873
  - Rouge2: 30.6667
  - Rougel: 40.1625
  - Rougelsum: 40.2529
  - Gen Len: 6.352
  
  ## Model description
  
  Model finetuned to generate answers based on an input paragraph.
  
  Example input:
  
  
  > extract answer: \<hl\> The most recent major version of Python is Python 3, which we shall be using in this tutorial. \<hl\> However, Python 2, although not being updated with anything other than security updates, is still quite popular.
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  Model trained on Google Natural Questions in this repo: https://huggingface.co/datasets/IsaacBot/Natural-Questions-Sentence-Highlight. The dataset allows training for question and/or answer generation. This model was trained to generate answer based on an input paragraph (no question hint).
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 0.0001
  - train_batch_size: 32
  - eval_batch_size: 32
  - seed: 42
  - gradient_accumulation_steps: 8
  - total_train_batch_size: 256
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - num_epochs: 5
  - mixed_precision_training: Native AMP
  - label_smoothing_factor: 0.1
  
  ### Training results
  
  | Training Loss | Epoch | Step | Validation Loss | Rouge1  | Rouge2  | Rougel  | Rougelsum | Gen Len |
  |:-------------:|:-----:|:----:|:---------------:|:-------:|:-------:|:-------:|:---------:|:-------:|
  | No log        | 0.27  | 100  | 2.3595          | 33.588  | 25.61   | 33.5609 | 33.6388   | 6.141   |
  | No log        | 0.53  | 200  | 2.2774          | 34.5528 | 26.0686 | 34.5097 | 34.58     | 6.3135  |
  | No log        | 0.8   | 300  | 2.2435          | 36.1641 | 27.857  | 36.134  | 36.1435   | 6.5255  |
  | No log        | 1.06  | 400  | 2.2117          | 36.7651 | 28.2274 | 36.7426 | 36.7278   | 6.3635  |
  | 2.3999        | 1.33  | 500  | 2.1987          | 38.0006 | 29.1016 | 37.9354 | 37.9785   | 6.485   |
  | 2.3999        | 1.6   | 600  | 2.1846          | 37.978  | 28.9293 | 37.8811 | 37.9299   | 6.321   |
  | 2.3999        | 1.86  | 700  | 2.1782          | 38.6531 | 29.582  | 38.5923 | 38.5797   | 6.484   |
  | 2.3999        | 2.13  | 800  | 2.1726          | 38.8536 | 29.9533 | 38.7785 | 38.8452   | 6.537   |
  | 2.3999        | 2.39  | 900  | 2.1640          | 38.7099 | 29.9414 | 38.6318 | 38.6736   | 6.4635  |
  | 2.2365        | 2.66  | 1000 | 2.1563          | 39.2126 | 30.0471 | 39.1838 | 39.1994   | 6.4135  |
  | 2.2365        | 2.93  | 1100 | 2.1579          | 39.6397 | 30.5926 | 39.5747 | 39.6701   | 6.4395  |
  | 2.2365        | 3.19  | 1200 | 2.1525          | 39.3201 | 30.1897 | 39.2362 | 39.3391   | 6.4305  |
  | 2.2365        | 3.46  | 1300 | 2.1514          | 39.4479 | 30.2987 | 39.3595 | 39.374    | 6.311   |
  | 2.2365        | 3.72  | 1400 | 2.1478          | 39.7449 | 30.6635 | 39.6403 | 39.6993   | 6.372   |
  | 2.1979        | 3.99  | 1500 | 2.1453          | 39.7789 | 30.4461 | 39.6917 | 39.8043   | 6.4215  |
  | 2.1979        | 4.26  | 1600 | 2.1427          | 39.8127 | 30.3727 | 39.6894 | 39.7522   | 6.3165  |
  | 2.1979        | 4.52  | 1700 | 2.1426          | 40.3472 | 30.7014 | 40.2174 | 40.3229   | 6.452   |
  | 2.1979        | 4.79  | 1800 | 2.1413          | 40.2873 | 30.6667 | 40.1625 | 40.2529   | 6.352   |
  
  
  ### Framework versions
  
  - Transformers 4.17.0
  - Pytorch 1.10.0+cu111
  - Datasets 2.0.0
  - Tokenizers 0.11.6
  