readme: ---
  tags:
  - generated_from_trainer
  metrics:
  - wer
  model-index:
  - name: timit-distil-kl-alpha-0.25-T-1-take-2
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # timit-distil-kl-alpha-0.25-T-1-take-2
  
  This model is a fine-tuned version of [](https://huggingface.co/) on the None dataset.
  It achieves the following results on the evaluation set:
  - Loss: 170.1500
  - Wer: 0.7698
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 0.0001
  - train_batch_size: 28
  - eval_batch_size: 14
  - seed: 42
  - gradient_accumulation_steps: 2
  - total_train_batch_size: 56
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - lr_scheduler_warmup_ratio: 0.2
  - num_epochs: 50
  - mixed_precision_training: Native AMP
  
  ### Training results
  
  | Training Loss | Epoch | Step | Validation Loss | Wer    |
  |:-------------:|:-----:|:----:|:---------------:|:------:|
  | 583.5526      | 1.22  | 100  | 354.4852        | 1.0483 |
  | 367.4895      | 2.44  | 200  | 246.2063        | 0.9295 |
  | 275.3467      | 3.65  | 300  | 216.6919        | 0.8582 |
  | 235.4134      | 4.87  | 400  | 200.3912        | 0.8223 |
  | 217.7628      | 6.1   | 500  | 191.3873        | 0.8050 |
  | 196.5266      | 7.32  | 600  | 186.2596        | 0.7891 |
  | 182.3409      | 8.53  | 700  | 180.5465        | 0.7824 |
  | 173.6425      | 9.75  | 800  | 180.9611        | 0.7786 |
  | 158.4616      | 10.97 | 900  | 174.4431        | 0.7700 |
  | 162.9191      | 12.19 | 1000 | 174.9140        | 0.7767 |
  | 152.7855      | 13.41 | 1100 | 175.0573        | 0.7697 |
  | 146.0713      | 14.63 | 1200 | 174.0519        | 0.7735 |
  | 142.7635      | 15.85 | 1300 | 171.1087        | 0.7729 |
  | 143.8564      | 17.07 | 1400 | 172.2125        | 0.7665 |
  | 138.0579      | 18.29 | 1500 | 171.2589        | 0.7690 |
  | 133.162       | 19.51 | 1600 | 169.6842        | 0.7716 |
  | 132.8703      | 20.73 | 1700 | 173.7567        | 0.7750 |
  | 132.5092      | 21.95 | 1800 | 171.5918        | 0.7658 |
  | 133.7408      | 23.17 | 1900 | 170.9486        | 0.7692 |
  | 130.2913      | 24.39 | 2000 | 170.2246        | 0.7666 |
  | 127.7704      | 25.61 | 2100 | 169.7522        | 0.7680 |
  | 126.3399      | 26.82 | 2200 | 171.0318        | 0.7682 |
  | 127.5717      | 28.05 | 2300 | 170.2780        | 0.7665 |
  | 125.7467      | 29.27 | 2400 | 170.7915        | 0.7689 |
  | 119.2796      | 30.48 | 2500 | 170.7032        | 0.7691 |
  | 122.8742      | 31.7  | 2600 | 170.5696        | 0.7737 |
  | 121.9309      | 32.92 | 2700 | 170.1012        | 0.7721 |
  | 122.2507      | 34.15 | 2800 | 170.2254        | 0.7645 |
  | 120.9862      | 35.36 | 2900 | 170.4729        | 0.7752 |
  | 121.0826      | 36.58 | 3000 | 170.5613        | 0.7747 |
  | 119.0979      | 37.8  | 3100 | 170.4102        | 0.7695 |
  | 118.2004      | 39.02 | 3200 | 169.9209        | 0.7642 |
  | 116.4097      | 40.24 | 3300 | 170.4418        | 0.7685 |
  | 117.4168      | 41.46 | 3400 | 171.0443        | 0.7705 |
  | 118.362       | 42.68 | 3500 | 169.6040        | 0.7692 |
  | 117.1554      | 43.9  | 3600 | 169.7565        | 0.7682 |
  | 118.7433      | 45.12 | 3700 | 169.9207        | 0.7719 |
  | 115.2184      | 46.34 | 3800 | 170.1183        | 0.7711 |
  | 115.6529      | 47.56 | 3900 | 170.0537        | 0.7689 |
  | 115.4671      | 48.78 | 4000 | 170.2999        | 0.7700 |
  | 114.1326      | 49.99 | 4100 | 170.1500        | 0.7698 |
  
  
  ### Framework versions
  
  - Transformers 4.25.1
  - Pytorch 1.12.1
  - Datasets 2.8.0
  - Tokenizers 0.13.2
  