readme: ---
  tags:
  - generated_from_trainer
  metrics:
  - wer
  model-index:
  - name: timit-distil-kl-alpha-0.25-T-1-take-3
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # timit-distil-kl-alpha-0.25-T-1-take-3
  
  This model is a fine-tuned version of [](https://huggingface.co/) on the None dataset.
  It achieves the following results on the evaluation set:
  - Loss: 362.6431
  - Wer: 0.8022
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 0.0001
  - train_batch_size: 56
  - eval_batch_size: 56
  - seed: 42
  - gradient_accumulation_steps: 2
  - total_train_batch_size: 112
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - lr_scheduler_warmup_ratio: 0.2
  - num_epochs: 50
  - mixed_precision_training: Native AMP
  
  ### Training results
  
  | Training Loss | Epoch | Step | Validation Loss | Wer    |
  |:-------------:|:-----:|:----:|:---------------:|:------:|
  | 836.3694      | 2.43  | 100  | 652.1216        | 1.0487 |
  | 438.9098      | 4.87  | 200  | 438.7045        | 0.8833 |
  | 320.2348      | 7.31  | 300  | 396.8615        | 0.8490 |
  | 267.4869      | 9.75  | 400  | 381.3956        | 0.8325 |
  | 243.05        | 12.19 | 500  | 374.6377        | 0.8292 |
  | 226.4688      | 14.63 | 600  | 372.4966        | 0.8197 |
  | 220.0781      | 17.07 | 700  | 368.0202        | 0.8213 |
  | 206.6639      | 19.51 | 800  | 366.3605        | 0.8112 |
  | 199.0381      | 21.94 | 900  | 366.9292        | 0.8271 |
  | 198.3046      | 24.39 | 1000 | 365.8394        | 0.8088 |
  | 188.066       | 26.82 | 1100 | 364.1574        | 0.8057 |
  | 188.2653      | 29.27 | 1200 | 364.2211        | 0.8025 |
  | 181.248       | 31.7  | 1300 | 363.9985        | 0.8071 |
  | 182.5918      | 34.14 | 1400 | 363.5379        | 0.8042 |
  | 177.1421      | 36.58 | 1500 | 363.5888        | 0.8032 |
  | 179.904       | 39.02 | 1600 | 362.6931        | 0.8038 |
  | 174.7976      | 41.46 | 1700 | 362.9397        | 0.8053 |
  | 173.5596      | 43.89 | 1800 | 362.9175        | 0.8011 |
  | 176.6717      | 46.34 | 1900 | 363.0562        | 0.8013 |
  | 173.9602      | 48.77 | 2000 | 362.6431        | 0.8022 |
  
  
  ### Framework versions
  
  - Transformers 4.25.1
  - Pytorch 1.12.1
  - Datasets 2.8.0
  - Tokenizers 0.13.2
  