readme: ---
  tags:
  - generated_from_trainer
  metrics:
  - wer
  model-index:
  - name: libri-alpha-0.75-Temp-1-attention-3-layers-distil-with-6-layers-mse-take-4-unfreeze-extractor
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # libri-alpha-0.75-Temp-1-attention-3-layers-distil-with-6-layers-mse-take-4-unfreeze-extractor
  
  This model is a fine-tuned version of [rohitp1/libri-alpha-0.75-Temp-1-attention-3-layers-distil-with-6-layers-mse](https://huggingface.co/rohitp1/libri-alpha-0.75-Temp-1-attention-3-layers-distil-with-6-layers-mse) on the None dataset.
  It achieves the following results on the evaluation set:
  - Loss: 35.4977
  - Wer: 0.2414
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 0.0002
  - train_batch_size: 8
  - eval_batch_size: 1
  - seed: 42
  - gradient_accumulation_steps: 2
  - total_train_batch_size: 16
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - lr_scheduler_warmup_ratio: 0.2
  - num_epochs: 40
  - mixed_precision_training: Native AMP
  
  ### Training results
  
  | Training Loss | Epoch | Step  | Validation Loss | Wer    |
  |:-------------:|:-----:|:-----:|:---------------:|:------:|
  | 557.7442      | 0.45  | 400   | 28.5437         | 0.3344 |
  | 576.4579      | 0.9   | 800   | 28.0582         | 0.3313 |
  | 557.924       | 1.35  | 1200  | 28.2032         | 0.3285 |
  | 558.8386      | 1.79  | 1600  | 28.0733         | 0.3327 |
  | 583.0312      | 2.24  | 2000  | 28.3506         | 0.3254 |
  | 559.6182      | 2.69  | 2400  | 27.7517         | 0.3245 |
  | 555.811       | 3.14  | 2800  | 28.1994         | 0.3275 |
  | 555.9074      | 3.59  | 3200  | 28.2289         | 0.3267 |
  | 569.4283      | 4.04  | 3600  | 27.9987         | 0.3247 |
  | 523.5996      | 4.48  | 4000  | 27.9328         | 0.3178 |
  | 543.8255      | 4.93  | 4400  | 28.0181         | 0.3192 |
  | 508.707       | 5.38  | 4800  | 27.8667         | 0.3172 |
  | 518.0536      | 5.83  | 5200  | 28.0461         | 0.3120 |
  | 516.7025      | 6.28  | 5600  | 28.6324         | 0.3193 |
  | 509.9804      | 6.73  | 6000  | 28.8554         | 0.3202 |
  | 522.2005      | 7.17  | 6400  | 28.4986         | 0.3173 |
  | 501.0925      | 7.62  | 6800  | 28.5744         | 0.3095 |
  | 506.2044      | 8.07  | 7200  | 29.1753         | 0.3108 |
  | 464.1213      | 8.52  | 7600  | 28.5564         | 0.3080 |
  | 483.3067      | 8.97  | 8000  | 28.3099         | 0.3063 |
  | 463.7952      | 9.42  | 8400  | 28.4788         | 0.2990 |
  | 474.824       | 9.87  | 8800  | 27.5007         | 0.2959 |
  | 441.7981      | 10.31 | 9200  | 28.3279         | 0.2906 |
  | 445.6532      | 10.76 | 9600  | 27.6901         | 0.2881 |
  | 427.3226      | 11.21 | 10000 | 28.5749         | 0.2860 |
  | 419.5903      | 11.66 | 10400 | 27.3023         | 0.2825 |
  | 425.3329      | 12.11 | 10800 | 28.3225         | 0.2803 |
  | 401.3551      | 12.56 | 11200 | 28.1836         | 0.2814 |
  | 409.8571      | 13.0  | 11600 | 27.9721         | 0.2806 |
  | 382.0269      | 13.45 | 12000 | 28.2285         | 0.2798 |
  | 363.1065      | 13.9  | 12400 | 28.9252         | 0.2821 |
  | 386.975       | 14.35 | 12800 | 28.7444         | 0.2778 |
  | 370.1886      | 14.8  | 13200 | 28.3816         | 0.2738 |
  | 385.9398      | 15.25 | 13600 | 29.5411         | 0.2759 |
  | 347.4368      | 15.7  | 14000 | 28.5876         | 0.2710 |
  | 338.2872      | 16.14 | 14400 | 28.9052         | 0.2709 |
  | 347.3471      | 16.59 | 14800 | 28.3766         | 0.2679 |
  | 344.1634      | 17.04 | 15200 | 29.3270         | 0.2669 |
  | 333.9699      | 17.49 | 15600 | 29.2184         | 0.2656 |
  | 326.7914      | 17.94 | 16000 | 29.4644         | 0.2659 |
  | 328.6156      | 18.39 | 16400 | 30.1155         | 0.2686 |
  | 314.8902      | 18.83 | 16800 | 29.8135         | 0.2653 |
  | 320.2311      | 19.28 | 17200 | 30.4169         | 0.2654 |
  | 311.5116      | 19.73 | 17600 | 30.7323         | 0.2654 |
  | 320.7442      | 20.18 | 18000 | 30.3148         | 0.2616 |
  | 310.1395      | 20.63 | 18400 | 30.3432         | 0.2626 |
  | 298.6844      | 21.08 | 18800 | 30.3217         | 0.2611 |
  | 294.7287      | 21.52 | 19200 | 30.4799         | 0.2574 |
  | 301.9398      | 21.97 | 19600 | 29.9043         | 0.2562 |
  | 285.6117      | 22.42 | 20000 | 30.6270         | 0.2574 |
  | 299.511       | 22.87 | 20400 | 30.4342         | 0.2580 |
  | 271.373       | 23.32 | 20800 | 31.1784         | 0.2583 |
  | 289.4111      | 23.77 | 21200 | 30.8436         | 0.2562 |
  | 266.0083      | 24.22 | 21600 | 31.6785         | 0.2576 |
  | 271.6104      | 24.66 | 22000 | 31.7733         | 0.2565 |
  | 280.7621      | 25.11 | 22400 | 32.7097         | 0.2564 |
  | 254.1648      | 25.56 | 22800 | 33.1091         | 0.256