readme: ---
  tags:
  - generated_from_trainer
  model-index:
  - name: libri-alpha-0.75-Temp-1-attention-3-layers-distil-with-6-layers-take-2-train-extractor
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # libri-alpha-0.75-Temp-1-attention-3-layers-distil-with-6-layers-take-2-train-extractor
  
  This model is a fine-tuned version of [rohitp1/libri-alpha-0.75-Temp-1-attention-3-layers-distil-with-6-layers](https://huggingface.co/rohitp1/libri-alpha-0.75-Temp-1-attention-3-layers-distil-with-6-layers) on the None dataset.
  It achieves the following results on the evaluation set:
  - Loss: 123.6555
  - Wer: 0.2525
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 0.0002
  - train_batch_size: 8
  - eval_batch_size: 8
  - seed: 42
  - gradient_accumulation_steps: 2
  - total_train_batch_size: 16
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - lr_scheduler_warmup_ratio: 0.3
  - num_epochs: 30
  - mixed_precision_training: Native AMP
  
  ### Training results
  
  | Training Loss | Epoch | Step  | Validation Loss | Wer    |
  |:-------------:|:-----:|:-----:|:---------------:|:------:|
  | 488.7409      | 0.22  | 200   | 175.5911        | 0.4211 |
  | 470.3788      | 0.45  | 400   | 174.7645        | 0.4192 |
  | 472.5283      | 0.67  | 600   | 173.8402        | 0.4184 |
  | 474.1535      | 0.9   | 800   | 173.4610        | 0.4162 |
  | 488.9395      | 1.12  | 1000  | 172.2722        | 0.4172 |
  | 468.5794      | 1.35  | 1200  | 170.7173        | 0.4134 |
  | 473.337       | 1.57  | 1400  | 171.2823        | 0.4069 |
  | 453.5572      | 1.79  | 1600  | 168.4595        | 0.4093 |
  | 456.1514      | 2.02  | 1800  | 166.4398        | 0.4000 |
  | 447.1798      | 2.24  | 2000  | 167.9152        | 0.3994 |
  | 438.2698      | 2.47  | 2200  | 166.1868        | 0.3974 |
  | 438.1535      | 2.69  | 2400  | 164.5998        | 0.3946 |
  | 442.7301      | 2.91  | 2600  | 162.8684        | 0.3956 |
  | 440.5328      | 3.14  | 2800  | 162.3347        | 0.3861 |
  | 449.2731      | 3.36  | 3000  | 160.7815        | 0.3847 |
  | 436.718       | 3.59  | 3200  | 158.1402        | 0.3849 |
  | 425.2622      | 3.81  | 3400  | 157.0624        | 0.3778 |
  | 430.4346      | 4.04  | 3600  | 156.7345        | 0.3764 |
  | 402.7262      | 4.26  | 3800  | 154.0662        | 0.3635 |
  | 405.4374      | 4.48  | 4000  | 153.8651        | 0.3683 |
  | 395.4657      | 4.71  | 4200  | 152.3929        | 0.3609 |
  | 401.6397      | 4.93  | 4400  | 150.4990        | 0.3576 |
  | 397.0791      | 5.16  | 4600  | 151.3244        | 0.3634 |
  | 399.281       | 5.38  | 4800  | 149.6291        | 0.3513 |
  | 392.448       | 5.61  | 5000  | 149.6411        | 0.3474 |
  | 396.3989      | 5.83  | 5200  | 148.5435        | 0.3459 |
  | 381.1296      | 6.05  | 5400  | 147.9963        | 0.3501 |
  | 384.1926      | 6.28  | 5600  | 145.6473        | 0.3435 |
  | 364.3308      | 6.5   | 5800  | 145.9607        | 0.3381 |
  | 365.9475      | 6.73  | 6000  | 142.4151        | 0.3382 |
  | 359.6295      | 6.95  | 6200  | 139.8908        | 0.3315 |
  | 361.9945      | 7.17  | 6400  | 143.2300        | 0.3403 |
  | 370.9596      | 7.4   | 6600  | 140.1414        | 0.3280 |
  | 363.0185      | 7.62  | 6800  | 140.3988        | 0.3240 |
  | 354.5542      | 7.85  | 7000  | 143.5237        | 0.3286 |
  | 356.7341      | 8.07  | 7200  | 145.7105        | 0.3229 |
  | 342.3261      | 8.3   | 7400  | 137.8948        | 0.3188 |
  | 343.8778      | 8.52  | 7600  | 138.7520        | 0.3085 |
  | 327.9473      | 8.74  | 7800  | 136.1127        | 0.3122 |
  | 339.7105      | 8.97  | 8000  | 136.3135        | 0.3084 |
  | 322.9032      | 9.19  | 8200  | 136.0534        | 0.3089 |
  | 332.4099      | 9.42  | 8400  | 136.3784        | 0.3079 |
  | 333.1054      | 9.64  | 8600  | 136.3690        | 0.3020 |
  | 325.0327      | 9.87  | 8800  | 138.1514        | 0.3022 |
  | 326.1452      | 10.09 | 9000  | 130.8793        | 0.2944 |
  | 319.7307      | 10.31 | 9200  | 133.0722        | 0.2945 |
  | 322.89        | 10.54 | 9400  | 131.6615        | 0.2961 |
  | 307.7924      | 10.76 | 9600  | 129.8601        | 0.2917 |
  | 322.2392      | 10.99 | 9800  | 131.7703        | 0.2911 |
  | 306.9055      | 11.21 | 10000 | 130.2165        | 0.2878 |
  | 297.5498      | 11.43 | 10200 | 130.4440        | 0.2920 |
  | 300.9818      | 11.66 | 10400 | 130.6544        | 0.2862 |
  | 300.7568      | 11.88 | 10600 | 128.4007        | 0.2857 |
  | 298.6313      | 12.11 | 10800 | 129.3903        | 0.2808 |
  | 286.8174      | 12.33 | 11000 | 129.0809        | 0.2824 |
  | 290.7518      | 12.56 | 11200 | 130.4312        | 0.2827 |
  | 292.7182      | 12.78 | 11400 | 129.6407        | 0.2829 |
  | 287.0013      | 13.0  | 11600 