readme: ---
  tags:
  - generated_from_trainer
  model-index:
  - name: libri-alpha-0.75-Temp-1-attention-3-layers-distil-with-6-layers
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # libri-alpha-0.75-Temp-1-attention-3-layers-distil-with-6-layers
  
  This model is a fine-tuned version of [](https://huggingface.co/) on the None dataset.
  It achieves the following results on the evaluation set:
  - Loss: 176.0337
  - Wer: 0.4211
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 2e-05
  - train_batch_size: 8
  - eval_batch_size: 8
  - seed: 42
  - gradient_accumulation_steps: 2
  - total_train_batch_size: 16
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - lr_scheduler_warmup_ratio: 0.2
  - num_epochs: 30
  - mixed_precision_training: Native AMP
  
  ### Training results
  
  | Training Loss | Epoch | Step  | Validation Loss | Wer    |
  |:-------------:|:-----:|:-----:|:---------------:|:------:|
  | 818.3828      | 0.19  | 100   | 299.9131        | 0.6582 |
  | 836.7686      | 0.37  | 200   | 297.5387        | 0.6554 |
  | 787.5465      | 0.56  | 300   | 294.7996        | 0.6512 |
  | 804.7234      | 0.75  | 400   | 292.2293        | 0.6463 |
  | 812.2401      | 0.93  | 500   | 289.8000        | 0.6398 |
  | 796.3071      | 1.12  | 600   | 287.0509        | 0.6351 |
  | 755.1163      | 1.31  | 700   | 283.9792        | 0.6282 |
  | 789.9813      | 1.49  | 800   | 280.8894        | 0.6241 |
  | 746.3034      | 1.68  | 900   | 278.0494        | 0.6193 |
  | 762.4461      | 1.87  | 1000  | 276.0318        | 0.6131 |
  | 750.3801      | 2.06  | 1100  | 273.0303        | 0.6117 |
  | 759.8897      | 2.24  | 1200  | 270.4828        | 0.6072 |
  | 766.8752      | 2.43  | 1300  | 267.3370        | 0.6017 |
  | 747.1767      | 2.62  | 1400  | 264.8568        | 0.5978 |
  | 709.7859      | 2.8   | 1500  | 262.1042        | 0.5940 |
  | 682.8446      | 2.99  | 1600  | 258.8314        | 0.5878 |
  | 678.7934      | 3.18  | 1700  | 255.1844        | 0.5842 |
  | 688.9065      | 3.36  | 1800  | 253.7106        | 0.5789 |
  | 676.8735      | 3.55  | 1900  | 252.6307        | 0.5784 |
  | 671.5541      | 3.74  | 2000  | 249.2319        | 0.5723 |
  | 661.5926      | 3.92  | 2100  | 247.3242        | 0.5669 |
  | 669.4624      | 4.11  | 2200  | 243.3662        | 0.5623 |
  | 643.5416      | 4.3   | 2300  | 241.4045        | 0.5609 |
  | 663.1716      | 4.49  | 2400  | 239.8308        | 0.5571 |
  | 645.2939      | 4.67  | 2500  | 236.4551        | 0.5498 |
  | 643.487       | 4.86  | 2600  | 234.4706        | 0.5463 |
  | 661.5359      | 5.05  | 2700  | 233.3655        | 0.5420 |
  | 620.0827      | 5.23  | 2800  | 230.9604        | 0.5410 |
  | 623.4608      | 5.42  | 2900  | 229.2104        | 0.5346 |
  | 614.5616      | 5.61  | 3000  | 227.1311        | 0.5315 |
  | 610.5325      | 5.79  | 3100  | 224.7402        | 0.5308 |
  | 609.3737      | 5.98  | 3200  | 224.0515        | 0.5268 |
  | 577.5857      | 6.17  | 3300  | 222.1016        | 0.5211 |
  | 600.8658      | 6.35  | 3400  | 220.7187        | 0.5190 |
  | 579.8273      | 6.54  | 3500  | 219.0504        | 0.5167 |
  | 577.1318      | 6.73  | 3600  | 216.5892        | 0.5138 |
  | 598.4185      | 6.92  | 3700  | 214.8921        | 0.5092 |
  | 574.2131      | 7.1   | 3800  | 214.3345        | 0.5069 |
  | 558.845       | 7.29  | 3900  | 212.7756        | 0.5027 |
  | 562.5259      | 7.48  | 4000  | 210.9550        | 0.5001 |
  | 542.4756      | 7.66  | 4100  | 210.1631        | 0.4984 |
  | 531.1471      | 7.85  | 4200  | 208.7503        | 0.4963 |
  | 578.4951      | 8.04  | 4300  | 206.8714        | 0.4942 |
  | 549.3659      | 8.22  | 4400  | 206.2005        | 0.4905 |
  | 549.3715      | 8.41  | 4500  | 204.9378        | 0.4897 |
  | 526.3116      | 8.6   | 4600  | 204.2942        | 0.4877 |
  | 556.9892      | 8.78  | 4700  | 203.8197        | 0.4838 |
  | 536.3161      | 8.97  | 4800  | 201.7986        | 0.4811 |
  | 566.8901      | 9.16  | 4900  | 201.8660        | 0.4819 |
  | 527.6058      | 9.35  | 5000  | 201.0037        | 0.4783 |
  | 537.7548      | 9.53  | 5100  | 199.5784        | 0.4765 |
  | 521.8008      | 9.72  | 5200  | 199.2429        | 0.4740 |
  | 527.2388      | 9.91  | 5300  | 199.3108        | 0.4754 |
  | 510.5078      | 10.09 | 5400  | 198.8395        | 0.4722 |
  | 543.6402      | 10.28 | 5500  | 198.2866        | 0.4706 |
  | 534.623       | 10.47 | 5600  | 197.0690        | 0.4686 |
  | 504.127       | 10.65 | 5700  | 198.0624        | 0.4706 |
  | 522.8051      | 10.84 | 5800  | 196.0252        | 0.4649 |
  | 543.651       | 11.03 | 5900  | 193.4206        | 0.4655 |
  | 515.7955      | 11.21 | 6000  | 193.9807        | 0.4640 |
  | 504.2811      | 11.4  | 6100  | 193.