readme: ---
  tags:
  - generated_from_trainer
  metrics:
  - wer
  model-index:
  - name: libri-alpha-1-Temp-1-attention-4
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # libri-alpha-1-Temp-1-attention-4
  
  This model is a fine-tuned version of [](https://huggingface.co/) on the None dataset.
  It achieves the following results on the evaluation set:
  - Loss: 254.9241
  - Wer: 0.2989
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 2e-05
  - train_batch_size: 32
  - eval_batch_size: 32
  - seed: 42
  - gradient_accumulation_steps: 2
  - total_train_batch_size: 64
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - lr_scheduler_warmup_steps: 500
  - num_epochs: 30
  - mixed_precision_training: Native AMP
  
  ### Training results
  
  | Training Loss | Epoch | Step | Validation Loss | Wer    |
  |:-------------:|:-----:|:----:|:---------------:|:------:|
  | 1911.3686     | 0.75  | 100  | 495.1192        | 0.5273 |
  | 1556.7764     | 1.49  | 200  | 406.3832        | 0.4759 |
  | 1347.0447     | 2.24  | 300  | 371.9665        | 0.4413 |
  | 1303.7259     | 2.99  | 400  | 350.5383        | 0.4145 |
  | 1200.0694     | 3.73  | 500  | 336.2006        | 0.3933 |
  | 1177.6688     | 4.48  | 600  | 325.6101        | 0.3803 |
  | 1174.224      | 5.22  | 700  | 318.7997        | 0.3697 |
  | 1037.0171     | 5.97  | 800  | 310.2211        | 0.3616 |
  | 1012.8402     | 6.72  | 900  | 301.0019        | 0.3544 |
  | 1012.0772     | 7.46  | 1000 | 295.5239        | 0.3495 |
  | 956.3445      | 8.21  | 1100 | 290.5795        | 0.3450 |
  | 935.5793      | 8.96  | 1200 | 286.1022        | 0.3408 |
  | 955.9658      | 9.7   | 1300 | 281.6091        | 0.3377 |
  | 936.7212      | 10.45 | 1400 | 278.7719        | 0.3349 |
  | 863.0611      | 11.19 | 1500 | 275.7689        | 0.3268 |
  | 871.686       | 11.94 | 1600 | 274.4354        | 0.3296 |
  | 891.1862      | 12.69 | 1700 | 271.1785        | 0.3226 |
  | 905.8692      | 13.43 | 1800 | 269.1327        | 0.3209 |
  | 840.9667      | 14.18 | 1900 | 268.0669        | 0.3160 |
  | 856.374       | 14.93 | 2000 | 266.1927        | 0.3174 |
  | 853.3527      | 15.67 | 2100 | 265.2582        | 0.3138 |
  | 845.1686      | 16.42 | 2200 | 265.0404        | 0.3135 |
  | 863.8627      | 17.16 | 2300 | 264.0161        | 0.3127 |
  | 812.8962      | 17.91 | 2400 | 262.0590        | 0.3104 |
  | 791.2973      | 18.66 | 2500 | 260.7250        | 0.3081 |
  | 823.047       | 19.4  | 2600 | 260.4928        | 0.3057 |
  | 808.3427      | 20.15 | 2700 | 260.0193        | 0.3066 |
  | 787.3638      | 20.9  | 2800 | 258.6302        | 0.3057 |
  | 788.9616      | 21.64 | 2900 | 258.6351        | 0.3041 |
  | 794.2102      | 22.39 | 3000 | 258.3393        | 0.3027 |
  | 805.2969      | 23.13 | 3100 | 257.5751        | 0.3031 |
  | 799.9891      | 23.88 | 3200 | 257.0037        | 0.3034 |
  | 784.6213      | 24.63 | 3300 | 256.6911        | 0.3024 |
  | 809.3116      | 25.37 | 3400 | 255.7936        | 0.3005 |
  | 790.4498      | 26.12 | 3500 | 255.7168        | 0.3001 |
  | 778.9627      | 26.87 | 3600 | 255.7248        | 0.2995 |
  | 754.2412      | 27.61 | 3700 | 255.3348        | 0.3000 |
  | 778.785       | 28.36 | 3800 | 254.8843        | 0.2991 |
  | 754.3623      | 29.1  | 3900 | 255.0664        | 0.2991 |
  | 803.0266      | 29.85 | 4000 | 254.9241        | 0.2989 |
  
  
  ### Framework versions
  
  - Transformers 4.24.0
  - Pytorch 1.12.1
  - Datasets 2.7.0
  - Tokenizers 0.11.0
  