readme: ---
  tags:
  - generated_from_trainer
  metrics:
  - wer
  model-index:
  - name: libri-alpha-0.75-Temp-1-attention-3-layers-distil-with-6-layers-loss-att-take-2
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # libri-alpha-0.75-Temp-1-attention-3-layers-distil-with-6-layers-loss-att-take-2
  
  This model was trained from scratch on the None dataset.
  It achieves the following results on the evaluation set:
  - Loss: 26.4101
  - Wer: 0.2791
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 0.0002
  - train_batch_size: 16
  - eval_batch_size: 4
  - seed: 42
  - gradient_accumulation_steps: 2
  - total_train_batch_size: 32
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - lr_scheduler_warmup_ratio: 0.2
  - num_epochs: 40
  - mixed_precision_training: Native AMP
  
  ### Training results
  
  | Training Loss | Epoch | Step | Validation Loss | Wer    |
  |:-------------:|:-----:|:----:|:---------------:|:------:|
  | 202.4293      | 0.45  | 200  | 26.7777         | 0.2779 |
  | 197.6471      | 0.9   | 400  | 25.8300         | 0.2760 |
  | 204.8931      | 1.35  | 600  | 25.6774         | 0.2747 |
  | 193.3182      | 1.79  | 800  | 25.6049         | 0.2737 |
  | 205.2241      | 2.24  | 1000 | 25.5552         | 0.2739 |
  | 186.0407      | 2.69  | 1200 | 25.4364         | 0.2737 |
  | 191.7055      | 3.14  | 1400 | 25.7949         | 0.2764 |
  | 185.0721      | 3.59  | 1600 | 26.1202         | 0.2753 |
  | 198.8579      | 4.04  | 1800 | 25.8496         | 0.2763 |
  | 185.7877      | 4.48  | 2000 | 27.0753         | 0.2731 |
  | 194.9394      | 4.93  | 2200 | 25.6920         | 0.2775 |
  | 188.2296      | 5.38  | 2400 | 25.7362         | 0.2742 |
  | 188.0202      | 5.83  | 2600 | 25.9170         | 0.2755 |
  | 191.5541      | 6.28  | 2800 | 26.8590         | 0.2771 |
  | 198.2817      | 6.73  | 3000 | 26.4101         | 0.2791 |
  
  
  ### Framework versions
  
  - Transformers 4.24.0
  - Pytorch 1.12.1
  - Datasets 2.7.0
  - Tokenizers 0.11.0
  