readme: ---
  tags:
  - generated_from_trainer
  model-index:
  - name: libri-finetune-6-layers-transformer
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # libri-finetune-6-layers-transformer
  
  This model is a fine-tuned version of [](https://huggingface.co/) on the None dataset.
  It achieves the following results on the evaluation set:
  - Loss: 83.8115
  - Wer: 0.7450
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 0.0001
  - train_batch_size: 32
  - eval_batch_size: 32
  - seed: 42
  - gradient_accumulation_steps: 2
  - total_train_batch_size: 64
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - lr_scheduler_warmup_ratio: 0.2
  - num_epochs: 100
  - mixed_precision_training: Native AMP
  
  ### Training results
  
  | Training Loss | Epoch | Step | Validation Loss | Wer    |
  |:-------------:|:-----:|:----:|:---------------:|:------:|
  | 15064.3113    | 2.99  | 400  | 5870.5229       | 0.9999 |
  | 7699.3169     | 5.97  | 800  | 4746.7100       | 0.9999 |
  | 6725.2825     | 8.96  | 1200 | 2085.7102       | 1.0483 |
  | 2585.5492     | 11.94 | 1600 | 627.4510        | 0.8690 |
  | 1256.1203     | 14.93 | 2000 | 305.7397        | 0.8220 |
  | 829.551       | 17.91 | 2400 | 195.8744        | 0.8150 |
  | 562.8857      | 20.9  | 2800 | 134.9562        | 0.8043 |
  | 383.4462      | 23.88 | 3200 | 103.3252        | 0.7821 |
  | 251.6083      | 26.87 | 3600 | 83.2899         | 0.7851 |
  | 176.5321      | 29.85 | 4000 | 72.0544         | 0.7668 |
  | 107.0474      | 32.84 | 4400 | 81.2677         | 0.7692 |
  | 47.8587       | 35.82 | 4800 | 61.3930         | 0.7735 |
  | 23.0181       | 38.81 | 5200 | 77.2446         | 0.7795 |
  | -41.5376      | 41.79 | 5600 | 55.3390         | 0.7637 |
  | -82.3179      | 44.78 | 6000 | 60.2555         | 0.7582 |
  | -89.828       | 47.76 | 6400 | 68.0762         | 0.7566 |
  | -132.0839     | 50.75 | 6800 | 78.1840         | 0.7478 |
  | -149.8906     | 53.73 | 7200 | 87.0740         | 0.7535 |
  | -181.8106     | 56.72 | 7600 | 83.8115         | 0.7450 |
  
  
  ### Framework versions
  
  - Transformers 4.23.1
  - Pytorch 1.12.1
  - Datasets 2.6.1
  - Tokenizers 0.13.1
  