readme: ---
  license: apache-2.0
  tags:
  - automatic-speech-recognition
  - NbAiLab/NST
  - generated_from_trainer
  model-index:
  - name: wav2vec2-1b-nst
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # wav2vec2-1b-nst
  
  This model is a fine-tuned version of [facebook/wav2vec2-xls-r-1b](https://huggingface.co/facebook/wav2vec2-xls-r-1b) on the NBAILAB/NST - NO-CLOSE dataset.
  It achieves the following results on the evaluation set:
  - Loss: 0.1921
  - Wer: 0.0469
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 2e-05
  - train_batch_size: 12
  - eval_batch_size: 12
  - seed: 42
  - gradient_accumulation_steps: 2
  - total_train_batch_size: 24
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - lr_scheduler_warmup_steps: 2000
  - num_epochs: 40.0
  - mixed_precision_training: Native AMP
  
  ### Training results
  
  | Training Loss | Epoch | Step   | Validation Loss | Wer    |
  |:-------------:|:-----:|:------:|:---------------:|:------:|
  | 3.3367        | 0.04  | 500    | 3.4163          | 0.9938 |
  | 2.9738        | 0.08  | 1000   | 2.9497          | 0.9950 |
  | 2.8161        | 0.12  | 1500   | 2.7508          | 1.0139 |
  | 1.6716        | 0.17  | 2000   | 1.2780          | 0.9512 |
  | 0.4444        | 0.21  | 2500   | 0.2886          | 0.3434 |
  | 0.4584        | 0.25  | 3000   | 0.2339          | 0.2520 |
  | 0.3229        | 0.29  | 3500   | 0.2077          | 0.2047 |
  | 0.3201        | 0.33  | 4000   | 0.1753          | 0.1647 |
  | 0.3029        | 0.37  | 4500   | 0.2083          | 0.1939 |
  | 0.2539        | 0.41  | 5000   | 0.1839          | 0.1389 |
  | 0.2613        | 0.46  | 5500   | 0.1561          | 0.1328 |
  | 0.2288        | 0.5   | 6000   | 0.1552          | 0.1404 |
  | 0.2261        | 0.54  | 6500   | 0.1471          | 0.1172 |
  | 0.1993        | 0.58  | 7000   | 0.1663          | 0.1344 |
  | 0.2082        | 0.62  | 7500   | 0.1424          | 0.1086 |
  | 0.1976        | 0.66  | 8000   | 0.1395          | 0.1102 |
  | 0.1854        | 0.7   | 8500   | 0.1331          | 0.1184 |
  | 0.1855        | 0.75  | 9000   | 0.1306          | 0.1049 |
  | 0.1898        | 0.79  | 9500   | 0.1563          | 0.1053 |
  | 0.1874        | 0.83  | 10000  | 0.1361          | 0.1011 |
  | 0.1693        | 0.87  | 10500  | 0.1317          | 0.0975 |
  | 0.1731        | 0.91  | 11000  | 0.1552          | 0.1119 |
  | 0.1739        | 0.95  | 11500  | 0.1256          | 0.0863 |
  | 0.1621        | 0.99  | 12000  | 0.1359          | 0.0890 |
  | 0.1561        | 1.03  | 12500  | 0.1252          | 0.0833 |
  | 0.161         | 1.08  | 13000  | 0.1243          | 0.0802 |
  | 0.1512        | 1.12  | 13500  | 0.1287          | 0.0829 |
  | 0.145         | 1.16  | 14000  | 0.1273          | 0.0809 |
  | 0.1452        | 1.2   | 14500  | 0.1277          | 0.0797 |
  | 0.1506        | 1.24  | 15000  | 0.1358          | 0.0795 |
  | 0.1463        | 1.28  | 15500  | 0.1276          | 0.0774 |
  | 0.1361        | 1.32  | 16000  | 0.1206          | 0.0789 |
  | 0.1405        | 1.37  | 16500  | 0.1303          | 0.0791 |
  | 0.1439        | 1.41  | 17000  | 0.1319          | 0.0804 |
  | 0.1429        | 1.45  | 17500  | 0.1179          | 0.0768 |
  | 0.1299        | 1.49  | 18000  | 0.1380          | 0.0812 |
  | 0.1423        | 1.53  | 18500  | 0.1228          | 0.0768 |
  | 0.1432        | 1.57  | 19000  | 0.1221          | 0.0774 |
  | 0.1261        | 1.61  | 19500  | 0.1309          | 0.0781 |
  | 0.1338        | 1.66  | 20000  | 0.1332          | 0.0761 |
  | 0.1345        | 1.7   | 20500  | 0.1198          | 0.0770 |
  | 0.1312        | 1.74  | 21000  | 0.1226          | 0.0746 |
  | 0.1269        | 1.78  | 21500  | 0.1312          | 0.0759 |
  | 0.1322        | 1.82  | 22000  | 0.1251          | 0.0751 |
  | 0.1329        | 1.86  | 22500  | 0.1192          | 0.0715 |
  | 0.1423        | 1.9   | 23000  | 0.1114          | 0.0718 |
  | 0.1223        | 1.95  | 23500  | 0.1277          | 0.0733 |
  | 0.1278        | 1.99  | 24000  | 0.1287          | 0.0751 |
  | 0.1128        | 2.03  | 24500  | 0.1256          | 0.0702 |
  | 0.112         | 2.07  | 25000  | 0.1258          | 0.0711 |
  | 0.115         | 2.11  | 25500  | 0.1271          | 0.0718 |
  | 0.1131        | 2.15  | 26000  | 0.1223          | 0.0717 |
  | 0.1175        | 2.19  | 26500  | 0.1229          | 0.0725 |
  | 0.1156        | 2.24  | 27000  | 0.1235          | 0.0719 |
  | 0.104         | 2.28  | 27500  | 0.1273          | 0.0698 |
  | 0.1201        | 2.32  | 28000  | 0.1236          | 0.0710 |
  | 0.1182        | 2.36  | 28500  | 0.1191          | 0.0691 |
  | 0.123         | 2.4   | 29000  | 0.1221          | 0.0706 |
  | 0.1244        | 2.44  | 29500  | 0.1154          | 0.071