---
library_name: transformers
license: apache-2.0
base_model: facebook/wav2vec2-xls-r-300m
tags:
- generated_from_trainer
metrics:
- wer
model-index:
- name: wav2vec2-xls-r-300m-cv7-istech
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-xls-r-300m-cv7-istech

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7978
- Wer: 0.7496

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 20
- num_epochs: 5
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch  | Step | Validation Loss | Wer    |
|:-------------:|:------:|:----:|:---------------:|:------:|
| 11.1946       | 0.0803 | 20   | 6.7455          | 1.0    |
| 5.1957        | 0.1606 | 40   | 3.7715          | 1.0    |
| 3.5889        | 0.2410 | 60   | 3.4781          | 1.0    |
| 3.3656        | 0.3213 | 80   | 3.3373          | 1.0    |
| 3.5262        | 0.4016 | 100  | 3.3093          | 1.0    |
| 3.1816        | 0.4819 | 120  | 3.2444          | 1.0    |
| 3.3344        | 0.5622 | 140  | 3.2887          | 1.0    |
| 3.2522        | 0.6426 | 160  | 3.2321          | 1.0    |
| 3.1819        | 0.7229 | 180  | 3.2538          | 1.0    |
| 3.3665        | 0.8032 | 200  | 3.2018          | 1.0    |
| 3.1257        | 0.8835 | 220  | 3.1751          | 1.0    |
| 3.2982        | 0.9639 | 240  | 3.1929          | 1.0    |
| 3.2181        | 1.0442 | 260  | 3.1781          | 1.0    |
| 3.1455        | 1.1245 | 280  | 3.3508          | 1.0    |
| 3.2722        | 1.2048 | 300  | 3.2343          | 1.0    |
| 3.1217        | 1.2851 | 320  | 3.1378          | 0.9999 |
| 3.2419        | 1.3655 | 340  | 3.2108          | 0.9999 |
| 3.1825        | 1.4458 | 360  | 3.1363          | 0.9999 |
| 3.1316        | 1.5261 | 380  | 3.1685          | 0.9999 |
| 3.2373        | 1.6064 | 400  | 3.1731          | 1.0    |
| 3.0634        | 1.6867 | 420  | 3.1083          | 0.9999 |
| 3.1367        | 1.7671 | 440  | 3.1582          | 0.9999 |
| 3.0831        | 1.8474 | 460  | 3.0410          | 0.9999 |
| 2.9524        | 1.9277 | 480  | 2.9578          | 0.9999 |
| 3.1076        | 2.0080 | 500  | 2.9558          | 0.9999 |
| 2.8085        | 2.0884 | 520  | 2.8155          | 0.9999 |
| 2.7768        | 2.1687 | 540  | 2.8155          | 0.9998 |
| 2.6186        | 2.2490 | 560  | 2.5386          | 1.0    |
| 2.3452        | 2.3293 | 580  | 2.3387          | 0.9994 |
| 2.4213        | 2.4096 | 600  | 2.2745          | 0.9997 |
| 1.8149        | 2.4900 | 620  | 1.8076          | 0.9906 |
| 1.747         | 2.5703 | 640  | 1.8517          | 0.9962 |
| 1.6369        | 2.6506 | 660  | 1.4838          | 0.9662 |
| 1.3732        | 2.7309 | 680  | 1.4990          | 0.9852 |
| 1.6638        | 2.8112 | 700  | 1.3854          | 0.9383 |
| 1.1034        | 2.8916 | 720  | 1.2258          | 0.9112 |
| 1.3003        | 2.9719 | 740  | 1.3696          | 0.9174 |
| 1.2128        | 3.0522 | 760  | 1.1576          | 0.8966 |
| 0.9759        | 3.1325 | 780  | 1.1325          | 0.8838 |
| 1.2551        | 3.2129 | 800  | 1.1489          | 0.8831 |
| 0.8371        | 3.2932 | 820  | 1.0224          | 0.8460 |
| 1.0434        | 3.3735 | 840  | 1.0927          | 0.8680 |
| 1.0386        | 3.4538 | 860  | 0.9731          | 0.8383 |
| 0.879         | 3.5341 | 880  | 0.9916          | 0.8428 |
| 1.1771        | 3.6145 | 900  | 0.9857          | 0.8406 |
| 0.6948        | 3.6948 | 920  | 0.9493          | 0.8203 |
| 0.9657        | 3.7751 | 940  | 0.9960          | 0.8364 |
| 0.8991        | 3.8554 | 960  | 0.8949          | 0.8013 |
| 0.7712        | 3.9357 | 980  | 0.9039          | 0.8096 |
| 1.0684        | 4.0161 | 1000 | 0.9024          | 0.8012 |
| 0.5559        | 4.0964 | 1020 | 0.8611          | 0.7872 |
| 0.8446        | 4.1767 | 1040 | 0.9017          | 0.8021 |
| 0.7911        | 4.2570 | 1060 | 0.8596          | 0.7848 |
| 0.6178        | 4.3373 | 1080 | 0.8612          | 0.7778 |
| 0.9147        | 4.4177 | 1100 | 0.8654          | 0.7756 |
| 0.5448        | 4.4980 | 1120 | 0.8222          | 0.7649 |
| 0.7812        | 4.5783 | 1140 | 0.8337          | 0.7697 |
| 0.6784        | 4.6586 | 1160 | 0.8146          | 0.7588 |
| 0.6022        | 4.7390 | 1180 | 0.8077          | 0.7538 |
| 0.8592        | 4.8193 | 1200 | 0.8121          | 0.7621 |
| 0.4884        | 4.8996 | 1220 | 0.7982          | 0.7487 |
| 0.7429        | 4.9799 | 1240 | 0.7978          | 0.7496 |


### Framework versions

- Transformers 4.45.2
- Pytorch 2.5.1
- Datasets 2.19.1
- Tokenizers 0.20.1