xls-r-1b-bigcgen-combined-10hrs-model

This model is a fine-tuned version of facebook/wav2vec2-xls-r-1b on the BIGCGEN - NA dataset. It achieves the following results on the evaluation set:

• Loss: 0.5381
• Wer: 0.5532

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30.0

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
No log	0.2114	100	3.6878	1.0
No log	0.4228	200	2.6383	1.0
No log	0.6342	300	0.9092	0.9117
No log	0.8457	400	0.7955	0.7866
5.7153	1.0571	500	0.7448	0.8103
5.7153	1.2685	600	0.7003	0.7530
5.7153	1.4799	700	0.6151	0.6454
5.7153	1.6913	800	0.6108	0.6133
5.7153	1.9027	900	0.5825	0.6153
1.3819	2.1142	1000	0.5743	0.5955
1.3819	2.3256	1100	0.5598	0.5770
1.3819	2.5370	1200	0.5867	0.6105
1.3819	2.7484	1300	0.5734	0.5588
1.3819	2.9598	1400	0.5357	0.5405
1.1226	3.1712	1500	0.5339	0.5415
1.1226	3.3827	1600	0.5582	0.5586
1.1226	3.5941	1700	0.5485	0.5572
1.1226	3.8055	1800	0.5342	0.5404
1.1226	4.0169	1900	0.5381	0.5532

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0