w2v-bert-2.0-hausa_naijavoices_100h

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the CLEAR-GLOBAL/NAIJAVOICES_100H - NA dataset. It achieves the following results on the evaluation set:

• Loss: 0.2644
• Wer: 0.3398
• Cer: 0.1916

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 160
• eval_batch_size: 160
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• total_train_batch_size: 320
• total_eval_batch_size: 320
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 250.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
0.346	2.7933	1000	0.4367	0.4197	0.2131
0.2972	5.5866	2000	0.3150	0.3691	0.1998
0.2638	8.3799	3000	0.2892	0.3556	0.1959
0.2308	11.1732	4000	0.2728	0.3471	0.1938
0.2338	13.9665	5000	0.2707	0.3430	0.1929
0.2105	16.7598	6000	0.2687	0.3389	0.1917
0.1732	19.5531	7000	0.2710	0.3437	0.1935
0.1638	22.3464	8000	0.2657	0.3426	0.1927
0.1933	25.1397	9000	0.2787	0.3413	0.1918
0.144	27.9330	10000	0.2651	0.3397	0.1916
0.1493	30.7263	11000	0.2757	0.3415	0.1923
0.1267	33.5196	12000	0.2826	0.3482	0.1924
0.1045	36.3128	13000	0.3057	0.3480	0.1930
0.066	39.1061	14000	0.3314	0.3526	0.1942
0.0564	41.8994	15000	0.3840	0.3541	0.1939

Framework versions

• Transformers 4.48.1
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1