w2v-bert-2.0-hausa_naijavoices_500h

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the CLEAR-GLOBAL/NAIJAVOICES_500H - NA dataset. It achieves the following results on the evaluation set:

• Loss: 0.2268
• Wer: 0.3250
• Cer: 0.1875

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 160
• eval_batch_size: 160
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• total_train_batch_size: 320
• total_eval_batch_size: 320
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 50.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
0.7206	0.5587	1000	0.4335	0.4167	0.2119
0.2797	1.1173	2000	0.3227	0.3731	0.1999
0.2159	1.6760	3000	0.2890	0.3517	0.1951
0.2734	2.2346	4000	0.2733	0.3458	0.1932
0.1894	2.7933	5000	0.2602	0.3431	0.1917
0.2076	3.3520	6000	0.2577	0.3413	0.1918
0.2168	3.9106	7000	0.2481	0.3386	0.1911
0.1709	4.4693	8000	0.2492	0.3407	0.1913
0.3026	5.0279	9000	0.2520	0.3386	0.1914
0.2794	5.5866	10000	0.2597	0.3395	0.1910
0.1957	6.1453	11000	0.2412	0.3370	0.1903
0.1605	6.7039	12000	0.2393	0.3319	0.1893
0.1902	7.2626	13000	0.2426	0.3335	0.1896
0.1637	7.8212	14000	0.2348	0.3319	0.1888
0.1693	8.3799	15000	0.2320	0.3289	0.1885
0.1584	8.9385	16000	0.2341	0.3298	0.1885
0.1671	9.4972	17000	0.2274	0.3253	0.1876
0.244	10.0559	18000	0.2337	0.3305	0.1885
0.2167	10.6145	19000	0.2335	0.3264	0.1877
0.165	11.1732	20000	0.2276	0.3269	0.1875
0.1679	11.7318	21000	0.2303	0.3322	0.1886
0.1746	12.2905	22000	0.2277	0.3247	0.1867

Framework versions

• Transformers 4.48.1
• Pytorch 2.6.0+cu126
• Datasets 3.5.0
• Tokenizers 0.21.1