w2v-bert-2.0-hausa_naijavoices_250h

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the CLEAR-GLOBAL/NAIJAVOICES_250H - NA dataset. It achieves the following results on the evaluation set:

• Loss: 0.2333
• Wer: 0.3302
• Cer: 0.1886

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 160
• eval_batch_size: 160
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• total_train_batch_size: 320
• total_eval_batch_size: 320
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
0.3125	1.1173	1000	0.4133	0.4180	0.2128
0.2781	2.2346	2000	0.3208	0.3702	0.1998
0.2616	3.3520	3000	0.2858	0.3534	0.1950
0.2068	4.4693	4000	0.2772	0.3420	0.1923
0.3346	5.5866	5000	0.2620	0.3419	0.1915
0.2176	6.7039	6000	0.2535	0.3410	0.1916
0.1969	7.8212	7000	0.2466	0.3380	0.1909
0.1992	8.9385	8000	0.2467	0.3322	0.1896
0.197	10.0559	9000	0.2489	0.3322	0.1893
0.1832	11.1732	10000	0.2533	0.3385	0.1909
0.1859	12.2905	11000	0.2410	0.3334	0.1898
0.1751	13.4078	12000	0.2419	0.3307	0.1886
0.1569	14.5251	13000	0.2418	0.3321	0.1895
0.2345	15.6425	14000	0.2480	0.3352	0.1898
0.1769	16.7598	15000	0.2365	0.3280	0.1889
0.1634	17.8771	16000	0.2401	0.3345	0.1895
0.1647	18.9944	17000	0.2329	0.3303	0.1888
0.1937	20.1117	18000	0.2485	0.3273	0.1881
0.1671	21.2291	19000	0.2568	0.3327	0.1891
0.1229	22.3464	20000	0.2460	0.3304	0.1887
0.1252	23.4637	21000	0.2437	0.3337	0.1889
0.1186	24.5810	22000	0.2544	0.3309	0.1892

Framework versions

• Transformers 4.48.1
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1