w2v-bert-2.0-hausa_250_250h-v2

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the CLEAR-GLOBAL/HAUSA_250_250H - NA dataset. It achieves the following results on the evaluation set:

• Loss: 0.2342
• Wer: 0.3272
• Cer: 0.1883

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 160
• eval_batch_size: 160
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• total_train_batch_size: 320
• total_eval_batch_size: 320
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 50.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Cer	Validation Loss	Wer
0.7042	0.6406	1000	0.2138	0.4636	0.4197
0.1899	1.2812	2000	0.2016	0.3279	0.3774
0.1452	1.9218	3000	0.3020	0.3606	0.1979
0.0677	2.5625	4000	0.2921	0.3504	0.1944
0.2191	3.2031	5000	0.2762	0.3494	0.1949
0.179	3.8437	6000	0.2679	0.3459	0.1931
0.0563	4.4843	7000	0.2740	0.3457	0.1932
0.2152	5.1249	8000	0.2632	0.3475	0.1930
0.1954	5.7655	9000	0.2553	0.3437	0.1928
0.0463	6.4061	10000	0.2516	0.3383	0.1919
0.1641	7.0468	11000	0.2535	0.3350	0.1909
0.2033	7.6874	12000	0.2436	0.3362	0.1907
0.0648	8.3280	13000	0.2619	0.3364	0.1907
0.1271	8.9686	14000	0.2534	0.3392	0.1907
0.1678	9.6092	15000	0.2463	0.3328	0.1899
0.0971	10.2498	16000	0.2505	0.3353	0.1907
0.0785	10.8905	17000	0.2412	0.3334	0.1901
0.0536	11.5311	18000	0.2420	0.3323	0.1895
0.1352	12.1717	19000	0.2458	0.3341	0.1902
0.1101	12.8123	20000	0.2385	0.3339	0.1896
0.0352	13.4529	21000	0.2454	0.3297	0.1890
0.1061	14.0935	22000	0.2433	0.3271	0.1885
0.1361	14.7341	23000	0.2415	0.3343	0.1893
0.0583	15.3748	24000	0.2517	0.3347	0.1896
0.1463	16.0154	25000	0.2341	0.3274	0.1884
0.1579	16.6560	26000	0.2450	0.3292	0.1887
0.0675	17.2966	27000	0.2523	0.3293	0.1888
0.0609	17.9372	28000	0.2474	0.3297	0.1887
0.0624	18.5778	29000	0.2439	0.3285	0.1883
0.079	19.2184	30000	0.2480	0.3292	0.1888

Framework versions

• Transformers 4.48.1
• Pytorch 2.6.0+cu126
• Datasets 3.5.0
• Tokenizers 0.21.1