w2v-bert-2.0-hausa_579_993h_yourtts

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the CLEAR-GLOBAL/HAUSA_579_993H_YOURTTS - NA dataset. It achieves the following results on the evaluation set:

• Loss: 0.2249
• Wer: 0.3234
• Cer: 0.1866

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 160
• eval_batch_size: 160
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• total_train_batch_size: 320
• total_eval_batch_size: 320
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 16.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
0.6846	0.2491	1000	0.4530	0.4190	0.2127
0.4048	0.4981	2000	0.3258	0.3781	0.2019
0.4272	0.7472	3000	0.2893	0.3688	0.1988
0.3224	0.9963	4000	0.2852	0.3530	0.1947
0.0543	1.2453	5000	0.2782	0.3550	0.1956
0.0747	1.4944	6000	0.2941	0.3611	0.1983
0.0416	1.7435	7000	0.2793	0.3421	0.1931
0.0476	1.9925	8000	0.2707	0.3435	0.1928
0.1736	2.2416	9000	0.2480	0.3365	0.1909
0.1951	2.4907	10000	0.2494	0.3372	0.1906
0.2139	2.7397	11000	0.2422	0.3349	0.1899
0.1895	2.9888	12000	0.2422	0.3332	0.1894
0.0223	3.2379	13000	0.2406	0.3336	0.1899
0.0201	3.4869	14000	0.2428	0.3294	0.1888
0.0169	3.7360	15000	0.2421	0.3312	0.1895
0.0282	3.9851	16000	0.2356	0.3330	0.1899
0.1346	4.2341	17000	0.2405	0.3331	0.1901
0.1021	4.4832	18000	0.2305	0.3276	0.1885
0.1164	4.7323	19000	0.2317	0.3324	0.1894
0.1136	4.9813	20000	0.2372	0.3311	0.1890
0.2131	5.2304	21000	0.2394	0.3278	0.1887
0.2363	5.4795	22000	0.2289	0.3255	0.1879
0.2423	5.7285	23000	0.2250	0.3253	0.1879
0.2533	5.9776	24000	0.2248	0.3237	0.1867
0.0307	6.2267	25000	0.2300	0.3268	0.1880
0.0322	6.4757	26000	0.2266	0.3262	0.1872
0.0346	6.7248	27000	0.2257	0.3281	0.1889
0.055	6.9738	28000	0.2315	0.3246	0.1875
0.1326	7.2229	29000	0.2262	0.3255	0.1875

Framework versions

• Transformers 4.48.1
• Pytorch 2.7.0+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1