w2v-bert-2.0-hausa_100_400h

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the CLEAR-GLOBAL/HAUSA_100_400H - NA dataset. It achieves the following results on the evaluation set:

• Loss: 0.2569
• Wer: 0.3445
• Cer: 0.1935

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 160
• eval_batch_size: 160
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• total_train_batch_size: 320
• total_eval_batch_size: 320
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 50.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
0.6382	0.7027	1000	0.5262	0.4409	0.2196
0.0711	1.4055	2000	0.3727	0.3948	0.2065
0.0369	2.1082	3000	0.3289	0.3745	0.2023
0.0304	2.8110	4000	0.3093	0.3682	0.1996
0.0284	3.5137	5000	0.3339	0.3751	0.2009
0.0201	4.2164	6000	0.3029	0.3565	0.1965
0.0292	4.9192	7000	0.3215	0.3684	0.1987
0.0147	5.6219	8000	0.2950	0.3583	0.1980
0.014	6.3247	9000	0.2827	0.3561	0.1964
0.03	7.0274	10000	0.2848	0.3521	0.1958
0.0158	7.7301	11000	0.2805	0.3498	0.1947
0.0342	8.4329	12000	0.2716	0.3492	0.1949
0.0307	9.1356	13000	0.2778	0.3489	0.1952
0.0339	9.8384	14000	0.2676	0.3434	0.1933
0.0439	10.5411	15000	0.2674	0.3484	0.1941
0.0998	11.2439	16000	0.2659	0.3475	0.1946
0.1108	11.9466	17000	0.2568	0.3444	0.1934
0.176	12.6493	18000	0.2595	0.3395	0.1922
0.0564	13.3521	19000	0.2613	0.3419	0.1926
0.0147	14.0548	20000	0.2781	0.3450	0.1933
0.0173	14.7576	21000	0.2696	0.3427	0.1924
0.0085	15.4603	22000	0.2596	0.3418	0.1929

Framework versions

• Transformers 4.48.1
• Pytorch 2.6.0+cu126
• Datasets 3.5.0
• Tokenizers 0.21.1