mms-1b-naijavoices_50h-hau-ft

This model is a fine-tuned version of facebook/mms-1b-all on the /MNT/MD0/SYNVOICES/DATA/NAIJAVOICES_50H - NA dataset. It achieves the following results on the evaluation set:

• Loss: 0.3184
• Wer: 0.3325
• Cer: 0.0842

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• total_eval_batch_size: 16
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 20.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
0.4365	0.2793	500	0.3954	0.4125	0.1067
0.5478	0.5587	1000	0.3881	0.4025	0.1031
0.5289	0.8380	1500	0.3787	0.3857	0.0984
0.5134	1.1173	2000	0.3760	0.4047	0.1008
0.429	1.3966	2500	0.3699	0.3851	0.0977
0.6624	1.6760	3000	0.3696	0.3918	0.0995
0.4997	1.9553	3500	0.3674	0.3797	0.0969
0.3572	2.2346	4000	0.3659	0.3815	0.0972
0.4281	2.5140	4500	0.3614	0.3761	0.0962
0.4535	2.7933	5000	0.3592	0.3739	0.0950
0.5143	3.0726	5500	0.3574	0.3709	0.0938
0.3061	3.3520	6000	0.3541	0.3714	0.0941
0.4699	3.6313	6500	0.3576	0.3711	0.0941
0.4345	3.9106	7000	0.3578	0.3765	0.0947
0.5514	4.1899	7500	0.3549	0.3738	0.0944
0.4208	4.4693	8000	0.3547	0.3665	0.0929
0.4893	4.7486	8500	0.3532	0.3626	0.0919
0.3894	5.0279	9000	0.3561	0.3741	0.0949
0.4344	5.3073	9500	0.3531	0.3661	0.0939
0.5008	5.5866	10000	0.3484	0.3695	0.0932
0.4309	5.8659	10500	0.3469	0.3620	0.0916
0.5366	6.1453	11000	0.3471	0.3613	0.0917
0.5179	6.4246	11500	0.3529	0.3804	0.0949
0.7649	6.7039	12000	0.3450	0.3647	0.0916
0.3632	6.9832	12500	0.3444	0.3577	0.0907
0.4515	7.2626	13000	0.3405	0.3603	0.0914
0.5776	7.5419	13500	0.3409	0.3519	0.0893
0.4488	7.8212	14000	0.3415	0.3652	0.0916
0.4377	8.1006	14500	0.3466	0.3717	0.0922
0.4987	8.3799	15000	0.3384	0.3586	0.0902
0.4934	8.6592	15500	0.3394	0.3705	0.0926
0.3812	8.9385	16000	0.3387	0.3548	0.0899
0.5597	9.2179	16500	0.3365	0.3492	0.0883
0.3932	9.4972	17000	0.3347	0.3496	0.0886
0.4425	9.7765	17500	0.3368	0.3538	0.0891
0.3765	10.0559	18000	0.3334	0.3524	0.0887
0.4459	10.3352	18500	0.3355	0.3544	0.0891
0.417	10.6145	19000	0.3346	0.3530	0.0893
0.3967	10.8939	19500	0.3288	0.3509	0.0881
0.3371	11.1732	20000	0.3355	0.3480	0.0880
0.3387	11.4525	20500	0.3321	0.3454	0.0874
0.4536	11.7318	21000	0.3311	0.3475	0.0879
0.3799	12.0112	21500	0.3309	0.3437	0.0874
0.3194	12.2905	22000	0.3336	0.3434	0.0873
0.3527	12.5698	22500	0.3323	0.3423	0.0870
0.4748	12.8492	23000	0.3306	0.3402	0.0865
0.462	13.1285	23500	0.3285	0.3444	0.0870
0.5346	13.4078	24000	0.3315	0.3398	0.0864
0.3255	13.6872	24500	0.3273	0.3434	0.0867
0.3471	13.9665	25000	0.3287	0.3399	0.0863
0.3338	14.2458	25500	0.3270	0.3405	0.0862
0.4103	14.5251	26000	0.3259	0.3396	0.0857
0.5109	14.8045	26500	0.3254	0.3407	0.0862
0.473	15.0838	27000	0.3247	0.3409	0.0860
0.3641	15.3631	27500	0.3251	0.3391	0.0860
0.4245	15.6425	28000	0.3225	0.3384	0.0856
0.3648	15.9218	28500	0.3235	0.3416	0.0861
0.339	16.2011	29000	0.3217	0.3395	0.0856
0.3687	16.4804	29500	0.3221	0.3368	0.0853
0.4329	16.7598	30000	0.3233	0.3411	0.0861
0.5613	17.0391	30500	0.3211	0.3347	0.0848
0.4392	17.3184	31000	0.3217	0.3330	0.0845
0.4039	17.5978	31500	0.3200	0.3386	0.0853
0.4111	17.8771	32000	0.3201	0.3333	0.0843
0.3852	18.1564	32500	0.3206	0.3336	0.0843
0.6218	18.4358	33000	0.3204	0.3347	0.0848
0.4763	18.7151	33500	0.3193	0.3354	0.0848
0.3489	18.9944	34000	0.3186	0.3347	0.0844
0.3661	19.2737	34500	0.3195	0.3343	0.0844
0.3567	19.5531	35000	0.3191	0.3339	0.0843
0.3628	19.8324	35500	0.3184	0.3325	0.0842

Framework versions

• Transformers 4.48.1
• Pytorch 2.5.1+cu121
• Datasets 3.2.0
• Tokenizers 0.21.0