w2v-bert-2.0-igbo_naijavoices_1m

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the NAIJAVOICES_IGBO_1M - NA dataset. It achieves the following results on the evaluation set:

• Loss: 3.4809
• Wer: 1.0422
• Cer: 0.7654

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 160
• eval_batch_size: 160
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 320
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 1500000.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
3.3416	100.0	100	7.2459	1.0182	1.9933
3.2958	200.0	200	7.1312	1.0604	1.9640
3.1829	300.0	300	6.9248	1.2915	1.8969
3.0225	400.0	400	6.6020	1.8023	1.6681
2.7618	500.0	500	6.0535	1.6841	1.0481
2.3916	600.0	600	5.1821	0.9990	0.9265
2.0567	700.0	700	4.3341	1.0	0.9990
1.893	800.0	800	3.9690	1.0	0.9998
1.732	900.0	900	3.7480	0.9956	0.9750
1.5905	1000.0	1000	3.5703	0.9920	0.8322
1.4979	1100.0	1100	3.5028	1.0118	0.7842
1.4103	1200.0	1200	3.4832	1.0259	0.7739
1.3359	1300.0	1300	3.4809	1.0409	0.7656
1.2585	1400.0	1400	3.4905	1.0531	0.7611
1.1585	1500.0	1500	3.5001	1.0673	0.7592
1.0887	1600.0	1600	3.5293	1.0837	0.7582
1.0196	1700.0	1700	3.5624	1.0921	0.7587
0.9474	1800.0	1800	3.5983	1.1052	0.7618

Framework versions

• Transformers 4.48.1
• Pytorch 2.7.1+cu126
• Datasets 4.0.0
• Tokenizers 0.21.2