AfriHuBERT-nyagen-female-62

This model is a fine-tuned version of ajesujoba/AfriHuBERT on the NYAGEN - NYA dataset. It achieves the following results on the evaluation set:

• Loss: 2.9280
• Wer: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 4
• seed: 62
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
13.7875	1.1117	100	9.1052	1.0
7.8647	2.2235	200	7.7800	1.0
7.0846	3.3352	300	7.0396	1.0
6.412	4.4469	400	6.3844	1.0
5.8098	5.5587	500	5.7318	1.0
5.2621	6.6704	600	5.1867	1.0
4.801	7.7821	700	4.7182	1.0
4.3883	8.8939	800	4.3299	1.0
4.0244	10.0	900	3.9940	1.0
3.7806	11.1117	1000	3.7389	1.0
3.5478	12.2235	1100	3.5239	1.0
3.3912	13.3352	1200	3.3683	1.0
3.2542	14.4469	1300	3.2510	1.0
3.1659	15.5587	1400	3.1533	1.0
3.0904	16.6704	1500	3.0935	1.0
3.0407	17.7821	1600	3.0407	1.0
2.9971	18.8939	1700	3.0047	1.0
2.9567	20.0	1800	2.9820	1.0
2.9519	21.1117	1900	2.9651	1.0
2.9356	22.2235	2000	2.9540	1.0
2.9285	23.3352	2100	2.9461	1.0
2.9216	24.4469	2200	2.9379	1.0
2.9159	25.5587	2300	2.9352	1.0
2.9116	26.6704	2400	2.9326	1.0
2.9087	27.7821	2500	2.9297	1.0
2.9097	28.8939	2600	2.9291	1.0
2.8922	30.0	2700	2.9284	1.0

Framework versions

• Transformers 4.53.0.dev0
• Pytorch 2.6.0+cu124
• Datasets 3.6.0
• Tokenizers 0.21.0