w2v-bert-2.0-chichewa_34_102h

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the CLEAR-GLOBAL/CHICHEWA_34_102H - NA dataset. It achieves the following results on the evaluation set:

• Loss: 0.2991
• Wer: 0.3874
• Cer: 0.1111

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• training_steps: 100000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
2.2628	0.7877	1000	2.6113	0.9981	0.7664
0.1158	1.5750	2000	0.7048	0.6111	0.1786
0.0535	2.3623	3000	0.5161	0.5307	0.1527
0.0471	3.1497	4000	0.4501	0.4873	0.1434
0.0452	3.9374	5000	0.4284	0.4806	0.1410
0.0277	4.7247	6000	0.3880	0.4649	0.1387
0.0441	5.5120	7000	0.4015	0.4461	0.1294
0.0177	6.2993	8000	0.3798	0.4290	0.1209
0.0198	7.0866	9000	0.3330	0.4027	0.1171
0.0141	7.8744	10000	0.3333	0.4307	0.1213
0.0237	8.6617	11000	0.3653	0.4294	0.1259
0.014	9.4490	12000	0.3118	0.4048	0.1162
0.0079	10.2363	13000	0.2991	0.3874	0.1109
0.0106	11.0236	14000	0.3455	0.4008	0.1193
0.0089	11.8113	15000	0.3658	0.4091	0.1249
0.0068	12.5987	16000	0.3054	0.3918	0.1124
0.007	13.3860	17000	0.3255	0.3785	0.1114
0.0108	14.1733	18000	0.3393	0.4045	0.1152

Framework versions

• Transformers 4.48.1
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1