w2v-bert-2.0-chichewa_34h-v2

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the CLEAR-GLOBAL/CHICHEWA_34H - NA dataset. It achieves the following results on the evaluation set:

• Loss: 0.3425
• Wer: 0.4186
• Cer: 0.1164

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.025
• training_steps: 100000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
0.2324	5.6197	1000	0.4302	0.4770	0.1334
0.053	11.2366	2000	0.3425	0.4186	0.1163
0.0562	16.8563	3000	0.3435	0.3949	0.1094
0.0138	22.4732	4000	0.4020	0.3878	0.1101
0.0319	28.0901	5000	0.4283	0.3707	0.1063
0.0068	33.7099	6000	0.5047	0.3828	0.1078
0.0223	39.3268	7000	0.4749	0.3638	0.1044

Framework versions

• Transformers 4.48.1
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1