mms-1b-all-bemgen-female-adv-52

This model is a fine-tuned version of facebook/mms-1b-all on the BEMGEN - BEM dataset. It achieves the following results on the evaluation set:

• Loss: 0.2930
• Wer: 0.4244

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 4
• seed: 52
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
6.9581	0.4926	100	3.2458	1.1902
3.1144	0.9852	200	2.8261	0.9746
2.8743	1.4778	300	2.7030	0.9861
2.7704	1.9704	400	2.6256	0.9984
2.5944	2.4631	500	2.1930	0.9962
1.0912	2.9557	600	0.3834	0.5565
0.3683	3.4483	700	0.3502	0.4739
0.3258	3.9409	800	0.3427	0.5147
0.3071	4.4335	900	0.3270	0.4581
0.2899	4.9261	1000	0.3229	0.4407
0.2719	5.4187	1100	0.3153	0.4382
0.26	5.9113	1200	0.3114	0.4229
0.2495	6.4039	1300	0.3093	0.4438
0.2539	6.8966	1400	0.3007	0.4322
0.2431	7.3892	1500	0.3070	0.4288
0.2328	7.8818	1600	0.3056	0.4296
0.2327	8.3744	1700	0.3000	0.4247
0.2264	8.8670	1800	0.3089	0.4230
0.2146	9.3596	1900	0.3086	0.4643
0.2133	9.8522	2000	0.2930	0.4242
0.2162	10.3448	2100	0.2960	0.4137
0.2082	10.8374	2200	0.2998	0.4147
0.2093	11.3300	2300	0.2990	0.4188

Framework versions

• Transformers 4.53.0.dev0
• Pytorch 2.6.0+cu124
• Datasets 3.6.0
• Tokenizers 0.21.0