mHuBERT-147-nyagen-female-42

This model is a fine-tuned version of utter-project/mHuBERT-147 on the NYAGEN - NYA dataset. It achieves the following results on the evaluation set:

• Loss: 2.8660
• Wer: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
7.4055	1.1117	100	2.9290	1.0
2.8696	2.2235	200	2.8770	1.0
2.8609	3.3352	300	2.8685	1.0
2.8599	4.4469	400	2.8804	1.0
2.8595	5.5587	500	2.8648	1.0
2.8609	6.6704	600	2.8640	1.0
2.8649	7.7821	700	2.8774	1.0
2.8584	8.8939	800	2.8657	1.0
2.845	10.0	900	2.8860	1.0
2.8587	11.1117	1000	2.8642	1.0
2.8592	12.2235	1100	2.8651	1.0
2.8606	13.3352	1200	2.8684	1.0
2.856	14.4469	1300	2.8628	1.0
2.8574	15.5587	1400	2.8674	1.0
2.8559	16.6704	1500	2.8634	1.0
2.8563	17.7821	1600	2.8654	1.0
2.8551	18.8939	1700	2.8630	1.0
2.842	20.0	1800	2.8673	1.0
2.8549	21.1117	1900	2.8628	1.0
2.8547	22.2235	2000	2.8640	1.0
2.8547	23.3352	2100	2.8667	1.0
2.8551	24.4469	2200	2.8628	1.0
2.8535	25.5587	2300	2.8675	1.0
2.8557	26.6704	2400	2.8659	1.0
2.8532	27.7821	2500	2.8660	1.0
2.8537	28.8939	2600	2.8639	1.0
2.839	30.0	2700	2.8652	1.0

Framework versions

• Transformers 4.53.0.dev0
• Pytorch 2.6.0+cu124
• Datasets 3.6.0
• Tokenizers 0.21.0