mouseyy/uk_wav2vec2_with_stress_mark

Ukrainian Wav2Vec2 Model for Transcription with Lexical Stress Marking (+)

This model is a fine-tuned version of mouseyy/result_data_2-3 on the common_voice_17_0 dataset. It achieves the following results on the evaluation set:

• Loss: 0.2132
• Wer: 0.2984
• Cer: 0.1512

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• total_train_batch_size: 32
• total_eval_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 15.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
0.1936	0.4550	500	0.2082	0.3305	0.1590
0.174	0.9099	1000	0.2082	0.3284	0.1586
0.1855	1.3649	1500	0.1981	0.3292	0.1585
0.1724	1.8198	2000	0.1998	0.3265	0.1580
0.1667	2.2748	2500	0.1994	0.3288	0.1583
0.1635	2.7298	3000	0.2050	0.3223	0.1569
0.14	3.1847	3500	0.2069	0.3205	0.1567
0.1573	3.6397	4000	0.2067	0.3207	0.1569
0.1487	4.0946	4500	0.2101	0.3224	0.1572
0.1501	4.5496	5000	0.2110	0.3176	0.1563
0.1486	5.0045	5500	0.2040	0.3159	0.1557
0.1342	5.4595	6000	0.2041	0.3144	0.1551
0.1396	5.9145	6500	0.2057	0.3143	0.1552
0.136	6.3694	7000	0.2098	0.3131	0.1545
0.1266	6.8244	7500	0.2095	0.3106	0.1542
0.1283	7.2793	8000	0.2160	0.3085	0.1538
0.1229	7.7343	8500	0.2175	0.3076	0.1538
0.1267	8.1893	9000	0.2114	0.3057	0.1531
0.1127	8.6442	9500	0.2063	0.3069	0.1528
0.1165	9.0992	10000	0.2094	0.3048	0.1532
0.1222	9.5541	10500	0.2079	0.3067	0.1532
0.1127	10.0091	11000	0.2089	0.3056	0.1531
0.1084	10.4641	11500	0.2117	0.3032	0.1526
0.1155	10.9190	12000	0.2075	0.3045	0.1527
0.0955	11.3740	12500	0.2183	0.3026	0.1523
0.1146	11.8289	13000	0.2116	0.3015	0.1521
0.1094	12.2839	13500	0.2090	0.2993	0.1516
0.1072	12.7389	14000	0.2124	0.3002	0.1517
0.1125	13.1938	14500	0.2131	0.3000	0.1517
0.1058	13.6488	15000	0.2170	0.2992	0.1515
0.0951	14.1037	15500	0.2160	0.2986	0.1513
0.1035	14.5587	16000	0.2134	0.2986	0.1513

Framework versions

• Transformers 4.49.0
• Pytorch 2.6.0+cu124
• Datasets 3.3.2
• Tokenizers 0.21.0