xlsr-big-kynnn

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0000
• Wer: 0.0571

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0004
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 132
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
2.1593	1.6327	200	0.9001	0.6127
0.5422	3.2653	400	0.0827	0.1256
0.1415	4.8980	600	0.0325	0.0792
0.0749	6.5306	800	0.0092	0.0589
0.056	8.1633	1000	0.0181	0.0701
0.0432	9.7959	1200	0.0125	0.0591
0.0412	11.4286	1400	0.0063	0.0615
0.0425	13.0612	1600	0.0070	0.0854
0.0288	14.6939	1800	0.0069	0.0549
0.0308	16.3265	2000	0.0701	0.0802
0.0237	17.9592	2200	0.0009	0.0510
0.0213	19.5918	2400	0.0071	0.0553
0.0261	21.2245	2600	0.0039	0.0529
0.0229	22.8571	2800	0.0034	0.0545
0.0219	24.4898	3000	0.0049	0.0543
0.0176	26.1224	3200	0.0013	0.0549
0.0122	27.7551	3400	0.0013	0.0575
0.014	29.3878	3600	0.0041	0.0571
0.0146	31.0204	3800	0.0046	0.0527
0.0126	32.6531	4000	0.0003	0.0516
0.0099	34.2857	4200	0.0038	0.0561
0.0113	35.9184	4400	0.0009	0.0613
0.0121	37.5510	4600	0.0006	0.0523
0.0119	39.1837	4800	0.0035	0.0541
0.0104	40.8163	5000	0.0057	0.0520
0.0107	42.4490	5200	0.0001	0.0516
0.0076	44.0816	5400	0.0015	0.0531
0.0076	45.7143	5600	0.0008	0.0510
0.0099	47.3469	5800	0.0002	0.0514
0.0076	48.9796	6000	0.0001	0.0543
0.0049	50.6122	6200	0.0001	0.0691
0.0087	52.2449	6400	0.0004	0.0523
0.0082	53.8776	6600	0.0008	0.0567
0.009	55.5102	6800	0.0042	0.0547
0.0065	57.1429	7000	0.0003	0.0541
0.0044	58.7755	7200	0.0000	0.0543
0.0051	60.4082	7400	0.0001	0.0529
0.0058	62.0408	7600	0.0001	0.0508
0.0049	63.6735	7800	0.0003	0.0518
0.0048	65.3061	8000	0.0005	0.0520
0.005	66.9388	8200	0.0001	0.0665
0.0044	68.5714	8400	0.0010	0.0577
0.0041	70.2041	8600	0.0000	0.0535
0.0028	71.8367	8800	0.0000	0.0520
0.0044	73.4694	9000	0.0000	0.0561
0.0025	75.1020	9200	0.0000	0.0591
0.002	76.7347	9400	0.0000	0.0553
0.0032	78.3673	9600	0.0000	0.0512
0.0031	80.0	9800	0.0003	0.0520
0.0023	81.6327	10000	0.0000	0.0512
0.0018	83.2653	10200	0.0000	0.0512
0.0017	84.8980	10400	0.0000	0.0529
0.0011	86.5306	10600	0.0000	0.0520
0.0014	88.1633	10800	0.0000	0.0518
0.0015	89.7959	11000	0.0000	0.0541
0.0012	91.4286	11200	0.0000	0.0523
0.0015	93.0612	11400	0.0000	0.0543
0.0017	94.6939	11600	0.0000	0.0575
0.0008	96.3265	11800	0.0000	0.0581
0.001	97.9592	12000	0.0000	0.0569
0.0011	99.5918	12200	0.0000	0.0571

Framework versions

• Transformers 4.45.0.dev0
• Pytorch 2.1.2
• Datasets 2.20.0
• Tokenizers 0.19.1