w2v-bert-2.0-yogera-fleurs-cv-lg-10hrs-v1

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the None dataset. It achieves the following results on the evaluation set:

• Loss: inf
• Wer: 0.3393
• Cer: 0.0784

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Cer	Validation Loss	Wer
0.8447	1.0	442	0.1168	inf	0.4979
0.4024	2.0	884	0.0901	inf	0.4103
0.3074	3.0	1326	0.0869	inf	0.3889
0.2534	4.0	1768	0.0919	inf	0.4043
0.2113	5.0	2210	0.0852	inf	0.3766
0.1793	6.0	2652	0.0767	inf	0.3569
0.1581	7.0	3094	0.0878	inf	0.3718
0.141	8.0	3536	0.0794	inf	0.3591
0.1226	9.0	3978	0.0766	inf	0.3461
0.108	10.0	4420	0.0847	inf	0.3761
0.0942	11.0	4862	0.0827	inf	0.3667
0.0836	12.0	5304	0.0776	inf	0.3536
0.074	13.0	5746	0.0833	inf	0.3731
0.0719	14.0	6188	0.0856	inf	0.3886
0.0643	15.0	6630	0.0849	inf	0.3744
0.0597	16.0	7072	0.0788	inf	0.3532
0.0612	17.0	7514	0.0862	inf	0.3818
0.064	18.0	7956	0.0893	inf	0.3918
0.0705	19.0	8398	0.0898	inf	0.4048
0.047	20.0	8840	0.0779	inf	0.3518
0.0365	21.0	9282	inf	0.3493	0.0773
0.0346	22.0	9724	inf	0.3370	0.0747
0.0318	23.0	10166	inf	0.3435	0.0781
0.024	24.0	10608	inf	0.3322	0.0751
0.0265	25.0	11050	inf	0.3460	0.0778
0.0252	26.0	11492	inf	0.3466	0.0784
0.0229	27.0	11934	inf	0.3449	0.0770
0.0226	28.0	12376	inf	0.3384	0.0771
0.0201	29.0	12818	inf	0.3336	0.0755
0.0183	30.0	13260	inf	0.3332	0.0762
0.0174	31.0	13702	inf	0.3283	0.0745
0.0161	32.0	14144	inf	0.3308	0.0744
0.0157	33.0	14586	inf	0.3299	0.0767
0.0153	34.0	15028	inf	0.3385	0.0751
0.0133	35.0	15470	inf	0.3397	0.0787
0.0136	36.0	15912	inf	0.3382	0.0771
0.0139	37.0	16354	inf	0.3318	0.0761
0.0168	38.0	16796	inf	0.3409	0.0765
0.0138	39.0	17238	inf	0.3381	0.0769
0.0198	40.0	17680	inf	0.3522	0.0807
0.0204	41.0	18122	inf	0.3393	0.0784

Framework versions

• Transformers 4.45.2
• Pytorch 2.1.0+cu118
• Datasets 3.0.1
• Tokenizers 0.20.1