whisper-small-lg-CV-v4

This model is a fine-tuned version of openai/whisper-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4866
• Wer: 0.1829
• Cer: 0.0436

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use adamw_hf with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 80
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
1.545	1.0	4442	0.5612	0.8131	0.3503
0.4678	2.0	8884	0.3652	0.8879	0.3107
0.3088	3.0	13326	0.3027	0.5332	0.1688
0.2227	4.0	17768	0.2697	0.5612	0.2191
0.1615	5.0	22210	0.2638	0.2890	0.0834
0.1138	6.0	26652	0.2823	0.3445	0.1247
0.0783	7.0	31094	0.2914	0.2683	0.0754
0.0554	8.0	35536	0.3115	0.2488	0.0658
0.0396	9.0	39978	0.3147	0.2329	0.0524
0.0279	10.0	44420	0.3267	0.2213	0.0499
0.0207	11.0	48862	0.3438	0.2167	0.0494
0.0167	12.0	53304	0.3460	0.2090	0.0449
0.0138	13.0	57746	0.3553	0.2043	0.0447
0.0118	14.0	62188	0.3806	0.2167	0.0486
0.0101	15.0	66630	0.3755	0.2066	0.0463
0.0089	16.0	71072	0.3849	0.2045	0.0450
0.0079	17.0	75514	0.3918	0.1974	0.0447
0.0071	18.0	79956	0.3998	0.1968	0.0435
0.0064	19.0	84398	0.4117	0.2016	0.0442
0.0059	20.0	88840	0.4077	0.1985	0.0443
0.0052	21.0	93282	0.4198	0.1976	0.0434
0.005	22.0	97724	0.4248	0.1964	0.0427
0.0044	23.0	102166	0.4303	0.1964	0.0445
0.0038	24.0	106608	0.4429	0.1970	0.0439
0.0036	25.0	111050	0.4448	0.2014	0.0463
0.0035	26.0	115492	0.4585	0.1952	0.0435
0.003	27.0	119934	0.4450	0.1836	0.0404
0.003	28.0	124376	0.4620	0.1869	0.0409
0.0028	29.0	128818	0.4665	0.1885	0.0401
0.0026	30.0	133260	0.4837	0.1937	0.0440
0.0024	31.0	137702	0.4758	0.1850	0.0399
0.0023	32.0	142144	0.4879	0.1891	0.0418
0.0021	33.0	146586	0.4674	0.1883	0.0414
0.0019	34.0	151028	0.4782	0.1852	0.0413
0.0018	35.0	155470	0.4834	0.1840	0.0411
0.0016	36.0	159912	0.4990	0.1912	0.0428
0.0017	37.0	164354	0.4866	0.1829	0.0436

Framework versions

• Transformers 4.47.0
• Pytorch 2.1.0+cu118
• Datasets 3.1.0
• Tokenizers 0.21.0