Hugging Face's logo

cantillation
/
Teamim-Large-v3-Turbo_WeightDecay-0.005_Augmented_WithSRT_date-18-04-2025

Automatic Speech Recognition
Transformers
TensorBoard
Safetensors
Hebrew
whisper
hf-asr-leaderboard
Generated from Trainer
Model card Files Metrics Community
Teamim-Large-v3-Turbo_WeightDecay-0.005_Augmented_WithSRT_date-18-04-2025 / README.md
cantillation's picture
cantillation
End of training
9096449 verified
preview code
|
raw
history blame contribute delete
28 kB
metadata
library_name: transformers
language:
  - he
license: mit
base_model: openai/whisper-Large-v3-Turbo
tags:
  - hf-asr-leaderboard
  - generated_from_trainer
metrics:
  - wer
model-index:
  - name: he-cantillation
    results: []

he-cantillation

This model is a fine-tuned version of openai/whisper-Large-v3-Turbo on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.4731
  • Wer: 37.9958
  • Avg Precision Exact: 0.4726
  • Avg Recall Exact: 0.4821
  • Avg F1 Exact: 0.4762
  • Avg Precision Letter Shift: 0.4946
  • Avg Recall Letter Shift: 0.5062
  • Avg F1 Letter Shift: 0.4989
  • Avg Precision Word Level: 0.5073
  • Avg Recall Word Level: 0.5201
  • Avg F1 Word Level: 0.5116
  • Avg Precision Word Shift: 0.6840
  • Avg Recall Word Shift: 0.7091
  • Avg F1 Word Shift: 0.6925
  • Precision Median Exact: 0.35
  • Recall Median Exact: 0.3709
  • F1 Median Exact: 0.3577
  • Precision Max Exact: 1.0
  • Recall Max Exact: 1.0
  • F1 Max Exact: 1.0
  • Precision Min Exact: 0.0
  • Recall Min Exact: 0.0
  • F1 Min Exact: 0.0
  • Precision Min Letter Shift: 0.0
  • Recall Min Letter Shift: 0.0
  • F1 Min Letter Shift: 0.0
  • Precision Min Word Level: 0.0
  • Recall Min Word Level: 0.0
  • F1 Min Word Level: 0.0
  • Precision Min Word Shift: 0.0
  • Recall Min Word Shift: 0.0
  • F1 Min Word Shift: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 16
  • eval_batch_size: 2
  • seed: 42
  • optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 1000
  • training_steps: 80000
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer Avg Precision Exact Avg Recall Exact Avg F1 Exact Avg Precision Letter Shift Avg Recall Letter Shift Avg F1 Letter Shift Avg Precision Word Level Avg Recall Word Level Avg F1 Word Level Avg Precision Word Shift Avg Recall Word Shift Avg F1 Word Shift Precision Median Exact Recall Median Exact F1 Median Exact Precision Max Exact Recall Max Exact F1 Max Exact Precision Min Exact Recall Min Exact F1 Min Exact Precision Min Letter Shift Recall Min Letter Shift F1 Min Letter Shift Precision Min Word Level Recall Min Word Level F1 Min Word Level Precision Min Word Shift Recall Min Word Shift F1 Min Word Shift
No log 0.0001 1 6.2394 110.8685 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0.1211 0.2962 2500 0.7739 60.3892 0.2896 0.3023 0.2948 0.3230 0.3381 0.3289 0.3400 0.3560 0.3460 0.5354 0.5766 0.5518 0.1538 0.1732 0.1611 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0734 0.5925 5000 0.6116 52.4895 0.3503 0.3636 0.3556 0.3787 0.3954 0.3854 0.3946 0.4131 0.4020 0.5867 0.6260 0.6026 0.2 0.2188 0.2064 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0701 0.8887 7500 0.5577 49.6667 0.3772 0.3910 0.3826 0.4071 0.4245 0.4137 0.4220 0.4407 0.4287 0.6095 0.6498 0.6242 0.2294 0.25 0.2371 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0455 1.1850 10000 0.5711 49.5251 0.3459 0.3571 0.3502 0.3737 0.3881 0.3792 0.3910 0.4083 0.3973 0.5849 0.6229 0.5993 0.1786 0.2 0.1885 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0426 1.4812 12500 0.5363 47.2362 0.3891 0.3960 0.3916 0.4174 0.4267 0.4208 0.4314 0.4450 0.4364 0.6261 0.6527 0.6360 0.2343 0.25 0.2407 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0639 1.7775 15000 0.5074 45.9014 0.4086 0.4159 0.4110 0.4382 0.4475 0.4411 0.4531 0.4635 0.4563 0.6393 0.6673 0.6493 0.25 0.2710 0.2568 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0197 2.0737 17500 0.5560 47.4725 0.3848 0.3992 0.3897 0.4105 0.4288 0.4169 0.4245 0.4459 0.4320 0.6001 0.6449 0.6157 0.2045 0.2353 0.2171 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0185 2.3699 20000 0.5010 45.9830 0.4033 0.4160 0.4083 0.4301 0.4455 0.4360 0.4452 0.4636 0.4517 0.6296 0.6678 0.6443 0.2347 0.2632 0.2455 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.035 2.6662 22500 0.4793 44.1070 0.4046 0.4156 0.4085 0.4318 0.4452 0.4367 0.4469 0.4620 0.4518 0.6434 0.6753 0.6545 0.2630 0.2873 0.2692 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0125 2.9624 25000 0.4838 43.4315 0.4177 0.4315 0.4231 0.4443 0.4613 0.4509 0.4589 0.4782 0.4661 0.6398 0.6802 0.6556 0.2650 0.2948 0.2748 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0141 3.2587 27500 0.4849 42.0296 0.4273 0.4390 0.4319 0.4507 0.4649 0.4562 0.4636 0.4790 0.4694 0.6480 0.6827 0.6611 0.2857 0.3150 0.3014 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0458 3.5549 30000 0.4973 45.4608 0.4103 0.4230 0.4150 0.4362 0.4511 0.4413 0.4489 0.4653 0.4545 0.6252 0.6608 0.6377 0.2411 0.2620 0.2483 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0229 3.8512 32500 0.4870 43.0668 0.4337 0.4459 0.4384 0.4585 0.4739 0.4644 0.4721 0.4896 0.4786 0.6519 0.6883 0.6656 0.2903 0.3158 0.2987 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0151 4.1474 35000 0.4685 41.9114 0.4322 0.4410 0.4351 0.4558 0.4669 0.4596 0.4700 0.4830 0.4743 0.6506 0.6798 0.6608 0.2667 0.2910 0.2798 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0175 4.4437 37500 0.5112 42.8421 0.4149 0.4269 0.4190 0.4405 0.4558 0.4459 0.4547 0.4715 0.4601 0.6293 0.6670 0.6422 0.2411 0.2667 0.2516 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0097 4.7399 40000 0.4808 42.9647 0.4254 0.4353 0.4290 0.4481 0.4613 0.4531 0.4618 0.4777 0.4676 0.6326 0.6680 0.6452 0.2407 0.2639 0.25 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0072 5.0361 42500 0.4667 40.1506 0.4493 0.4608 0.4538 0.4747 0.4878 0.4793 0.4891 0.5041 0.4940 0.6741 0.7059 0.6854 0.3 0.3333 0.3171 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0092 5.3324 45000 0.4639 40.5386 0.4385 0.4534 0.4446 0.4630 0.4804 0.4697 0.4766 0.4966 0.4840 0.6494 0.6895 0.6645 0.2903 0.3247 0.3041 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0098 5.6286 47500 0.5158 43.3221 0.4220 0.4344 0.4268 0.4456 0.4628 0.4519 0.4580 0.4782 0.4651 0.6228 0.6661 0.6383 0.2474 0.2784 0.2584 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0043 5.9249 50000 0.4820 41.5204 0.4424 0.4514 0.4455 0.4657 0.4765 0.4695 0.4774 0.4899 0.4819 0.6544 0.6795 0.6632 0.2941 0.3207 0.3000 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0043 6.2211 52500 0.4708 41.5802 0.4544 0.4684 0.4588 0.4779 0.4944 0.4832 0.4905 0.5092 0.4964 0.6580 0.6914 0.6685 0.3243 0.3571 0.3333 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0105 6.5174 55000 0.5031 42.6977 0.4399 0.4539 0.4452 0.4629 0.4799 0.4693 0.4762 0.4950 0.4830 0.6501 0.6888 0.6642 0.3037 0.3422 0.32 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0161 6.8136 57500 0.4820 39.5101 0.4516 0.4628 0.4556 0.4728 0.4866 0.4776 0.4855 0.5023 0.4910 0.6528 0.6893 0.6656 0.3118 0.3448 0.3231 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0069 7.1098 60000 0.4743 39.6487 0.4535 0.4651 0.4577 0.4759 0.4897 0.4807 0.4889 0.5049 0.4944 0.6699 0.7018 0.6815 0.3333 0.3507 0.3333 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0098 7.4061 62500 0.4884 40.9938 0.4399 0.4524 0.4447 0.4623 0.4774 0.4677 0.4744 0.4932 0.4806 0.6489 0.6895 0.6630 0.2896 0.32 0.3038 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0012 7.7023 65000 0.4967 40.0236 0.4449 0.4579 0.4498 0.4664 0.4817 0.4720 0.4785 0.4957 0.4844 0.6567 0.6922 0.6690 0.3086 0.3409 0.3205 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0025 7.9986 67500 0.4861 39.8325 0.4618 0.4724 0.4657 0.4854 0.4981 0.4902 0.4981 0.5122 0.5031 0.6722 0.7016 0.6828 0.3459 0.375 0.3529 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0018 8.2948 70000 0.4827 39.1016 0.4563 0.4680 0.4607 0.4785 0.4923 0.4836 0.4907 0.5061 0.4961 0.6707 0.7028 0.6821 0.3333 0.3624 0.3485 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0027 8.5911 72500 0.4808 38.9368 0.4607 0.4718 0.4647 0.4825 0.4965 0.4875 0.4955 0.5120 0.5010 0.6703 0.7022 0.6813 0.3333 0.3709 0.3521 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0018 8.8873 75000 0.4793 38.9076 0.4630 0.4733 0.4669 0.4852 0.4984 0.4901 0.4975 0.5123 0.5025 0.6681 0.6997 0.6789 0.3438 0.3718 0.3519 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0015 9.1836 77500 0.4782 38.2453 0.4716 0.4820 0.4755 0.4931 0.5057 0.4978 0.5052 0.5191 0.5100 0.6811 0.7099 0.6913 0.3667 0.3908 0.3774 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.001 9.4798 80000 0.4731 37.9958 0.4726 0.4821 0.4762 0.4946 0.5062 0.4989 0.5073 0.5201 0.5116 0.6840 0.7091 0.6925 0.35 0.3709 0.3577 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Framework versions

  • Transformers 4.49.0
  • Pytorch 2.6.0+cu126
  • Datasets 2.12.0
  • Tokenizers 0.20.1