File size: 22,114 Bytes
96dd540 3cb8c2d 96dd540 3cb8c2d 96dd540 3cb8c2d 96dd540 3cb8c2d 96dd540 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 |
---
library_name: transformers
language:
- he
base_model: ivrit-ai/whisper-large-v3-turbo
tags:
- hf-asr-leaderboard
- generated_from_trainer
metrics:
- wer
model-index:
- name: he-cantillation
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# he-cantillation
This model is a fine-tuned version of [ivrit-ai/whisper-large-v3-turbo](https://huggingface.co/ivrit-ai/whisper-large-v3-turbo) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4220
- Wer: 35.5610
- Avg Precision Exact: 0.4895
- Avg Recall Exact: 0.5009
- Avg F1 Exact: 0.4943
- Avg Precision Letter Shift: 0.5105
- Avg Recall Letter Shift: 0.5234
- Avg F1 Letter Shift: 0.5157
- Avg Precision Word Level: 0.5228
- Avg Recall Word Level: 0.5364
- Avg F1 Word Level: 0.5279
- Avg Precision Word Shift: 0.6946
- Avg Recall Word Shift: 0.7228
- Avg F1 Word Shift: 0.7055
- Precision Median Exact: 0.4524
- Recall Median Exact: 0.4833
- F1 Median Exact: 0.4706
- Precision Max Exact: 1.0
- Recall Max Exact: 1.0
- F1 Max Exact: 1.0
- Precision Min Exact: 0.0
- Recall Min Exact: 0.0
- F1 Min Exact: 0.0
- Precision Min Letter Shift: 0.0
- Recall Min Letter Shift: 0.0
- F1 Min Letter Shift: 0.0
- Precision Min Word Level: 0.0
- Recall Min Word Level: 0.0
- F1 Min Word Level: 0.0
- Precision Min Word Shift: 0.0
- Recall Min Word Shift: 0.0
- F1 Min Word Shift: 0.0
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 16
- eval_batch_size: 2
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- training_steps: 60000
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Wer | Avg Precision Exact | Avg Recall Exact | Avg F1 Exact | Avg Precision Letter Shift | Avg Recall Letter Shift | Avg F1 Letter Shift | Avg Precision Word Level | Avg Recall Word Level | Avg F1 Word Level | Avg Precision Word Shift | Avg Recall Word Shift | Avg F1 Word Shift | Precision Median Exact | Recall Median Exact | F1 Median Exact | Precision Max Exact | Recall Max Exact | F1 Max Exact | Precision Min Exact | Recall Min Exact | F1 Min Exact | Precision Min Letter Shift | Recall Min Letter Shift | F1 Min Letter Shift | Precision Min Word Level | Recall Min Word Level | F1 Min Word Level | Precision Min Word Shift | Recall Min Word Shift | F1 Min Word Shift |
|:-------------:|:------:|:-----:|:---------------:|:--------:|:-------------------:|:----------------:|:------------:|:--------------------------:|:-----------------------:|:-------------------:|:------------------------:|:---------------------:|:-----------------:|:------------------------:|:---------------------:|:-----------------:|:----------------------:|:-------------------:|:---------------:|:-------------------:|:----------------:|:------------:|:-------------------:|:----------------:|:------------:|:--------------------------:|:-----------------------:|:-------------------:|:------------------------:|:---------------------:|:-----------------:|:------------------------:|:---------------------:|:-----------------:|
| No log | 0.0001 | 1 | 7.2503 | 108.6320 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 0.1342 | 0.2962 | 2500 | 0.6879 | 55.3095 | 0.3378 | 0.3531 | 0.3445 | 0.3736 | 0.3930 | 0.3815 | 0.3902 | 0.4089 | 0.3972 | 0.5839 | 0.6279 | 0.6015 | 0.1935 | 0.2174 | 0.2021 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0737 | 0.5925 | 5000 | 0.6062 | 50.9023 | 0.3793 | 0.3990 | 0.3878 | 0.4084 | 0.4316 | 0.4184 | 0.4244 | 0.4481 | 0.4344 | 0.6099 | 0.6620 | 0.6313 | 0.25 | 0.2772 | 0.2647 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0637 | 0.8887 | 7500 | 0.5195 | 47.3558 | 0.3950 | 0.4133 | 0.4024 | 0.4216 | 0.4437 | 0.4298 | 0.4354 | 0.4590 | 0.4441 | 0.6160 | 0.6641 | 0.6339 | 0.2581 | 0.2917 | 0.2692 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0499 | 1.1850 | 10000 | 0.5286 | 46.2515 | 0.4063 | 0.4231 | 0.4125 | 0.4327 | 0.4530 | 0.4401 | 0.4477 | 0.4708 | 0.4561 | 0.6263 | 0.6728 | 0.6433 | 0.2667 | 0.3 | 0.2817 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0468 | 1.4812 | 12500 | 0.5122 | 46.7592 | 0.4081 | 0.4204 | 0.4130 | 0.4345 | 0.4489 | 0.4401 | 0.4476 | 0.4627 | 0.4534 | 0.6198 | 0.6532 | 0.6324 | 0.2571 | 0.2806 | 0.2667 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.053 | 1.7775 | 15000 | 0.4899 | 45.7365 | 0.4067 | 0.4211 | 0.4119 | 0.4348 | 0.4524 | 0.4408 | 0.4490 | 0.4689 | 0.4556 | 0.6308 | 0.6698 | 0.6455 | 0.2727 | 0.2934 | 0.2778 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0357 | 2.0737 | 17500 | 0.4765 | 44.8087 | 0.4227 | 0.4343 | 0.4277 | 0.4494 | 0.4636 | 0.4554 | 0.4654 | 0.4795 | 0.4709 | 0.6496 | 0.6805 | 0.6614 | 0.2857 | 0.3095 | 0.2934 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0452 | 2.3699 | 20000 | 0.4817 | 43.5380 | 0.4225 | 0.4342 | 0.4263 | 0.4483 | 0.4632 | 0.4533 | 0.4617 | 0.4777 | 0.4666 | 0.6398 | 0.6742 | 0.6514 | 0.2642 | 0.2857 | 0.2689 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0384 | 2.6662 | 22500 | 0.4465 | 42.9034 | 0.4410 | 0.4517 | 0.4450 | 0.4656 | 0.4779 | 0.4700 | 0.4783 | 0.4919 | 0.4829 | 0.6630 | 0.6879 | 0.6717 | 0.3134 | 0.3333 | 0.32 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0171 | 2.9624 | 25000 | 0.4247 | 39.2373 | 0.4574 | 0.4690 | 0.4619 | 0.4819 | 0.4952 | 0.4868 | 0.4948 | 0.5104 | 0.5006 | 0.6782 | 0.7077 | 0.6892 | 0.3433 | 0.3703 | 0.3504 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0142 | 3.2587 | 27500 | 0.4705 | 43.7539 | 0.4300 | 0.4472 | 0.4361 | 0.4545 | 0.4748 | 0.4613 | 0.4665 | 0.4887 | 0.4739 | 0.6422 | 0.6844 | 0.6568 | 0.3103 | 0.3333 | 0.3220 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0163 | 3.5549 | 30000 | 0.4480 | 41.7772 | 0.4505 | 0.4613 | 0.4548 | 0.4759 | 0.4873 | 0.4798 | 0.4900 | 0.5018 | 0.4937 | 0.6693 | 0.6940 | 0.6777 | 0.3455 | 0.375 | 0.3523 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0081 | 3.8512 | 32500 | 0.4588 | 40.7224 | 0.4470 | 0.4565 | 0.4506 | 0.4701 | 0.4810 | 0.4741 | 0.4829 | 0.4946 | 0.4872 | 0.6673 | 0.6950 | 0.6775 | 0.3333 | 0.35 | 0.3418 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0156 | 4.1474 | 35000 | 0.4382 | 38.0863 | 0.4597 | 0.4695 | 0.4639 | 0.4835 | 0.4956 | 0.4882 | 0.4964 | 0.5099 | 0.5012 | 0.6763 | 0.7053 | 0.6873 | 0.3563 | 0.3846 | 0.3704 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0146 | 4.4437 | 37500 | 0.4453 | 39.8763 | 0.4662 | 0.4771 | 0.4704 | 0.4896 | 0.5038 | 0.4946 | 0.5019 | 0.5173 | 0.5072 | 0.6714 | 0.7033 | 0.6826 | 0.3690 | 0.4118 | 0.3795 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0075 | 4.7399 | 40000 | 0.4473 | 37.8747 | 0.4659 | 0.4760 | 0.4700 | 0.4880 | 0.5008 | 0.4930 | 0.5004 | 0.5147 | 0.5059 | 0.6739 | 0.7035 | 0.6852 | 0.3732 | 0.4064 | 0.3867 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0054 | 5.0361 | 42500 | 0.4328 | 37.4954 | 0.4664 | 0.4775 | 0.4711 | 0.4898 | 0.5035 | 0.4952 | 0.5020 | 0.5167 | 0.5074 | 0.6801 | 0.7111 | 0.6920 | 0.3611 | 0.3934 | 0.3743 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0019 | 5.3324 | 45000 | 0.4505 | 38.5108 | 0.4614 | 0.4718 | 0.4656 | 0.4856 | 0.4984 | 0.4907 | 0.4985 | 0.5121 | 0.5037 | 0.6791 | 0.7063 | 0.6895 | 0.3606 | 0.3901 | 0.3709 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0089 | 5.6286 | 47500 | 0.4472 | 38.6348 | 0.4593 | 0.4684 | 0.4628 | 0.4828 | 0.4941 | 0.4871 | 0.4949 | 0.5074 | 0.4995 | 0.6734 | 0.7023 | 0.6841 | 0.3438 | 0.3624 | 0.3530 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0171 | 5.9249 | 50000 | 0.4119 | 36.0337 | 0.4815 | 0.4914 | 0.4854 | 0.5048 | 0.5163 | 0.5092 | 0.5169 | 0.5291 | 0.5214 | 0.6935 | 0.7229 | 0.7049 | 0.4286 | 0.4535 | 0.4407 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0025 | 6.2211 | 52500 | 0.4380 | 36.5982 | 0.4804 | 0.4927 | 0.4853 | 0.5030 | 0.5172 | 0.5086 | 0.5136 | 0.5294 | 0.5197 | 0.6862 | 0.7181 | 0.6985 | 0.4068 | 0.4367 | 0.4205 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0062 | 6.5174 | 55000 | 0.4319 | 37.3904 | 0.4784 | 0.4895 | 0.4832 | 0.5006 | 0.5141 | 0.5063 | 0.5129 | 0.5269 | 0.5185 | 0.6844 | 0.7136 | 0.6958 | 0.4142 | 0.4390 | 0.4255 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0085 | 6.8136 | 57500 | 0.4211 | 36.6566 | 0.4856 | 0.4969 | 0.4903 | 0.5068 | 0.5195 | 0.5118 | 0.5188 | 0.5329 | 0.5239 | 0.6872 | 0.7160 | 0.6977 | 0.4353 | 0.4721 | 0.4552 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0048 | 7.1098 | 60000 | 0.4220 | 35.5610 | 0.4895 | 0.5009 | 0.4943 | 0.5105 | 0.5234 | 0.5157 | 0.5228 | 0.5364 | 0.5279 | 0.6946 | 0.7228 | 0.7055 | 0.4524 | 0.4833 | 0.4706 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
### Framework versions
- Transformers 4.49.0
- Pytorch 2.6.0+cu126
- Datasets 2.12.0
- Tokenizers 0.20.1
|