File size: 18,407 Bytes
9f0ee88 6864044 9f0ee88 6864044 9f0ee88 6864044 9f0ee88 6864044 9f0ee88 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 |
---
library_name: transformers
language:
- he
license: apache-2.0
base_model: openai/whisper-medium
tags:
- hf-asr-leaderboard
- generated_from_trainer
metrics:
- wer
model-index:
- name: he-cantillation
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# he-cantillation
This model is a fine-tuned version of [openai/whisper-medium](https://huggingface.co/openai/whisper-medium) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 2.8408
- Wer: 94.6548
- Avg Precision Exact: 0.0941
- Avg Recall Exact: 0.1218
- Avg F1 Exact: 0.1041
- Avg Precision Letter Shift: 0.1168
- Avg Recall Letter Shift: 0.1531
- Avg F1 Letter Shift: 0.1297
- Avg Precision Word Level: 0.1354
- Avg Recall Word Level: 0.1752
- Avg F1 Word Level: 0.1488
- Avg Precision Word Shift: 0.2611
- Avg Recall Word Shift: 0.3573
- Avg F1 Word Shift: 0.2941
- Precision Median Exact: 0.0513
- Recall Median Exact: 0.0769
- F1 Median Exact: 0.0615
- Precision Max Exact: 1.0
- Recall Max Exact: 1.0
- F1 Max Exact: 1.0
- Precision Min Exact: 0.0
- Recall Min Exact: 0.0
- F1 Min Exact: 0.0
- Precision Min Letter Shift: 0.0
- Recall Min Letter Shift: 0.0
- F1 Min Letter Shift: 0.0
- Precision Min Word Level: 0.0
- Recall Min Word Level: 0.0
- F1 Min Word Level: 0.0
- Precision Min Word Shift: 0.0
- Recall Min Word Shift: 0.0
- F1 Min Word Shift: 0.0
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 16
- eval_batch_size: 2
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- training_steps: 20000
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Wer | Avg Precision Exact | Avg Recall Exact | Avg F1 Exact | Avg Precision Letter Shift | Avg Recall Letter Shift | Avg F1 Letter Shift | Avg Precision Word Level | Avg Recall Word Level | Avg F1 Word Level | Avg Precision Word Shift | Avg Recall Word Shift | Avg F1 Word Shift | Precision Median Exact | Recall Median Exact | F1 Median Exact | Precision Max Exact | Recall Max Exact | F1 Max Exact | Precision Min Exact | Recall Min Exact | F1 Min Exact | Precision Min Letter Shift | Recall Min Letter Shift | F1 Min Letter Shift | Precision Min Word Level | Recall Min Word Level | F1 Min Word Level | Precision Min Word Shift | Recall Min Word Shift | F1 Min Word Shift |
|:-------------:|:------:|:-----:|:---------------:|:-------:|:-------------------:|:----------------:|:------------:|:--------------------------:|:-----------------------:|:-------------------:|:------------------------:|:---------------------:|:-----------------:|:------------------------:|:---------------------:|:-----------------:|:----------------------:|:-------------------:|:---------------:|:-------------------:|:----------------:|:------------:|:-------------------:|:----------------:|:------------:|:--------------------------:|:-----------------------:|:-------------------:|:------------------------:|:---------------------:|:-----------------:|:------------------------:|:---------------------:|:-----------------:|
| 0.1592 | 0.1502 | 1000 | 2.2652 | 96.3018 | 0.0694 | 0.0723 | 0.0702 | 0.0967 | 0.1020 | 0.0982 | 0.1222 | 0.1287 | 0.1233 | 0.2750 | 0.3083 | 0.2874 | 0.0408 | 0.0476 | 0.0440 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0531 | 0.3003 | 2000 | 2.1869 | 94.9407 | 0.0884 | 0.0901 | 0.0888 | 0.1171 | 0.1203 | 0.1176 | 0.1400 | 0.1427 | 0.1397 | 0.3029 | 0.3257 | 0.3105 | 0.0526 | 0.0556 | 0.0537 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0351 | 0.4505 | 3000 | 2.2520 | 94.7335 | 0.1005 | 0.1002 | 0.0999 | 0.1299 | 0.1303 | 0.1293 | 0.1541 | 0.1524 | 0.1519 | 0.3146 | 0.3288 | 0.3188 | 0.0588 | 0.0606 | 0.0588 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0225 | 0.6006 | 4000 | 2.4083 | 94.2536 | 0.0988 | 0.1023 | 0.1000 | 0.1257 | 0.1315 | 0.1278 | 0.1485 | 0.1531 | 0.1495 | 0.3094 | 0.3376 | 0.3199 | 0.0588 | 0.0645 | 0.0619 | 0.8333 | 1.0 | 0.8333 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0187 | 0.7508 | 5000 | 2.4580 | 94.8357 | 0.0903 | 0.0927 | 0.0910 | 0.1206 | 0.1251 | 0.1220 | 0.1462 | 0.1498 | 0.1466 | 0.3166 | 0.3405 | 0.3250 | 0.0526 | 0.0588 | 0.0548 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0184 | 0.9009 | 6000 | 2.4746 | 94.2536 | 0.1009 | 0.1047 | 0.1021 | 0.1297 | 0.1358 | 0.1315 | 0.1539 | 0.1606 | 0.1554 | 0.3207 | 0.3522 | 0.3317 | 0.0625 | 0.0714 | 0.0645 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0123 | 1.0511 | 7000 | 2.4247 | 93.6321 | 0.1010 | 0.1151 | 0.1060 | 0.1285 | 0.1477 | 0.1348 | 0.1499 | 0.1715 | 0.1568 | 0.3031 | 0.3655 | 0.3245 | 0.0588 | 0.0769 | 0.0645 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0075 | 1.2012 | 8000 | 2.4427 | 93.0880 | 0.1051 | 0.1187 | 0.1098 | 0.1328 | 0.1518 | 0.1389 | 0.1533 | 0.1746 | 0.1599 | 0.3065 | 0.3661 | 0.3267 | 0.0606 | 0.0795 | 0.0690 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0095 | 1.3514 | 9000 | 2.4336 | 93.8845 | 0.1049 | 0.1199 | 0.1103 | 0.1325 | 0.1528 | 0.1395 | 0.1540 | 0.1760 | 0.1609 | 0.3059 | 0.3700 | 0.3287 | 0.0606 | 0.0795 | 0.0690 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0034 | 1.5015 | 10000 | 2.5451 | 94.2798 | 0.0928 | 0.1197 | 0.1023 | 0.1156 | 0.1514 | 0.1277 | 0.1334 | 0.1737 | 0.1467 | 0.2597 | 0.3576 | 0.2926 | 0.0513 | 0.08 | 0.0615 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0108 | 1.6517 | 11000 | 2.5198 | 93.9239 | 0.0949 | 0.1097 | 0.1001 | 0.1187 | 0.1392 | 0.1257 | 0.1397 | 0.1618 | 0.1463 | 0.2837 | 0.3470 | 0.3046 | 0.05 | 0.0690 | 0.0571 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0062 | 1.8018 | 12000 | 2.5299 | 93.4104 | 0.1075 | 0.1185 | 0.1114 | 0.1344 | 0.1496 | 0.1394 | 0.1562 | 0.1735 | 0.1616 | 0.3054 | 0.3601 | 0.3247 | 0.0588 | 0.0741 | 0.0654 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0098 | 1.9520 | 13000 | 2.6702 | 94.7715 | 0.0914 | 0.1172 | 0.1009 | 0.1133 | 0.1474 | 0.1255 | 0.1318 | 0.1704 | 0.1449 | 0.2629 | 0.3586 | 0.2958 | 0.05 | 0.0714 | 0.0588 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0029 | 2.1021 | 14000 | 2.6491 | 94.6285 | 0.0972 | 0.1269 | 0.1076 | 0.1184 | 0.1579 | 0.1319 | 0.1361 | 0.1809 | 0.1508 | 0.2613 | 0.3698 | 0.2975 | 0.0541 | 0.0870 | 0.0645 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0027 | 2.2523 | 15000 | 2.7955 | 94.4928 | 0.0943 | 0.1149 | 0.1017 | 0.1179 | 0.1462 | 0.1280 | 0.1376 | 0.1676 | 0.1472 | 0.2708 | 0.3525 | 0.2989 | 0.05 | 0.0714 | 0.0580 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0047 | 2.4024 | 16000 | 2.9242 | 95.2923 | 0.0856 | 0.1242 | 0.0986 | 0.1045 | 0.1535 | 0.1207 | 0.1204 | 0.1750 | 0.1379 | 0.2282 | 0.3503 | 0.2669 | 0.0435 | 0.0769 | 0.0541 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.003 | 2.5526 | 17000 | 2.8360 | 94.2200 | 0.0998 | 0.1233 | 0.1084 | 0.1231 | 0.1546 | 0.1344 | 0.1422 | 0.1766 | 0.1537 | 0.2795 | 0.3672 | 0.3091 | 0.0541 | 0.08 | 0.0635 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0041 | 2.7027 | 18000 | 2.8231 | 94.8605 | 0.0889 | 0.1136 | 0.0979 | 0.1112 | 0.1436 | 0.1227 | 0.1305 | 0.1667 | 0.1428 | 0.2561 | 0.3469 | 0.2874 | 0.0460 | 0.0714 | 0.0556 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0012 | 2.8529 | 19000 | 2.9125 | 95.0735 | 0.0881 | 0.1236 | 0.1007 | 0.1093 | 0.1547 | 0.1252 | 0.1272 | 0.1778 | 0.1442 | 0.2429 | 0.3621 | 0.2828 | 0.0455 | 0.08 | 0.0580 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 0.0031 | 3.0030 | 20000 | 2.8408 | 94.6548 | 0.0941 | 0.1218 | 0.1041 | 0.1168 | 0.1531 | 0.1297 | 0.1354 | 0.1752 | 0.1488 | 0.2611 | 0.3573 | 0.2941 | 0.0513 | 0.0769 | 0.0615 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
### Framework versions
- Transformers 4.49.0
- Pytorch 2.6.0+cu126
- Datasets 2.12.0
- Tokenizers 0.20.1
|