Hugging Face's logo

Mihaj
/
wav2vec2-large-mms-1b-livvi-karelian-CodeSwitching

Automatic Speech Recognition
Transformers
TensorBoard
Safetensors
wav2vec2
Generated from Trainer
Model card Files Metrics Community
wav2vec2-large-mms-1b-livvi-karelian-CodeSwitching / README.md
Mihaj's picture
Mihaj
Upload tokenizer
0e82e65 verified
preview code
|
raw
history blame
8.76 kB
metadata
license: cc-by-nc-4.0
tags:
  - generated_from_trainer
base_model: facebook/mms-1b-all
metrics:
  - wer
model-index:
  - name: wav2vec2-large-mms-1b-livvi-karelian-CodeSwitching
    results: []

wav2vec2-large-mms-1b-livvi-karelian-CodeSwitching

This model is a fine-tuned version of facebook/mms-1b-all on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.4475
  • Wer: 0.4337
  • Cer: 0.0962

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0007
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • distributed_type: multi-GPU
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 32
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 100
  • training_steps: 10000
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer Cer
4.5669 0.9070 100 0.7117 0.7585 0.1988
0.687 1.8141 200 0.5073 0.6150 0.1476
0.5978 2.7211 300 0.4575 0.5917 0.1373
0.5536 3.6281 400 0.4288 0.5484 0.1297
0.524 4.5351 500 0.4283 0.5639 0.1295
0.4979 5.4422 600 0.4172 0.5474 0.1262
0.4791 6.3492 700 0.4048 0.5311 0.1207
0.4809 7.2562 800 0.4005 0.5217 0.1204
0.4434 8.1633 900 0.3877 0.5122 0.1171
0.437 9.0703 1000 0.3845 0.5105 0.1163
0.4251 9.9773 1100 0.3829 0.5037 0.1146
0.3866 10.8844 1200 0.3638 0.4861 0.1114
0.3956 11.7914 1300 0.3726 0.4922 0.1128
0.3796 12.6984 1400 0.3644 0.4922 0.1102
0.3698 13.6054 1500 0.3646 0.4800 0.1091
0.3657 14.5125 1600 0.3664 0.4780 0.1088
0.3506 15.4195 1700 0.3609 0.4740 0.1087
0.3476 16.3265 1800 0.3542 0.4770 0.1077
0.3461 17.2336 1900 0.3639 0.4760 0.1070
0.3194 18.1406 2000 0.3556 0.4733 0.1054
0.3191 19.0476 2100 0.3639 0.4692 0.1055
0.3076 19.9546 2200 0.3565 0.4614 0.1034
0.3056 20.8617 2300 0.3597 0.4652 0.1047
0.3068 21.7687 2400 0.3620 0.4740 0.1048
0.2913 22.6757 2500 0.3535 0.4601 0.1033
0.2875 23.5828 2600 0.3610 0.4672 0.1035
0.2845 24.4898 2700 0.3586 0.4560 0.1020
0.2793 25.3968 2800 0.3633 0.4587 0.1028
0.273 26.3039 2900 0.3638 0.4614 0.1028
0.2682 27.2109 3000 0.3725 0.4699 0.1029
0.2549 28.1179 3100 0.3669 0.4540 0.1015
0.2535 29.0249 3200 0.3630 0.4604 0.1026
0.2464 29.9320 3300 0.3601 0.4601 0.1023
0.2517 30.8390 3400 0.3680 0.4621 0.1024
0.2424 31.7460 3500 0.3773 0.4597 0.1028
0.2455 32.6531 3600 0.3706 0.4614 0.1019
0.2311 33.5601 3700 0.3667 0.4530 0.0992
0.2318 34.4671 3800 0.3750 0.4597 0.1022
0.2233 35.3741 3900 0.3643 0.4662 0.1027
0.2246 36.2812 4000 0.3737 0.4570 0.1013
0.2187 37.1882 4100 0.3864 0.4547 0.1001
0.2199 38.0952 4200 0.3925 0.4418 0.0980
0.2167 39.0023 4300 0.3808 0.4533 0.0992
0.2131 39.9093 4400 0.3861 0.4506 0.0996
0.2044 40.8163 4500 0.3771 0.4581 0.1005
0.1981 41.7234 4600 0.3759 0.4611 0.1021
0.2013 42.6304 4700 0.3817 0.4641 0.1028
0.198 43.5374 4800 0.3786 0.4479 0.0990
0.1905 44.4444 4900 0.3889 0.4465 0.0976
0.1829 45.3515 5000 0.3895 0.4496 0.0982
0.1912 46.2585 5100 0.3894 0.4459 0.0983
0.1879 47.1655 5200 0.3986 0.4486 0.0980
0.1817 48.0726 5300 0.3865 0.4476 0.0996
0.1803 48.9796 5400 0.4024 0.4408 0.0989
0.1752 49.8866 5500 0.3871 0.4493 0.0994
0.1755 50.7937 5600 0.3969 0.4442 0.0983
0.1794 51.7007 5700 0.4069 0.4364 0.0969
0.1739 52.6077 5800 0.4048 0.4357 0.0981
0.1691 53.5147 5900 0.4082 0.4469 0.0982
0.1648 54.4218 6000 0.4227 0.4422 0.0984
0.1748 55.3288 6100 0.4027 0.4435 0.0990
0.1668 56.2358 6200 0.4311 0.4438 0.0990
0.1617 57.1429 6300 0.4116 0.4472 0.0988
0.1617 58.0499 6400 0.4051 0.4432 0.0974
0.1625 58.9569 6500 0.4146 0.4374 0.0975
0.1594 59.8639 6600 0.4108 0.4435 0.0988
0.1668 60.7710 6700 0.4095 0.4327 0.0969
0.1512 61.6780 6800 0.4172 0.4378 0.0966
0.15 62.5850 6900 0.4135 0.4422 0.0982
0.1537 63.4921 7000 0.4326 0.4401 0.0980
0.151 64.3991 7100 0.4255 0.4411 0.0980
0.152 65.3061 7200 0.4236 0.4378 0.0976
0.1468 66.2132 7300 0.4207 0.4526 0.0996
0.1497 67.1202 7400 0.4160 0.4398 0.0980
0.1509 68.0272 7500 0.4210 0.4401 0.0975
0.138 68.9342 7600 0.4224 0.4469 0.0984
0.1435 69.8413 7700 0.4217 0.4425 0.0968
0.1465 70.7483 7800 0.4280 0.4354 0.0964
0.1367 71.6553 7900 0.4260 0.4405 0.0978
0.1445 72.5624 8000 0.4253 0.4394 0.0980
0.1379 73.4694 8100 0.4250 0.4347 0.0968
0.1385 74.3764 8200 0.4303 0.4391 0.0975
0.1334 75.2834 8300 0.4377 0.4357 0.0968
0.1372 76.1905 8400 0.4423 0.4323 0.0957
0.1329 77.0975 8500 0.4281 0.4334 0.0967
0.1322 78.0045 8600 0.4366 0.4313 0.0956
0.1327 78.9116 8700 0.4377 0.4344 0.0960
0.1313 79.8186 8800 0.4405 0.4361 0.0964
0.1315 80.7256 8900 0.4401 0.4350 0.0959
0.134 81.6327 9000 0.4370 0.4364 0.0963
0.1351 82.5397 9100 0.4389 0.4364 0.0962
0.1289 83.4467 9200 0.4504 0.4384 0.0972
0.1308 84.3537 9300 0.4447 0.4334 0.0963
0.1321 85.2608 9400 0.4509 0.4334 0.0965
0.1296 86.1678 9500 0.4489 0.4323 0.0960
0.1277 87.0748 9600 0.4489 0.4374 0.0963
0.1245 87.9819 9700 0.4475 0.4344 0.0965
0.1267 88.8889 9800 0.4459 0.4323 0.0952
0.1259 89.7959 9900 0.4457 0.4320 0.0959
0.1247 90.7029 10000 0.4475 0.4337 0.0962

Framework versions

  • Transformers 4.41.0.dev0
  • Pytorch 2.2.2
  • Datasets 2.19.0
  • Tokenizers 0.19.1