Hugging Face's logo

Mihaj
/
wav2vec2-large-mms-1b-livvi-karelian-CodeSwitching

Automatic Speech Recognition
Transformers
TensorBoard
Safetensors
wav2vec2
Generated from Trainer
Model card Files Metrics Community
wav2vec2-large-mms-1b-livvi-karelian-CodeSwitching / README.md
Mihaj's picture
Mihaj
Upload tokenizer
d217a3d verified
preview code
|
raw
history blame
8.76 kB
metadata
license: cc-by-nc-4.0
tags:
  - generated_from_trainer
base_model: facebook/mms-1b-all
metrics:
  - wer
model-index:
  - name: wav2vec2-large-mms-1b-livvi-karelian-CodeSwitching
    results: []

wav2vec2-large-mms-1b-livvi-karelian-CodeSwitching

This model is a fine-tuned version of facebook/mms-1b-all on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.3635
  • Wer: 0.4844
  • Cer: 0.1090

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • distributed_type: multi-GPU
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 32
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 100
  • training_steps: 10000
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer Cer
9.8411 0.9070 100 3.1112 0.9990 0.9430
1.7444 1.8141 200 0.8819 0.8271 0.2320
0.9096 2.7211 300 0.7172 0.7652 0.1963
0.781 3.6281 400 0.6279 0.7141 0.1759
0.711 4.5351 500 0.5565 0.6641 0.1597
0.6544 5.4422 600 0.5162 0.6346 0.1522
0.6186 6.3492 700 0.5023 0.6211 0.1489
0.6239 7.2562 800 0.4880 0.6042 0.1428
0.5876 8.1633 900 0.4737 0.6028 0.1429
0.5862 9.0703 1000 0.4668 0.5940 0.1400
0.5821 9.9773 1100 0.4593 0.5880 0.1385
0.5405 10.8844 1200 0.4491 0.5819 0.1368
0.5591 11.7914 1300 0.4459 0.5819 0.1365
0.5374 12.6984 1400 0.4409 0.5798 0.1358
0.535 13.6054 1500 0.4347 0.5765 0.1336
0.5332 14.5125 1600 0.4358 0.5717 0.1336
0.5142 15.4195 1700 0.4263 0.5643 0.1312
0.5156 16.3265 1800 0.4221 0.5626 0.1299
0.5212 17.2336 1900 0.4188 0.5599 0.1297
0.4892 18.1406 2000 0.4181 0.5585 0.1299
0.4954 19.0476 2100 0.4128 0.5524 0.1288
0.4867 19.9546 2200 0.4083 0.5531 0.1275
0.482 20.8617 2300 0.4064 0.5403 0.1254
0.4892 21.7687 2400 0.4070 0.5396 0.1256
0.4712 22.6757 2500 0.4009 0.5335 0.1245
0.4649 23.5828 2600 0.3978 0.5392 0.1253
0.4711 24.4898 2700 0.3958 0.5355 0.1242
0.4578 25.3968 2800 0.3937 0.5382 0.1236
0.4594 26.3039 2900 0.3952 0.5318 0.1224
0.453 27.2109 3000 0.3950 0.5457 0.1245
0.4453 28.1179 3100 0.3939 0.5260 0.1210
0.4428 29.0249 3200 0.3886 0.5233 0.1207
0.4335 29.9320 3300 0.3867 0.5271 0.1207
0.4426 30.8390 3400 0.3843 0.5237 0.1202
0.4342 31.7460 3500 0.3875 0.5203 0.1193
0.429 32.6531 3600 0.3803 0.5179 0.1187
0.4141 33.5601 3700 0.3794 0.5135 0.1173
0.428 34.4671 3800 0.3794 0.5179 0.1179
0.4043 35.3741 3900 0.3808 0.5135 0.1178
0.4203 36.2812 4000 0.3801 0.5152 0.1189
0.4078 37.1882 4100 0.3783 0.5152 0.1181
0.4073 38.0952 4200 0.3752 0.5145 0.1175
0.4152 39.0023 4300 0.3750 0.5166 0.1178
0.4027 39.9093 4400 0.3768 0.5162 0.1177
0.4013 40.8163 4500 0.3758 0.5159 0.1177
0.3903 41.7234 4600 0.3734 0.5098 0.1168
0.3905 42.6304 4700 0.3703 0.5037 0.1147
0.3942 43.5374 4800 0.3735 0.5058 0.1156
0.3835 44.4444 4900 0.3748 0.5061 0.1146
0.3752 45.3515 5000 0.3710 0.5085 0.1146
0.3899 46.2585 5100 0.3702 0.4980 0.1134
0.3774 47.1655 5200 0.3692 0.5081 0.1140
0.3749 48.0726 5300 0.3698 0.5030 0.1142
0.3769 48.9796 5400 0.3670 0.4983 0.1136
0.3654 49.8866 5500 0.3670 0.5041 0.1135
0.3637 50.7937 5600 0.3675 0.5030 0.1137
0.3728 51.7007 5700 0.3662 0.5003 0.1121
0.367 52.6077 5800 0.3650 0.4912 0.1119
0.3614 53.5147 5900 0.3664 0.4986 0.1126
0.358 54.4218 6000 0.3674 0.4963 0.1130
0.37 55.3288 6100 0.3646 0.4986 0.1131
0.3612 56.2358 6200 0.3673 0.4939 0.1120
0.3568 57.1429 6300 0.3635 0.4942 0.1113
0.3564 58.0499 6400 0.3667 0.4973 0.1120
0.3544 58.9569 6500 0.3642 0.4939 0.1110
0.355 59.8639 6600 0.3634 0.4939 0.1111
0.3546 60.7710 6700 0.3633 0.4915 0.1112
0.3479 61.6780 6800 0.3624 0.4926 0.1115
0.3433 62.5850 6900 0.3627 0.4932 0.1112
0.351 63.4921 7000 0.3657 0.4895 0.1107
0.3393 64.3991 7100 0.3663 0.4875 0.1103
0.3477 65.3061 7200 0.3633 0.4942 0.1114
0.3433 66.2132 7300 0.3637 0.4929 0.1103
0.3431 67.1202 7400 0.3629 0.4885 0.1102
0.3495 68.0272 7500 0.3624 0.4932 0.1101
0.3292 68.9342 7600 0.3615 0.4885 0.1095
0.342 69.8413 7700 0.3641 0.4875 0.1093
0.3406 70.7483 7800 0.3643 0.4878 0.1098
0.3308 71.6553 7900 0.3644 0.4899 0.1103
0.3362 72.5624 8000 0.3653 0.4844 0.1093
0.3265 73.4694 8100 0.3635 0.4902 0.1099
0.3327 74.3764 8200 0.3641 0.4844 0.1089
0.3259 75.2834 8300 0.3645 0.4871 0.1091
0.3327 76.1905 8400 0.3641 0.4848 0.1093
0.3261 77.0975 8500 0.3639 0.4875 0.1100
0.3332 78.0045 8600 0.3632 0.4885 0.1099
0.327 78.9116 8700 0.3635 0.4858 0.1096
0.3291 79.8186 8800 0.3635 0.4861 0.1090
0.3255 80.7256 8900 0.3636 0.4909 0.1101
0.3303 81.6327 9000 0.3633 0.4855 0.1098
0.3293 82.5397 9100 0.3625 0.4848 0.1093
0.3263 83.4467 9200 0.3632 0.4848 0.1093
0.3242 84.3537 9300 0.3641 0.4838 0.1094
0.327 85.2608 9400 0.3646 0.4861 0.1097
0.3297 86.1678 9500 0.3634 0.4851 0.1097
0.3259 87.0748 9600 0.3634 0.4841 0.1091
0.3242 87.9819 9700 0.3635 0.4855 0.1095
0.3156 88.8889 9800 0.3638 0.4851 0.1092
0.3231 89.7959 9900 0.3637 0.4858 0.1093
0.3193 90.7029 10000 0.3635 0.4844 0.1090

Framework versions

  • Transformers 4.41.0.dev0
  • Pytorch 2.2.2
  • Datasets 2.19.0
  • Tokenizers 0.19.1