Mihaj
/

wav2vec2-large-mms-1b-livvi-karelian-CodeSwitching

@@ -1,199 +1,165 @@
 ---
-library_name: transformers
-tags: []
 ---
-# Model Card for Model ID
-<!-- Provide a quick summary of what the model is/does. -->
-## Model Details
-### Model Description
-<!-- Provide a longer summary of what this model is. -->
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-### Model Sources [optional]
-<!-- Provide the basic links for the model. -->
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-## Uses
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-### Direct Use
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-[More Information Needed]
-### Downstream Use [optional]
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-[More Information Needed]
-### Out-of-Scope Use
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-[More Information Needed]
-## Bias, Risks, and Limitations
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-[More Information Needed]
-### Recommendations
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-## How to Get Started with the Model
-Use the code below to get started with the model.
-[More Information Needed]
-## Training Details
-### Training Data
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-[More Information Needed]
-### Training Procedure
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-#### Preprocessing [optional]
-[More Information Needed]
-#### Training Hyperparameters
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-#### Speeds, Sizes, Times [optional]
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-[More Information Needed]
-## Evaluation
-<!-- This section describes the evaluation protocols and provides the results. -->
-### Testing Data, Factors & Metrics
-#### Testing Data
-<!-- This should link to a Dataset Card if possible. -->
-[More Information Needed]
-#### Factors
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-[More Information Needed]
-#### Metrics
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-[More Information Needed]
-### Results
-[More Information Needed]
-#### Summary
-## Model Examination [optional]
-<!-- Relevant interpretability work for the model goes here -->
-[More Information Needed]
-## Environmental Impact
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-## Technical Specifications [optional]
-### Model Architecture and Objective
-[More Information Needed]
-### Compute Infrastructure
-[More Information Needed]
-#### Hardware
-[More Information Needed]
-#### Software
-[More Information Needed]
-## Citation [optional]
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-**BibTeX:**
-[More Information Needed]
-**APA:**
-[More Information Needed]
-## Glossary [optional]
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-[More Information Needed]
-## More Information [optional]
-[More Information Needed]
-## Model Card Authors [optional]
-[More Information Needed]
-## Model Card Contact
-[More Information Needed]

 ---
+license: cc-by-nc-4.0
+base_model: facebook/mms-1b-all
+tags:
+- generated_from_trainer
+metrics:
+- wer
+model-index:
+- name: wav2vec2-large-mms-1b-livvi-karelian-CodeSwitching
+  results: []
 ---
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+# wav2vec2-large-mms-1b-livvi-karelian-CodeSwitching
+This model is a fine-tuned version of [facebook/mms-1b-all](https://huggingface.co/facebook/mms-1b-all) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.3635
+- Wer: 0.4844
+- Cer: 0.1090
+## Model description
+More information needed
+## Intended uses & limitations
+More information needed
+## Training and evaluation data
+More information needed
+## Training procedure
+### Training hyperparameters
+The following hyperparameters were used during training:
+- learning_rate: 0.0001
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 42
+- distributed_type: multi-GPU
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 32
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 100
+- training_steps: 10000
+- mixed_precision_training: Native AMP
+### Training results
+| Training Loss | Epoch   | Step  | Validation Loss | Wer    | Cer    |
+|:-------------:|:-------:|:-----:|:---------------:|:------:|:------:|
+| 9.8411        | 0.9070  | 100   | 3.1112          | 0.9990 | 0.9430 |
+| 1.7444        | 1.8141  | 200   | 0.8819          | 0.8271 | 0.2320 |
+| 0.9096        | 2.7211  | 300   | 0.7172          | 0.7652 | 0.1963 |
+| 0.781         | 3.6281  | 400   | 0.6279          | 0.7141 | 0.1759 |
+| 0.711         | 4.5351  | 500   | 0.5565          | 0.6641 | 0.1597 |
+| 0.6544        | 5.4422  | 600   | 0.5162          | 0.6346 | 0.1522 |
+| 0.6186        | 6.3492  | 700   | 0.5023          | 0.6211 | 0.1489 |
+| 0.6239        | 7.2562  | 800   | 0.4880          | 0.6042 | 0.1428 |
+| 0.5876        | 8.1633  | 900   | 0.4737          | 0.6028 | 0.1429 |
+| 0.5862        | 9.0703  | 1000  | 0.4668          | 0.5940 | 0.1400 |
+| 0.5821        | 9.9773  | 1100  | 0.4593          | 0.5880 | 0.1385 |
+| 0.5405        | 10.8844 | 1200  | 0.4491          | 0.5819 | 0.1368 |
+| 0.5591        | 11.7914 | 1300  | 0.4459          | 0.5819 | 0.1365 |
+| 0.5374        | 12.6984 | 1400  | 0.4409          | 0.5798 | 0.1358 |
+| 0.535         | 13.6054 | 1500  | 0.4347          | 0.5765 | 0.1336 |
+| 0.5332        | 14.5125 | 1600  | 0.4358          | 0.5717 | 0.1336 |
+| 0.5142        | 15.4195 | 1700  | 0.4263          | 0.5643 | 0.1312 |
+| 0.5156        | 16.3265 | 1800  | 0.4221          | 0.5626 | 0.1299 |
+| 0.5212        | 17.2336 | 1900  | 0.4188          | 0.5599 | 0.1297 |
+| 0.4892        | 18.1406 | 2000  | 0.4181          | 0.5585 | 0.1299 |
+| 0.4954        | 19.0476 | 2100  | 0.4128          | 0.5524 | 0.1288 |
+| 0.4867        | 19.9546 | 2200  | 0.4083          | 0.5531 | 0.1275 |
+| 0.482         | 20.8617 | 2300  | 0.4064          | 0.5403 | 0.1254 |
+| 0.4892        | 21.7687 | 2400  | 0.4070          | 0.5396 | 0.1256 |
+| 0.4712        | 22.6757 | 2500  | 0.4009          | 0.5335 | 0.1245 |
+| 0.4649        | 23.5828 | 2600  | 0.3978          | 0.5392 | 0.1253 |
+| 0.4711        | 24.4898 | 2700  | 0.3958          | 0.5355 | 0.1242 |
+| 0.4578        | 25.3968 | 2800  | 0.3937          | 0.5382 | 0.1236 |
+| 0.4594        | 26.3039 | 2900  | 0.3952          | 0.5318 | 0.1224 |
+| 0.453         | 27.2109 | 3000  | 0.3950          | 0.5457 | 0.1245 |
+| 0.4453        | 28.1179 | 3100  | 0.3939          | 0.5260 | 0.1210 |
+| 0.4428        | 29.0249 | 3200  | 0.3886          | 0.5233 | 0.1207 |
+| 0.4335        | 29.9320 | 3300  | 0.3867          | 0.5271 | 0.1207 |
+| 0.4426        | 30.8390 | 3400  | 0.3843          | 0.5237 | 0.1202 |
+| 0.4342        | 31.7460 | 3500  | 0.3875          | 0.5203 | 0.1193 |
+| 0.429         | 32.6531 | 3600  | 0.3803          | 0.5179 | 0.1187 |
+| 0.4141        | 33.5601 | 3700  | 0.3794          | 0.5135 | 0.1173 |
+| 0.428         | 34.4671 | 3800  | 0.3794          | 0.5179 | 0.1179 |
+| 0.4043        | 35.3741 | 3900  | 0.3808          | 0.5135 | 0.1178 |
+| 0.4203        | 36.2812 | 4000  | 0.3801          | 0.5152 | 0.1189 |
+| 0.4078        | 37.1882 | 4100  | 0.3783          | 0.5152 | 0.1181 |
+| 0.4073        | 38.0952 | 4200  | 0.3752          | 0.5145 | 0.1175 |
+| 0.4152        | 39.0023 | 4300  | 0.3750          | 0.5166 | 0.1178 |
+| 0.4027        | 39.9093 | 4400  | 0.3768          | 0.5162 | 0.1177 |
+| 0.4013        | 40.8163 | 4500  | 0.3758          | 0.5159 | 0.1177 |
+| 0.3903        | 41.7234 | 4600  | 0.3734          | 0.5098 | 0.1168 |
+| 0.3905        | 42.6304 | 4700  | 0.3703          | 0.5037 | 0.1147 |
+| 0.3942        | 43.5374 | 4800  | 0.3735          | 0.5058 | 0.1156 |
+| 0.3835        | 44.4444 | 4900  | 0.3748          | 0.5061 | 0.1146 |
+| 0.3752        | 45.3515 | 5000  | 0.3710          | 0.5085 | 0.1146 |
+| 0.3899        | 46.2585 | 5100  | 0.3702          | 0.4980 | 0.1134 |
+| 0.3774        | 47.1655 | 5200  | 0.3692          | 0.5081 | 0.1140 |
+| 0.3749        | 48.0726 | 5300  | 0.3698          | 0.5030 | 0.1142 |
+| 0.3769        | 48.9796 | 5400  | 0.3670          | 0.4983 | 0.1136 |
+| 0.3654        | 49.8866 | 5500  | 0.3670          | 0.5041 | 0.1135 |
+| 0.3637        | 50.7937 | 5600  | 0.3675          | 0.5030 | 0.1137 |
+| 0.3728        | 51.7007 | 5700  | 0.3662          | 0.5003 | 0.1121 |
+| 0.367         | 52.6077 | 5800  | 0.3650          | 0.4912 | 0.1119 |
+| 0.3614        | 53.5147 | 5900  | 0.3664          | 0.4986 | 0.1126 |
+| 0.358         | 54.4218 | 6000  | 0.3674          | 0.4963 | 0.1130 |
+| 0.37          | 55.3288 | 6100  | 0.3646          | 0.4986 | 0.1131 |
+| 0.3612        | 56.2358 | 6200  | 0.3673          | 0.4939 | 0.1120 |
+| 0.3568        | 57.1429 | 6300  | 0.3635          | 0.4942 | 0.1113 |
+| 0.3564        | 58.0499 | 6400  | 0.3667          | 0.4973 | 0.1120 |
+| 0.3544        | 58.9569 | 6500  | 0.3642          | 0.4939 | 0.1110 |
+| 0.355         | 59.8639 | 6600  | 0.3634          | 0.4939 | 0.1111 |
+| 0.3546        | 60.7710 | 6700  | 0.3633          | 0.4915 | 0.1112 |
+| 0.3479        | 61.6780 | 6800  | 0.3624          | 0.4926 | 0.1115 |
+| 0.3433        | 62.5850 | 6900  | 0.3627          | 0.4932 | 0.1112 |
+| 0.351         | 63.4921 | 7000  | 0.3657          | 0.4895 | 0.1107 |
+| 0.3393        | 64.3991 | 7100  | 0.3663          | 0.4875 | 0.1103 |
+| 0.3477        | 65.3061 | 7200  | 0.3633          | 0.4942 | 0.1114 |
+| 0.3433        | 66.2132 | 7300  | 0.3637          | 0.4929 | 0.1103 |
+| 0.3431        | 67.1202 | 7400  | 0.3629          | 0.4885 | 0.1102 |
+| 0.3495        | 68.0272 | 7500  | 0.3624          | 0.4932 | 0.1101 |
+| 0.3292        | 68.9342 | 7600  | 0.3615          | 0.4885 | 0.1095 |
+| 0.342         | 69.8413 | 7700  | 0.3641          | 0.4875 | 0.1093 |
+| 0.3406        | 70.7483 | 7800  | 0.3643          | 0.4878 | 0.1098 |
+| 0.3308        | 71.6553 | 7900  | 0.3644          | 0.4899 | 0.1103 |
+| 0.3362        | 72.5624 | 8000  | 0.3653          | 0.4844 | 0.1093 |
+| 0.3265        | 73.4694 | 8100  | 0.3635          | 0.4902 | 0.1099 |
+| 0.3327        | 74.3764 | 8200  | 0.3641          | 0.4844 | 0.1089 |
+| 0.3259        | 75.2834 | 8300  | 0.3645          | 0.4871 | 0.1091 |
+| 0.3327        | 76.1905 | 8400  | 0.3641          | 0.4848 | 0.1093 |
+| 0.3261        | 77.0975 | 8500  | 0.3639          | 0.4875 | 0.1100 |
+| 0.3332        | 78.0045 | 8600  | 0.3632          | 0.4885 | 0.1099 |
+| 0.327         | 78.9116 | 8700  | 0.3635          | 0.4858 | 0.1096 |
+| 0.3291        | 79.8186 | 8800  | 0.3635          | 0.4861 | 0.1090 |
+| 0.3255        | 80.7256 | 8900  | 0.3636          | 0.4909 | 0.1101 |
+| 0.3303        | 81.6327 | 9000  | 0.3633          | 0.4855 | 0.1098 |
+| 0.3293        | 82.5397 | 9100  | 0.3625          | 0.4848 | 0.1093 |
+| 0.3263        | 83.4467 | 9200  | 0.3632          | 0.4848 | 0.1093 |
+| 0.3242        | 84.3537 | 9300  | 0.3641          | 0.4838 | 0.1094 |
+| 0.327         | 85.2608 | 9400  | 0.3646          | 0.4861 | 0.1097 |
+| 0.3297        | 86.1678 | 9500  | 0.3634          | 0.4851 | 0.1097 |
+| 0.3259        | 87.0748 | 9600  | 0.3634          | 0.4841 | 0.1091 |
+| 0.3242        | 87.9819 | 9700  | 0.3635          | 0.4855 | 0.1095 |
+| 0.3156        | 88.8889 | 9800  | 0.3638          | 0.4851 | 0.1092 |
+| 0.3231        | 89.7959 | 9900  | 0.3637          | 0.4858 | 0.1093 |
+| 0.3193        | 90.7029 | 10000 | 0.3635          | 0.4844 | 0.1090 |
+### Framework versions
+- Transformers 4.41.0.dev0
+- Pytorch 2.2.2
+- Datasets 2.19.0
+- Tokenizers 0.19.1

adapter.krl.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b3ae45289f7f364257a13434c3ae3b637f4fbf029ac5ae0d083dcb3e7f539c05
+size 4424388