readme: # wav2vec 2.0 multilingual ( Finetued )
  The base model pretrained on 16kHz sampled speech audio. When using the model make sure that your speech input is also sampled at 16Khz. Note that this model should be fine-tuned on a downstream task, like Automatic Speech Recognition. Check out [this blog](https://huggingface.co/blog/fine-tune-wav2vec2-english) for more information.
  
  [Paper](https://arxiv.org/abs/2006.13979)
  
  Authors: Alexis Conneau, Alexei Baevski, Ronan Collobert, Abdelrahman Mohamed, Michael Auli
  
  **Abstract** This paper presents XLSR which learns cross-lingual speech representations by pretraining a single model from the raw waveform of speech in multiple languages. We build on wav2vec 2.0 which is trained by solving a contrastive task over masked latent speech representations and jointly learns a quantization of the latents shared across languages. The resulting model is fine-tuned on labeled data and experiments show that cross-lingual pretraining significantly outperforms monolingual pretraining. On the CommonVoice benchmark, XLSR shows a relative phoneme error rate reduction of 72% compared to the best known results. On BABEL, our approach improves word error rate by 16% relative compared to a comparable system. Our approach enables a single multilingual speech recognition model which is competitive to strong individual models. Analysis shows that the latent discrete speech representations are shared across languages with increased sharing for related languages. We hope to catalyze research in low-resource speech understanding by releasing XLSR-53, a large model pretrained in 53 languages.
  
  The original model can be found under https://github.com/pytorch/fairseq/tree/master/examples/wav2vec#wav2vec-20.
  
  
  
  Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) in Persian (Farsi) using [Common Voice](https://huggingface.co/datasets/common_voice) plus Our own created Dataset(1/3 of total dataset). When using this model, make sure that your speech input is sampled at 16kHz.
  
  ## Evaluation: 🌡️
  We have evaluated the model on private dataset with different type of audios (unfortunately the dataset for testing and validation is not publicly available but to see a sample of the dataset [check this link)](https://github.com/shenasa-ai/speech2text#part-of-our-dataset-v01--) :
  
  |                             Name                             | test dataset  (wer) |
  | :----------------------------------------------------------: | :-----------------: |
  | [m3hrdadfi/wav2vec2-large-xlsr-persian-v3](https://huggingface.co/m3hrdadfi/wav2vec2-large-xlsr-persian-v3) |       0.56754       |
  | [This New Model](https://huggingface.co/masoudmzb/wav2vec2-xlsr-multilingual-53-fa) |     **0.40815**     |
  |                   Base Multilingual Model                    |       0.69746       |
  
  - This Table show if we add more data we will have much better result
  
  
  ## How to use❓
  
  ### Use FineTuned Model
  
  This model is finetuned on [m3hrdadfi/wav2vec2-large-xlsr-persian-v3](https://huggingface.co/m3hrdadfi/wav2vec2-large-xlsr-persian-v3) , so the process for train or evaluation is same
  
  > ```bash
  > # requirement packages
  > !pip install git+https://github.com/huggingface/datasets.git
  > !pip install git+https://github.com/huggingface/transformers.git
  > !pip install torchaudio
  > !pip install librosa
  > !pip install jiwer
  > !pip install parsivar
  > !pip install num2fawords
  > ```
  
  
  
  **Normalizer**
  
  ```bash
  # Normalizer
  !wget -O normalizer.py https://huggingface.co/m3hrdadfi/"wav2vec2-large-xlsr-persian-v3/raw/main/dictionary.py
  !wget -O normalizer.py https://huggingface.co/m3hrdadfi/"wav2vec2-large-xlsr-persian-v3/raw/main/normalizer.py
  
  ```
  
  
  
  If you are not sure your transcriptions are clean or not (having weird characters or any other alphabete chars ) use this code provided by  [m3hrdadfi/wav2vec2-large-xlsr-persian-v3](https://huggingface.co/m3hrdadfi/wav2vec2-large-xlsr-persian-v3)  
  
  
  
  **Cleaning** (Fill the data part with your own data dir)
  
  ```python
  
  from normalizer import normalizer
  
  def cleaning(text):
      if not isinstance(text, str):
          return None
  
      return normalizer({"sentence": text}, return_dict=False)
  
  # edit these parts with your own data directory
  
  data_dir = "data"
  
  
  test = pd.read_csv(f"{data_dir}/yourtest.tsv", sep="	")
  test["path"] = data_dir + "/clips/" + test["path"]
  print(f"Step 0: {len(test)}")
  
  test["status"] = test["path"].apply(lambda path: True if os.path.exists(path) else None)
  test = test.dropna(subset=["path"])
  test = test.drop("status", 1)
  print(f"Step 1: {len(test)}")
  
  test["sentence"] = test["sentence"].apply(lambda t: cleaning(t))
  test = test.dropna(subset=["sentence"])
  print(f"Step 2: {len(test)}")
  
  test = test.reset_index(drop=True)
  print(test.head())
  
  test = test[["path", "sentence"]]
  test.to_csv("/content/test.csv", sep="	", encoding="utf-8", index=False)
  ```
  
  
  
  **Prediction**
  
  ```python
  import numpy as np
  import pandas as pd
  
  import l