readme: ---
  language: [mr,hi]
  datasets:
  - openslr
  - interspeech_2021_asr
  metrics:
  - wer
  tags:
  - audio
  - automatic-speech-recognition
  - speech
  - xlsr-fine-tuning-week
  - hindi
  - marathi
  license: apache-2.0
  model-index:
  - name: XLSR Wav2Vec2 Large 53 Hindi-Marathi by Tanmay Laud
    results:
    - task: 
        name: Speech Recognition
        type: automatic-speech-recognition
      dataset:
        name: OpenSLR hi, OpenSLR mr
        type: openslr, interspeech_2021_asr
      metrics:
         - name: Test WER
           type: wer
           value: 23.736641
  ---
  
  # Wav2Vec2-Large-XLSR-53-Hindi-Marathi
  Fine-tuned facebook/wav2vec2-large-xlsr-53 on Hindi and Marathi using the OpenSLR SLR64 datasets. When using this model, make sure that your speech input is sampled at 16kHz.
  
  ## Installation 
  ```bash
  pip install git+https://github.com/huggingface/transformers.git datasets librosa torch==1.7.0 torchaudio==0.7.0 jiwer
  ```
  
  ## Eval dataset:
  ```bash
  wget https://www.openslr.org/resources/103/Marathi_test.zip  -P data/marathi
  unzip -P "K3[2?do9" data/marathi/Marathi_test.zip -d data/marathi/.
  tar -xzf data/marathi/Marathi_test.tar.gz -C data/marathi/.
  wget https://www.openslr.org/resources/103/Hindi_test.zip  -P data/hindi
  unzip -P "w9I2{3B*" data/hindi/Hindi_test.zip -d data/hindi/.
  tar -xzf data/hindi/Hindi_test.tar.gz -C data/hindi/.
  wget  -O test.csv 'https://filebin.net/snrz6bt13usv8w2e/test_large.csv?t=ps3n99ho'
  #If download does not work, paste this link in browser: https://filebin.net/snrz6bt13usv8w2e/test_large.csv
  ```
  ## Usage
   The model can be used directly (without a language model) as follows, assuming you have a dataset with Marathi text and path fields:
  
  
  
  ```python
  import torch
  import torchaudio
  import librosa
  from datasets import load_dataset
  from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
  
  from datasets import load_metric, Dataset
  from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC
  
  wer = load_metric("wer")
  processor = Wav2Vec2Processor.from_pretrained('tanmaylaud/wav2vec2-large-xlsr-hindi-marathi')
  model = Wav2Vec2ForCTC.from_pretrained('tanmaylaud/wav2vec2-large-xlsr-hindi-marathi').to("cuda")
  
  # Preprocessing the datasets.
  # We need to read the audio files as arrays
  def speech_file_to_array_fn(batch):
      batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"])
      speech_array, sampling_rate = torchaudio.load(batch["path"])
      batch["speech"] = speech_array[0].numpy()
      batch["sampling_rate"] = sampling_rate
      batch["target_text"] = batch["sentence"]
      batch["speech"] = librosa.resample(np.asarray(batch["speech"]), sampling_rate, 16_000)
      batch["sampling_rate"] = 16_000
      return batch
  
  test_data= test_data.map(speech_file_to_array_fn)
  inputs = processor(test_data["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)
  
  with torch.no_grad():
      logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits
  
  predicted_ids = torch.argmax(logits, dim=-1)
  
  print("Prediction:", processor.batch_decode(predicted_ids))
  print("Reference:", test_data["text"][:2])
  ```
  
  
  
  # Code For Evaluation on OpenSLR (Hindi + Marathi : https://filebin.net/snrz6bt13usv8w2e/test_large.csv)
  ```python
  import torchaudio
  import torch
  import librosa
  import numpy as np
  import re
  
  test = Dataset.from_csv('test.csv')
  
  
  chars_to_ignore_regex = '[\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\,\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\?\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\.\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\!\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\-\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\;\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\