readme: ---
  license: mit
  tags:
  - automatic-speech-recognition
  - asr
  - pytorch
  - wav2vec2
  - wolof
  - wo
  model-index:
  - name: wav2vec2-xls-r-300m-wolof-lm
    results:
    - task: 
        name: Speech Recognition
        type: automatic-speech-recognition
   
      metrics:
         - name: Test WER
           type: wer
           value: 21.25
         - name: Validation Loss
           type: Loss
           value: 0.36
  
  ---
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  # wav2vec2-xls-r-300m-wolof-lm 
  
  Wolof is a language spoken in Senegal and neighbouring countries, this language is not too well represented, there are few resources in the field of Text en speech
  In this sense we aim to bring our contribution to this, it is in this sense that enters this repo. 
  
  This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) ,with a language model that is fine-tuned  with the largest available speech dataset of the  [ALFFA_PUBLIC](https://github.com/besacier/ALFFA_PUBLIC/tree/master/ASR/WOLOF)
  
  It achieves the following results on the evaluation set:
  - Loss: 0.367826	
  - Wer: 0.212565
  
  ## Model description
  The duration of the training data is 16.8 hours, which we have divided into 10,000 audio files for the training and 3,339 for the test.
  ## Training and evaluation data
  We eval the model at every 1500 step , and log it .  and save at every 33340 step
  ### Training hyperparameters
  The following hyperparameters were used during training:
  - learning_rate: 1e-4
  - train_batch_size: 3
  - eval_batch_size : 8
  - total_train_batch_size: 64
  - total_eval_batch_size: 64
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - lr_scheduler_warmup_steps: 1000
  - num_epochs: 10.0
  
  ### Training results
  
  | Step    | Training Loss | Validation Loss | Wer    |
  |:-------:|:-------------:|:---------------:|:------:|
  | 1500	 | 2.854200	 |0.642243	 |0.543964  |
  | 3000	| 0.599200 |  0.468138 |  	0.429549|
  | 4500	| 0.468300 |	0.433436 |	0.405644|
  | 6000	| 0.427000 |	0.384873 |	0.344150|
  | 7500	| 0.377000 |	0.374003 |	0.323892|
  | 9000	| 0.337000 |	0.363674 |	0.306189|
  | 10500	| 0.302400	| 0.349884	 |0 .283908 |
  | 12000	| 0.264100	| 0.344104  |0.277120|
  | 13500	|0 .254000	|0.341820	 |0.271316|
  | 15000 |	0.208400|	0.326502 |	0.260695|
  | 16500 |	0.203500|	0.326209 |	0.250313|
  | 18000	|0.159800	|0.323539 |	0.239851|
  | 19500 |	0.158200 |	0.310694 |	0.230028|
  | 21000 |	0.132800 |	0.338318 |	0.229283|
  | 22500 |	0.112800 |	0.336765 |	0.224145|
  | 24000 |	0.103600 |	0.350208 |	0.227073 |
  | 25500 |	0.091400 |	0.353609 |	0.221589 |
  | 27000 | 0.084400 |	0.367826 |	0.212565 |
  
  
  ## Usage
  The model can be used directly as follows:
  ```python
  import librosa
  import warnings
  from transformers import AutoProcessor, AutoModelForCTC
  from datasets import Dataset, DatasetDict
  from datasets import load_metric
  
  wer_metric = load_metric("wer")
  
  wolof = pd.read_csv('Test.csv') # wolof contains the columns of file , and transcription
  wolof = DatasetDict({'test': Dataset.from_pandas(wolof)})
  
  chars_to_ignore_regex = '[\"\?\.\!\-\;\:\(\)\,]'
  
  def remove_special_characters(batch):
      batch["transcription"] = re.sub(chars_to_ignore_regex, '', batch["transcription"]).lower() + " "
      return batch
      
      
  wolof = wolof.map(remove_special_characters)
  
  processor = AutoProcessor.from_pretrained("abdouaziiz/wav2vec2-xls-r-300m-wolof-lm")
  model = AutoModelForCTC.from_pretrained("abdouaziiz/wav2vec2-xls-r-300m-wolof-lm")
  
  warnings.filterwarnings("ignore")
  def speech_file_to_array_fn(batch):
      speech_array, sampling_rate = librosa.load(batch["file"], sr = 16000)
      batch["speech"] = speech_array.astype('float16')
      batch["sampling_rate"] = sampling_rate
      batch["target_text"] = batch["transcription"]
      return batch
   
  wolof = wolof.map(speech_file_to_array_fn, remove_columns=wolof.column_names["test"], num_proc=1)   
    
  def map_to_result(batch):
      model.to("cuda")
      input_values = processor(
        batch["speech"], 
        sampling_rate=batch["sampling_rate"], 
        return_tensors="pt"
      ).input_values.to("cuda")
  
      with torch.no_grad():
          logits = model(input_values).logits
          pred_ids = torch.argmax(logits, dim=-1)
          batch["pred_str"] = processor.batch_decode(pred_ids)[0]
  
      return batch
     
   results = wolof["test"].map(map_to_result) 
   
   print("Test WER: {:.3f}".format(wer_metric.compute(predictions=results["pred_str"], references=results["transcription"])))
   
  ```
  
  ## PS:
  
  The results obtained can be improved by using :
  
  - Wav2vec2 + language model . 
  - Build a Spellcheker from the text of the data
  - Sentence Edit Distance