readme: ---
  language: pt
  datasets:
  - common_voice 
  - mls
  - cetuc
  - lapsbm
  - voxforge
  - tedx
  - sid
  metrics:
  - wer
  tags:
  - audio
  - speech
  - wav2vec2
  - pt
  - portuguese-speech-corpus
  - automatic-speech-recognition
  - speech
  - PyTorch
  license: apache-2.0
  ---
  
  # cetuc100-xlsr: Wav2vec 2.0 with CETUC Dataset
  
  This is a the demonstration of a fine-tuned Wav2vec model for Brazilian Portuguese using the [CETUC](http://www02.smt.ufrj.br/~igor.quintanilha/alcaim.tar.gz) dataset. This dataset contains approximately 145 hours of Brazilian Portuguese speech distributed among 50 male and 50 female speakers, each pronouncing approximately 1,000 phonetically balanced sentences selected from the [CETEN-Folha](https://www.linguateca.pt/cetenfolha/) corpus.
  
  In this notebook the model is tested against other available Brazilian Portuguese datasets. 
  
  | Dataset                        |  Train | Valid |  Test |
  |--------------------------------|-------:|------:|------:|
  | CETUC                          |  94h   |    -- |  5.4h |
  | Common Voice                   |        |    -- |  9.5h |
  | LaPS BM                        |        |    -- |  0.1h |
  | MLS                            |        |    -- |  3.7h |
  | Multilingual TEDx (Portuguese) |        |    -- |  1.8h |
  | SID                            |        |    -- |  1.0h |
  | VoxForge                       |        |    -- |  0.1h |
  | Total                          |        |    -- | 21.6h |
  
  
  #### Summary
  
  |  | CETUC          | CV             | LaPS           | MLS            | SID            | TEDx           | VF             | AVG            |
  |----------------------|---------------|----------------|----------------|----------------|----------------|----------------|----------------|----------------|
  | cetuc\_100 (demonstration below)| 0.446 | 0.856 | 0.089 | 0.967 | 1.172 | 0.929 | 0.902 | 0.765 | 
  | cetuc\_100 + 4-gram (demonstration below)|0.339 | 0.734 | 0.076 | 0.961 | 1.188 | 1.227 | 0.801 | 0.760 |
  
  ## Demonstration
  
  
  ```python
  MODEL_NAME = "lgris/cetuc100-xlsr" 
  ```
  
  ### Imports and dependencies
  
  
  ```python
  %%capture
  !pip install torch==1.8.2+cu111 torchvision==0.9.2+cu111 torchaudio===0.8.2 -f https://download.pytorch.org/whl/lts/1.8/torch_lts.html
  !pip install datasets
  !pip install jiwer
  !pip install transformers
  !pip install soundfile
  !pip install pyctcdecode
  !pip install https://github.com/kpu/kenlm/archive/master.zip
  ```
  
  
  ```python
  import jiwer
  import torchaudio
  from datasets import load_dataset, load_metric
  from transformers import (
      Wav2Vec2ForCTC,
      Wav2Vec2Processor,
  )
  from pyctcdecode import build_ctcdecoder
  import torch
  import re
  import sys
  ```
  
  ### Helpers
  
  
  ```python
  chars_to_ignore_regex = '[\,\?\.\!\;\:\"]'  # noqa: W605
  
  def map_to_array(batch):
      speech, _ = torchaudio.load(batch["path"])
      batch["speech"] = speech.squeeze(0).numpy() 
      batch["sampling_rate"] = 16_000 
      batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower().replace("’", "'")
      batch["target"] = batch["sentence"]
      return batch
  ```
  
  
  ```python
  def calc_metrics(truths, hypos):
      wers = []
      mers = []
      wils = []
      for t, h in zip(truths, hypos):
          try:
              wers.append(jiwer.wer(t, h))
              mers.append(jiwer.mer(t, h))
              wils.append(jiwer.wil(t, h))
          except: # Empty string?
              pass
      wer = sum(wers)/len(wers)
      mer = sum(mers)/len(mers)
      wil = sum(wils)/len(wils)
      return wer, mer, wil
  ```
  
  
  ```python
  def load_data(dataset):
      data_files = {'test': f'{dataset}/test.csv'}
      dataset = load_dataset('csv', data_files=data_files)["test"]
      return dataset.map(map_to_array)
  ```
  
  ### Model
  
  
  ```python
  class STT:
  
      def __init__(self, 
                   model_name, 
                   device='cuda' if torch.cuda.is_available() else 'cpu', 
                   lm=None):
          self.model_name = model_name
          self.model = Wav2Vec2ForCTC.from_pretrained(model_name).to(device)
          self.processor = Wav2Vec2Processor.from_pretrained(model_name)
          self.vocab_dict = self.processor.tokenizer.get_vocab()
          self.sorted_dict = {
              k.lower(): v for k, v in sorted(self.vocab_dict.items(), 
                                              key=lambda item: item[1])
          }
          self.device = device
          self.lm = lm
          if self.lm:            
              self.lm_decoder = build_ctcdecoder(
                  list(self.sorted_dict.keys()),
                  self.lm
              )
  
      def batch_predict(self, batch):
          features = self.processor(batch["speech"], 
                                    sampling_rate=batch["sampling_rate"][0], 
                                    padding=True, 
                                    return_tensors="pt")
          input_values = features.input_values.to(self.device)
          attention_mask = features.attention_mask.to(self.device)
          with torch.no_grad():
              logits = self.model(input_values, attention_mask=attention_mask).logits
    