readme: ---
  language: vi
  datasets:
  - vivos
  - common_voice
  metrics:
  - wer
  pipeline_tag: automatic-speech-recognition
  tags:
  - audio
  - speech
  - speechbrain
  - Transformer
  license: cc-by-nc-4.0
  widget:
  - example_title: Example 1
    src: https://huggingface.co/dragonSwing/wav2vec2-base-vn-270h/raw/main/example.mp3
  - example_title: Example 2
    src: https://huggingface.co/dragonSwing/wav2vec2-base-vn-270h/raw/main/example2.mp3
  model-index:
  - name: Wav2vec2 Base Vietnamese 270h
    results:
    - task: 
        name: Speech Recognition
        type: automatic-speech-recognition
      dataset:
        name: Common Voice vi
        type: common_voice
        args: vi
      metrics:
         - name: Test WER
           type: wer
           value: 9.66
    - task: 
        name: Speech Recognition
        type: automatic-speech-recognition
      dataset:
        name: Common Voice 7.0
        type: mozilla-foundation/common_voice_7_0
        args: vi
      metrics:
         - name: Test WER
           type: wer
           value: 5.57
    - task: 
        name: Speech Recognition
        type: automatic-speech-recognition
      dataset:
        name: Common Voice 8.0
        type: mozilla-foundation/common_voice_8_0
        args: vi
      metrics:
         - name: Test WER
           type: wer
           value: 5.76
    - task: 
        name: Speech Recognition
        type: automatic-speech-recognition
      dataset:
        name: VIVOS
        type: vivos
        args: vi
      metrics:
         - name: Test WER
           type: wer
           value: 3.70
  ---
  # Wav2Vec2-Base-Vietnamese-270h
  Fine-tuned Wav2Vec2 model on Vietnamese Speech Recognition task using about 270h labelled data combined from multiple datasets including [Common Voice](https://huggingface.co/datasets/common_voice), [VIVOS](https://huggingface.co/datasets/vivos), [VLSP2020](https://vlsp.org.vn/vlsp2020/eval/asr). The model was fine-tuned using SpeechBrain toolkit with a custom tokenizer. For a better experience, we encourage you to learn more about [SpeechBrain](https://speechbrain.github.io/).  
  When using this model, make sure that your speech input is sampled at 16kHz.  
  Please refer to [huggingface blog](https://huggingface.co/blog/fine-tune-wav2vec2-english) or [speechbrain](https://github.com/speechbrain/speechbrain/tree/develop/recipes/CommonVoice/ASR/CTC) on how to fine-tune Wav2Vec2 model on a specific language.
  
  ### Benchmark WER result:
  | | [VIVOS](https://huggingface.co/datasets/vivos) | [COMMON VOICE 7.0](https://huggingface.co/datasets/mozilla-foundation/common_voice_7_0) | [COMMON VOICE 8.0](https://huggingface.co/datasets/mozilla-foundation/common_voice_8_0) |
  |---|---|---|---|
  |without LM| 8.23 | 12.15 | 12.15 |
  |with 4-grams LM| 3.70 | 5.57 | 5.76 |
  
  The language model was trained using [OSCAR](https://huggingface.co/datasets/oscar-corpus/OSCAR-2109) dataset on about 32GB of crawled text.
  
  ### Install SpeechBrain
  To use this model, you should install speechbrain > 0.5.10
  
  ### Usage
  The model can be used directly (without a language model) as follows:
  ```python
  from speechbrain.pretrained import EncoderASR
  
  model = EncoderASR.from_hparams(source="dragonSwing/wav2vec2-base-vn-270h", savedir="pretrained_models/asr-wav2vec2-vi")
  model.transcribe_file('dragonSwing/wav2vec2-base-vn-270h/example.mp3')
  # Output: được hồ chí minh coi là một động lực lớn của sự phát triển đất nước
  ```
  
  ### Inference on GPU
  To perform inference on the GPU, add  `run_opts={"device":"cuda"}`  when calling the `from_hparams` method.
  
  ### Evaluation
  The model can be evaluated as follows on the Vietnamese test data of Common Voice 8.0.
  ```python
  import torch
  import torchaudio
  from datasets import load_dataset, load_metric, Audio
  from transformers import Wav2Vec2FeatureExtractor
  from speechbrain.pretrained import EncoderASR
  import re
  test_dataset = load_dataset("mozilla-foundation/common_voice_8_0", "vi", split="test", use_auth_token=True)
  test_dataset = test_dataset.cast_column("audio", Audio(sampling_rate=16_000))
  device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
  wer = load_metric("wer")
  extractor = Wav2Vec2FeatureExtractor.from_pretrained("dragonSwing/wav2vec2-base-vn-270h")
  model = EncoderASR.from_hparams(source="dragonSwing/wav2vec2-base-vn-270h", savedir="pretrained_models/asr-wav2vec2-vi", run_opts={'device': device})
  chars_to_ignore_regex = r'[,?.!\-;:"“%\'�]'
  # Preprocessing the datasets.
  # We need to read the audio files as arrays
  def speech_file_to_array_fn(batch):
    audio = batch["audio"]
    batch["target_text"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()
    batch['speech'] = audio['array']
    return batch
  test_dataset = test_dataset.map(speech_file_to_array_fn)
  
  def evaluate(batch):
    # For padding inputs only
    inputs = extractor(
      batch['speech'], 
      sampling_rate=16000, 
      return_tensors="pt", 
      padding=True, 
      do_normalize=False
    ).input_values
    input_lens = torch.ones(inputs.shape[0])
    pred_str, pred_tokens = model.transcribe_batch(inputs, input_lens)
    batch["pred_strings"] = pred_str
    
    re