readme: ---
  language: zh
  tags:
  - automatic-speech-recognition
  license: cc-by-sa-4.0
  datasets:
  - common_voice
  metrics:
  - cer
  ---
  
  # Wav2vec2-large-xlsr-cantonese
  This model was based on [wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53), finetuned using Common Voice/zh-HK/6.1.0.
  
  The training code is similar to [user ctl](https://huggingface.co/ctl/wav2vec2-large-xlsr-cantonese), except that the number of training epochs was 80 (doubled) and fp16_backend is apex. The model was trained using a single RTX 3090 and docker image is nvidia/cuda:11.1-cudnn8-devel.
  
  CER is 15.11% when evaluate against common voice zh-HK test set.
  
  # Result (CER)
  15.11% 
  
  # Source Code
  See this GitHub Repo [cantonese-selfish-project](https://github.com/scottykwok/cantonese-selfish-project/) and [demo video](https://youtu.be/k_9RQ-ilGEc).
  
  # Usage
  ```python
  import soundfile as sf
  import torch
  from datasets import load_dataset
  from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
  
  # load pretrained model
  processor = Wav2Vec2Processor.from_pretrained("scottykwok/wav2vec2-large-xlsr-cantonese")
  model = Wav2Vec2ForCTC.from_pretrained("scottykwok/wav2vec2-large-xlsr-cantonese")
  
  # load audio - must be 16kHz mono
  audio_input, sample_rate = sf.read('audio.wav')
  
  # pad input values and return pt tensor
  input_values = processor(audio_input, sampling_rate=sample_rate, return_tensors="pt").input_values
  
  # INFERENCE
  # retrieve logits & take argmax
  logits = model(input_values).logits
  predicted_ids = torch.argmax(logits, dim=-1)
  
  # transcribe
  transcription = processor.decode(predicted_ids[0])
  print("-" *20)
  print("Transcription:\n", transcription.lower())
  print("-" *20)
  
  ```
  