readme: ---
  language: zh-HK
  license: apache-2.0
  tags:
  - automatic-speech-recognition
  - generated_from_trainer
  - hf-asr-leaderboard
  - robust-speech-event
  datasets:
  - common_voice
  model-index:
  - name: Wav2Vec2 XLS-R 300M Cantonese (zh-HK) LM
    results:
    - task:
        name: Automatic Speech Recognition
        type: automatic-speech-recognition
      dataset:
        name: Common Voice
        type: common_voice
        args: zh-HK
      metrics:
      - name: Test CER
        type: cer
        value: 24.09
    - task:
        name: Automatic Speech Recognition
        type: automatic-speech-recognition
      dataset:
        name: Common Voice 7
        type: mozilla-foundation/common_voice_7_0
        args: zh-HK
      metrics:
      - name: Test CER
        type: cer
        value: 23.1
    - task:
        name: Automatic Speech Recognition
        type: automatic-speech-recognition
      dataset:
        name: Common Voice 8
        type: mozilla-foundation/common_voice_8_0
        args: zh-HK
      metrics:
      - name: Test CER
        type: cer
        value: 23.02
    - task:
        name: Automatic Speech Recognition
        type: automatic-speech-recognition
      dataset:
        name: Robust Speech Event - Dev Data
        type: speech-recognition-community-v2/dev_data
        args: zh-HK
      metrics:
      - name: Test CER
        type: cer
        value: 56.86
    - task:
        name: Automatic Speech Recognition
        type: automatic-speech-recognition
      dataset:
        name: Robust Speech Event - Test Data
        type: speech-recognition-community-v2/eval_data
        args: zh-HK
      metrics:
      - name: Test CER
        type: cer
        value: 55.76
  ---
  
  # Wav2Vec2 XLS-R 300M Cantonese (zh-HK) LM
  
  Wav2Vec2 XLS-R 300M Cantonese (zh-HK) LM is an automatic speech recognition model based on the [XLS-R](https://arxiv.org/abs/2111.09296) architecture. This model is a fine-tuned version of [Wav2Vec2-XLS-R-300M](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the `zh-HK` subset of the [Common Voice](https://huggingface.co/datasets/common_voice) dataset. A 5-gram Language model, trained on multiple [PyCantonese](https://pycantonese.org/data.html) corpora, was then subsequently added to this model.
  
  This model was trained using HuggingFace's PyTorch framework and is part of the [Robust Speech Challenge Event](https://discuss.huggingface.co/t/open-to-the-community-robust-speech-recognition-challenge/13614) organized by HuggingFace. All training was done on a Tesla V100, sponsored by OVH.
  
  All necessary scripts used for training could be found in the [Files and versions](https://huggingface.co/w11wo/wav2vec2-xls-r-300m-zh-HK-lm-v2/tree/main) tab, as well as the [Training metrics](https://huggingface.co/w11wo/wav2vec2-xls-r-300m-zh-HK-lm-v2/tensorboard) logged via Tensorboard.
  
  As for the N-gram language model training, we followed the [blog post tutorial](https://huggingface.co/blog/wav2vec2-with-ngram) provided by HuggingFace.
  
  ## Model
  
  | Model                             | #params | Arch. | Training/Validation data (text) |
  | --------------------------------- | ------- | ----- | ------------------------------- |
  | `wav2vec2-xls-r-300m-zh-HK-lm-v2` | 300M    | XLS-R | `Common Voice zh-HK` Dataset    |
  
  ## Evaluation Results
  
  The model achieves the following results on evaluation without a language model:
  
  | Dataset                          | CER    |
  | -------------------------------- | ------ |
  | `Common Voice`                   | 31.73% |
  | `Common Voice 7`                 | 23.11% |
  | `Common Voice 8`                 | 23.02% |
  | `Robust Speech Event - Dev Data` | 56.60% |
  
  With the addition of the language model, it achieves the following results:
  
  | Dataset                          | CER    |
  | -------------------------------- | ------ |
  | `Common Voice`                   | 24.09% |
  | `Common Voice 7`                 | 23.10% |
  | `Common Voice 8`                 | 23.02% |
  | `Robust Speech Event - Dev Data` | 56.86% |
  
  ## Training procedure
  
  The training process did not involve the addition of a language model. The following results were simply lifted from the original automatic speech recognition [model training](https://huggingface.co/w11wo/wav2vec2-xls-r-300m-zh-HK-v2).
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  
  - `learning_rate`: 0.0001
  - `train_batch_size`: 8
  - `eval_batch_size`: 8
  - `seed`: 42
  - `gradient_accumulation_steps`: 4
  - `total_train_batch_size`: 32
  - `optimizer`: Adam with `betas=(0.9, 0.999)` and `epsilon=1e-08`
  - `lr_scheduler_type`: linear
  - `lr_scheduler_warmup_steps`: 2000
  - `num_epochs`: 100.0
  - `mixed_precision_training`: Native AMP
  
  ### Training results
  
  | Training Loss | Epoch | Step  | Validation Loss |  Wer   |  Cer   |
  | :-----------: | :---: | :---: | :-------------: | :----: | :----: |
  |    69.8341    | 1.34  |  500  |     80.0722     |  1.0   |  1.0   |
  |    6.6418     | 2.68  | 1000  |     6.6346      |  1.0   |  1.0   |
  |    6.2419     | 4.02  | 1500  |     6.2909      |  1.0   |  1.0   |
  |    6.0813     | 