transformers safetensors whisper automatic-speech-recognition generated_from_trainer base_model:openai/whisper-large-v3 base_model:finetune:openai/whisper-large-v3 license:apache-2.0 endpoints_compatible region:us
---
base_model: openai/whisper-large-v3
library_name: transformers
license: apache-2.0
metrics:
- wer
tags:
- generated_from_trainer
model-index:
- name: whisper-large-v3-chichewa
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# whisper-large-v3-chichewa

This model is a fine-tuned version of [openai/whisper-large-v3](https://huggingface.co/openai/whisper-large-v3) on a Chichewa dataset. 
The dataset details will be provided at a later stage.
It achieves the following results on the evaluation set:
- Loss: 3.1427
- Wer: 101.3340

## Model description

More information needed

## Intended uses & limitations

The model is fine-tuned to perfom transcription of Chichewa. There are several versions of this model, please refer to the [usage example notebook](./usage-example.ipynb) to see how to find the commit which has model with best WER.
Alternatively, you can explore the Files and Versions tab, go to commits and find the commit with best WER (around 61). 
Also, its worth noting that the model repo doesnt have a ```tokenizer.json```, as a results the model needs to be loaded using WhisperProcessor 
instead of AutoModel or other modules in Transformer.

## Source of Funding for this Work
The dataset used to fine-tune this model, as well as the compute resources, were provided by [Opportunity International](https://www.globalcitizen.org/en/partners/opportunity-international/?gad_source=1&gbraid=0AAAAACnN8MzEIzvf0oKqHW5bw14A4IvGY&gclid=CjwKCAjw9p24BhB_EiwA8ID5Bptp-7RgECcozDIe_6Owjb2g0wClWOKv4-NsEdtXpKx4FGPvOlBPQBoC9SMQAvD_BwE).  
This was part of a project in Malawi aimed at supporting the deployment of an LLM-based chatbot for agriculture, with the capability to handle voice interactions in the local language, Chichewa. 
A total of 30 hours was collected for this dataset but due to data quality issues, only 25 hours was used. 
About 30 minutes was also removed to be used as hold-out for further model evaluation.

## Training and evaluation data

More information needed

## Training procedure
Most of the training for this model involved trying to varying speech dataset sizes (5 hours, 10 hours up to 24 hours). 
As such, the different model commits represent different data sizes. More details will be added to each model commit.

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 32
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- training_steps: 10000
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch   | Step  | Validation Loss | Wer      |
|:-------------:|:-------:|:-----:|:---------------:|:--------:|
| 0.9229        | 7.0423  | 1000  | 2.0780          | 86.3539  |
| 0.1427        | 14.0845 | 2000  | 2.5560          | 83.5493  |
| 0.087         | 21.1268 | 3000  | 2.6909          | 80.8704  |
| 0.0742        | 28.1690 | 4000  | 2.8007          | 81.8982  |
| 0.065         | 35.2113 | 5000  | 2.8871          | 84.3639  |
| 0.0627        | 42.2535 | 6000  | 2.9465          | 84.5334  |
| 0.0586        | 49.2958 | 7000  | 3.0451          | 114.1600 |
| 0.063         | 56.3380 | 8000  | 3.0983          | 82.6964  |
| 0.0588        | 63.3803 | 9000  | 3.1352          | 81.0180  |
| 0.0591        | 70.4225 | 10000 | 3.1427          | 101.3340 |


### Framework versions

- Transformers 4.45.1
- Pytorch 2.0.1
- Datasets 3.0.1
- Tokenizers 0.20.0
