---
base_model: openai/whisper-small
library_name: peft
license: mit
tags:
- whisper-small
- speech_to_text
- ASR
- french
language:
- fr
demo: https://huggingface.co/spaces/visalkao/whisper-small-french-finetuned
---

# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->



## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->



- **Developed by:** Visal KAO
- **Model type:** Speech Recognition 
- **Language(s) (NLP):** French
- **License:** MIT
- **Finetuned from model :** Whisper-small

### Model Sources [optional]

<!-- Provide the basic links for the model. -->

- **Repository:** openai/whisper-small
## Dataset
This model is finetuned on 50% of French Single Speaker Speech Dataset on kaggle (Only lesmis).
- **Link to dataset :** (https://www.kaggle.com/datasets/bryanpark/french-single-speaker-speech-dataset)

## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
The goal of this project is to finetune whisper-small model to improve its accuracy for french transcription. 

The reason why I chose Whisper-small is due to its size and versatility. My primary objective is to build/finetune a small model to get acceptable results.
### Direct Use

**Live Demo :** https://huggingface.co/spaces/visalkao/whisper-small-french-finetuned

## Bias, Risks, and Limitations

As this model has less than 250 millions parameters, which is quite small considering its objective is to transcribe speech, it also has its own limitation. 

The Word Error Rate (WER) of this finetuned model is approximately 0.17 (17%).

For reference, the original Whisper-small's WER is around 0.27 (27%) on the same dataset.




## Training Hyperparameters
This model is trained using LoRa with these hyperparamters: 

* per_device_train_batch_size=3,
* gradient_accumulation_steps=1,
* learning_rate=1e-3,
* num_train_epochs=7,
* evaluation_strategy="epoch",
* fp16=True,
* per_device_eval_batch_size=1,
* generation_max_length=225,
* logging_steps=10,
* remove_unused_columns=False, 
* label_names=["labels"], 
* predict_with_generate=True,


## Results

Before finetuning, The Word Error Rate on this dataset is approximately 0.27. 

After finetuning, it drops down 0.1 to 0.17 or 17% wer (On testing data).

Here is the training log:

| Epoch | Training Loss | Validation Loss | WER        |
|-------|--------------|----------------|------------|
| 1     | 0.369600     | 0.404414       | 26.665379  |
| 2     | 0.273200     | 0.361762       | 22.793976  |
| 3     | 0.308800     | 0.344289       | 24.454528  |
| 4     | 0.131600     | 0.318023       | 21.847847  |
| 5     | 0.117400     | 0.311023       | 19.134968  |
| 6     | 0.035700     | 0.301410       | 18.922572  |
| 7     | 0.013900     | 0.315151       | 16.972388  |