base_model: openai/whisper-small
library_name: peft
license: mit
tags:
- whisper-small
- speech_to_text
- ASR
- french
language:
- fr
demo: https://huggingface.co/spaces/visalkao/whisper-small-french-finetuned
Model Card for Model ID
Model Details
Model Description
- Developed by: Visal KAO
- Model type: Speech Recognition
- Language(s) (NLP): French
- License: MIT
- Finetuned from model : Whisper-small
Model Sources [optional]
- Repository: openai/whisper-small
Dataset
This model is finetuned on 50% of French Single Speaker Speech Dataset on kaggle (Only lesmis).
- Link to dataset : (https://www.kaggle.com/datasets/bryanpark/french-single-speaker-speech-dataset)
Uses
The goal of this project is to finetune whisper-small model to improve its accuracy for french transcription.
The reason why I chose Whisper-small is due to its size and versatility. My primary objective is to build/finetune a small model to get acceptable results.
Direct Use
Live Demo : https://huggingface.co/spaces/visalkao/whisper-small-french-finetuned
Bias, Risks, and Limitations
As this model has less than 250 millions parameters, which is quite small considering its objective is to transcribe speech, it also has its own limitation.
The Word Error Rate (WER) of this finetuned model is approximately 0.17 (17%).
For reference, the original Whisper-small's WER is around 0.27 (27%) on the same dataset.
Training Hyperparameters
This model is trained using LoRa with these hyperparamters:
- per_device_train_batch_size=3,
- gradient_accumulation_steps=1,
- learning_rate=1e-3,
- num_train_epochs=7,
- evaluation_strategy="epoch",
- fp16=True,
- per_device_eval_batch_size=1,
- generation_max_length=225,
- logging_steps=10,
- remove_unused_columns=False,
- label_names=["labels"],
- predict_with_generate=True,
Results
Before finetuning, The Word Error Rate on this dataset is approximately 0.27.
After finetuning, it drops down 0.1 to 0.17 or 17% wer (On testing data).
Here is the training log:
| Epoch | Training Loss | Validation Loss | WER |
|---|---|---|---|
| 1 | 0.369600 | 0.404414 | 26.665379 |
| 2 | 0.273200 | 0.361762 | 22.793976 |
| 3 | 0.308800 | 0.344289 | 24.454528 |
| 4 | 0.131600 | 0.318023 | 21.847847 |
| 5 | 0.117400 | 0.311023 | 19.134968 |
| 6 | 0.035700 | 0.301410 | 18.922572 |
| 7 | 0.013900 | 0.315151 | 16.972388 |