jankoko/PALF-LF-Whisper-small

This model is a fine-tuned version of openai/whisper-small on the wTIMIT-US dataset using the LF-Mask (Low-Frequency Masking) augmentation method. LF-Mask applies frequency masking below 1.5 kHz for voiced phonemes, motivated by findings from acoustic studies of whispered speech.

It was evaluated on both normal and whispered speech subsets using Word Error Rate (WER) as the primary metric. While LF-Mask maintained comparable performance to SpecAugment, it did not produce statistically significant improvements in either speaking mode.

For a full explanation of the masking range and its acoustic motivation, refer to Section 3 of the thesis linked below.

Evaluation Results on wTIMIT-US (Test Set)

Setup	Training Data	Augmentation	WER (Normal)	WER (Whispered)
No Fine-tuning	Zero-shot	None	5.0	13.7
Baseline	Both modes	None	5.8	11.7
SpecAugment	Both modes	SpecAugment (LD)	5.2	12.3
LF-Mask (Ours)	Both modes	Masking < vowel range	5.3 (p=0.624)	11.9 (p=0.384)

Compared to SpecAugment, LF-Mask showed no statistically significant change in WER for either normal or whispered speech (p=0.624 and p=0.384, respectively).

Cite as

Kokowski, J. (2025). F0-Based Masking Policies for Self-Supervised Whispered Speech Recognition. Master’s Thesis, University of Groningen, Campus Fryslân.
Available at: https://campus-fryslan.studenttheses.ub.rug.nl/id/eprint/674

If you use this model or build upon this work, please cite the thesis above.

Model: Whisper-small
Augmentation: LF-Mask
Evaluation toolkit: SCTK (sclite)
Notes: For complete results, including MAPSSWE and CER scores, refer to Section 5 of the thesis.

🔗 Related Models

• SpecAugment Baseline
• F0-Mask Version
• F1-Mask Version
• LF-Mask Version ← current