<p dir="auto">Hello everyone,</p>
<p dir="auto">I am having an issue when finetuning OpenAI's Whisper Medium on Mozilla's Common Voice 11 Dataset with the Arabic language.<br>
The training and validation loss are both decreasing but the WER is being 100% after some steps (specially when the loss becomes &lt; 1) and I see that the model is performing well and that WER is just miscalculated.<br>
<a target="_blank" rel="noopener noreferrer nofollow" href="https://user-images.githubusercontent.com/96656595/232808398-936c0e2b-f8b3-4289-a6bd-f21f64236efe.png"><img src="https://user-images.githubusercontent.com/96656595/232808398-936c0e2b-f8b3-4289-a6bd-f21f64236efe.png" alt="hf_issue" style="max-width: 100%;"></a></p>
<p dir="auto">Notes :</p>
<ul dir="auto">
<li>This error is just happening with the medium model, other models (small, tiny, large-v2 ,etc.) are working fine.</li>
<li>I am following the famous blog about Whisper's finetuning (<a href="https://huggingface.co/blog/fine-tune-whisper" rel="nofollow">https://huggingface.co/blog/fine-tune-whisper</a>).<br>
<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/sanchit-gandhi/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/sanchit-gandhi">@sanchit-gandhi</a></li>
</ul>