Answers.Body: <p>Curious about what you're building. I needed timestamps to create subtitles and followed this tutorial.
  <a href="https://picovoice.ai/blog/how-to-create-subtitles-for-any-video-with-python/" rel="nofollow noreferrer">https://picovoice.ai/blog/how-to-create-subtitles-for-any-video-with-python/</a>
  <a href="https://picovoice.ai/docs/leopard/#add-custom-vocabulary" rel="nofollow noreferrer">https://picovoice.ai/docs/leopard/#add-custom-vocabulary</a></p>
  <p>I was happy with the results, because I could add new words and boost them pretty quickly. I couldn't figure out how to customize VOSK though but it could be me. lol</p>
  <p>They were gonna add timestamp and diarization features to whisper, but i dont think they did, worth exploring though <a href="https://deepgram.com/openai-whisper" rel="nofollow noreferrer">https://deepgram.com/openai-whisper</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-09-07T16:50:08.717Z
Answers.LastActivityDate: 2023-09-07T16:50:08.717Z
Answers.UserDisplayName: mistykleinman
Answers.Body: <p>Checkout <a href="https://github.com/jianfch/stable-ts" rel="nofollow noreferrer">stable-ts</a> (Transcription, forced alignment, and audio indexing with OpenAI's Whisper)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-01T13:29:35.077Z
Answers.LastActivityDate: 2024-12-01T13:29:35.077Z
Answers.UserDisplayName: selvan
Body: <p>I need to transcript some speech files. I've played around with a number of tools and I found that:</p>
  <ul>
  <li><p><a href="https://github.com/openai/whisper" rel="nofollow noreferrer">OpenAI Whisper</a> transcribes speech with absolute accuracy. It's even more accurate than me.</p>
  <ul>
  <li>However the timestamps it gives are very imprecise: sometimes they are several seconds off.</li>
  </ul>
  </li>
  <li><p><a href="https://alphacephei.com/vosk/" rel="nofollow noreferrer">VOSK</a> has a fantastic support for timestamping words: if you crop the audio file at the start and end of any word detected by VOSK, you hear that word precisely.</p>
  <ul>
  <li>However it's far less accurate in transcription: it often misunderstand speech.</li>
  </ul>
  </li>
  </ul>
  <p>Is there any way to achieve high accuracy in both tasks?</p>
  <p>I have tried a few tools that aim at improving Whisper's timestamp accuracy. Namely <a href="https://github.com/m-bain/whisperX" rel="nofollow noreferrer">WhisperX</a> and <a href="https://github.com/linto-ai/whisper-timestamped" rel="nofollow noreferrer">whisper-timestamped</a>, but neither of them is remotely close to VOSK.</p>
  <hr />
  <p>I'm considering trying to combine Whisper and VOSK: maybe I could transcript some speech using both tools, then map each word detected by VOSK on those detected by Whisper, so that I can use VOSK's timestamps for Whisper's words.</p>
  <p>However this task seems quite tricky, since many words are going to differ. I wonder whether anyone has attempted this or found better solutions.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-09-07T00:32:47.470Z
Comments.Text: how can vosk be accurate for timestamps but bad at predicting words?
Comments.UserDisplayName: Alberto
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-09-07T00:34:23.607Z
Comments.Text: you can take the timestamps of Vosk and the words from Whisper, and should be good to go no? like "vosk tells you the 4th word starts at 4.1s and ends at 5.3s and whisper tells you that the 4th word is blablabla"
Comments.UserDisplayName: Alberto
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-09-07T08:02:54.960Z
Comments.Text: @Alberto: try Vosk and you'll see. There's no obvious mapping between the words generated by the two tools. Some words are missing, others are extra, whole phrases differ.
Comments.UserDisplayName: Blue Nebula
Tags: <speech-recognition>
Title: Is it possible to achieve the transcript accuracy of Whisper with the timestamp accuracy of Vosk in speech-to-text tasks?
UserDisplayName: Blue Nebula
ThreadText: # Is it possible to achieve the transcript accuracy of Whisper with the timestamp accuracy of Vosk in speech-to-text tasks?
  
  (Asked by: username_0 on 2023-09-06)
  
  I need to transcript some speech files. I've played around with a number of tools and I found that:
  
  * [OpenAI Whisper](https://github.com/openai/whisper) transcribes speech with absolute accuracy. It's even more accurate than me.
  
    + However the timestamps it gives are very imprecise: sometimes they are several seconds off.
  * [VOSK](https://alphacephei.com/vosk/) has a fantastic support for timestamping words: if you crop the audio file at the start and end of any word detected by VOSK, you hear that word precisely.
  
    + However it's far less accurate in transcription: it often misunderstand speech.
  
  Is there any way to achieve high accuracy in both tasks?
  
  I have tried a few tools that aim at improving Whisper's timestamp accuracy. Namely [WhisperX](https://github.com/m-bain/whisperX) and [whisper-timestamped](https://github.com/linto-ai/whisper-timestamped), but neither of them is remotely close to VOSK.
  
  ---
  
  I'm considering trying to combine Whisper and VOSK: maybe I could transcript some speech using both tools, then map each word detected by VOSK on those detected by Whisper, so that I can use VOSK's timestamps for Whisper's words.
  
  However this task seems quite tricky, since many words are going to differ. I wonder whether anyone has attempted this or found better solutions.
  
  **Tags:** speech-recognition
  > **Comments:**
  > **username_1**: how can vosk be accurate for timestamps but bad at predicting words?  
  > **username_1**: you can take the timestamps of Vosk and the words from Whisper, and should be good to go no? like "vosk tells you the 4th word starts at 4.1s and ends at 5.3s and whisper tells you that the 4th word is blablabla"  
  > **username_0**: @Alberto: try Vosk and you'll see. There's no obvious mapping between the words generated by the two tools. Some words are missing, others are extra, whole phrases differ.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2023-09-07. Score: 0
  
  Curious about what you're building. I needed timestamps to create subtitles and followed this tutorial.
  <https://picovoice.ai/blog/how-to-create-subtitles-for-any-video-with-python/>
  <https://picovoice.ai/docs/leopard/#add-custom-vocabulary>
  
  I was happy with the results, because I could add new words and boost them pretty quickly. I couldn't figure out how to customize VOSK though but it could be me. lol
  
  They were gonna add timestamp and diarization features to whisper, but i dont think they did, worth exploring though <https://deepgram.com/openai-whisper>
  
  
  ---
  
  ### Answer by username_3 on 2024-12-01. Score: 0
  
  Checkout [stable-ts](https://github.com/jianfch/stable-ts) (Transcription, forced alignment, and audio indexing with OpenAI's Whisper)
  
  
  ---
  
  