README.md

Whisper Streaming with FastAPI and WebSocket Integration

This project extends the Whisper Streaming implementation by incorporating few extras. The enhancements include:

1. FastAPI Server with WebSocket Endpoint: Enables real-time speech-to-text transcription directly from the browser.

2. Buffering Indication: Improves streaming display by showing the current processing status, providing users with immediate feedback.

3. Javascript Client implementation: Functionnal and minimalist MediaRecorder implementation that can be copied on your client side

4. MLX Whisper backend: Integrates the alternative backend option MLX Whisper, optimized for efficient speech recognition on Apple silicon.

Installation

1. Clone the Repository:

   git clone https://github.com/QuentinFuxa/whisper_streaming_web
   cd whisper_streaming_web
   

How to Launch the Server

1. Install Dependencies:

   pip install -r requirements.txt
   

2. Install a whisper backend among:

whisper
whisper-timestamped
faster-whisper (faster backend on NVIDIA GPU)
mlx-whisper (faster backend on Apple Silicon)

and torch if you want to use VAC (Voice Activity Controller)

3. Run the FastAPI Server:

   python whisper_fastapi_online_server.py --host 0.0.0.0 --port 8000
   

   • --host and --port let you specify the server’s IP/port.

4. Open the Provided HTML:

   • By default, the server root endpoint / serves a simple live_transcription.html page.
   • Open your browser at http://localhost:8000 (or replace localhost and 8000 with whatever you specified).
   • The page uses vanilla JavaScript and the WebSocket API to capture your microphone and stream audio to the server in real time.

How the Live Interface Works

• Once you allow microphone access, the page records small chunks of audio using the MediaRecorder API in webm/opus format.
• These chunks are sent over a WebSocket to the FastAPI endpoint at /ws.
• The Python server decodes .webm chunks on the fly using FFmpeg and streams them into the whisper streaming implementation for transcription.
• Partial transcription appears as soon as enough audio is processed. The “unvalidated” text is shown in lighter or grey color (i.e., an ‘aperçu’) to indicate it’s still buffered partial output. Once Whisper finalizes that segment, it’s displayed in normal text.
• You can watch the transcription update in near real time, ideal for demos, prototyping, or quick debugging.

Deploying to a Remote Server

If you want to deploy this setup:

1. Host the FastAPI app behind a production-grade HTTP(S) server (like Uvicorn + Nginx or Docker).
2. The HTML/JS page can be served by the same FastAPI app or a separate static host.
3. Users open the page in Chrome/Firefox (any modern browser that supports MediaRecorder + WebSocket).

No additional front-end libraries or frameworks are required. The WebSocket logic in live_transcription.html is minimal enough to adapt for your own custom UI or embed in other pages.

Acknowledgments

This project builds upon the foundational work of the Whisper Streaming project. We extend our gratitude to the original authors for their contributions.