Dominik Macháček
commited on
Commit
·
d497503
1
Parent(s):
6b1c2c5
COntributions at README.md
Browse files+ nicer formatting
+ #77
README.md
CHANGED
|
@@ -3,16 +3,12 @@ Whisper realtime streaming for long speech-to-text transcription and translation
|
|
| 3 |
|
| 4 |
**Turning Whisper into Real-Time Transcription System**
|
| 5 |
|
| 6 |
-
Demonstration paper, by Dominik Macháček, Raj Dabre, Ondřej Bojar, 2023
|
| 7 |
|
| 8 |
-
Abstract: Whisper is one of the recent state-of-the-art multilingual speech recognition and translation models, however, it is not designed for real
|
| 9 |
|
| 10 |
|
| 11 |
-
Paper PDF
|
| 12 |
-
https://aclanthology.org/2023.ijcnlp-demo.3.pdf
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
Demo video: https://player.vimeo.com/video/840442741
|
| 16 |
|
| 17 |
[Slides](http://ufallab.ms.mff.cuni.cz/~machacek/pre-prints/AACL23-2.11.2023-Turning-Whisper-oral.pdf) -- 15 minutes oral presentation at IJCNLP-AACL 2023
|
| 18 |
|
|
@@ -228,12 +224,20 @@ In more detail: we use the init prompt, we handle the inaccurate timestamps, we
|
|
| 228 |
re-process confirmed sentence prefixes and skip them, making sure they don't
|
| 229 |
overlap, and we limit the processing buffer window.
|
| 230 |
|
| 231 |
-
Contributions are welcome.
|
| 232 |
-
|
| 233 |
### Performance evaluation
|
| 234 |
|
| 235 |
[See the paper.](http://www.afnlp.org/conferences/ijcnlp2023/proceedings/main-demo/cdrom/pdf/2023.ijcnlp-demo.3.pdf)
|
| 236 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 237 |
|
| 238 |
## Contact
|
| 239 |
|
|
|
|
| 3 |
|
| 4 |
**Turning Whisper into Real-Time Transcription System**
|
| 5 |
|
| 6 |
+
Demonstration paper, by [Dominik Macháček](https://ufal.mff.cuni.cz/dominik-machacek), [Raj Dabre](https://prajdabre.github.io/), [Ondřej Bojar](https://ufal.mff.cuni.cz/ondrej-bojar), 2023
|
| 7 |
|
| 8 |
+
Abstract: Whisper is one of the recent state-of-the-art multilingual speech recognition and translation models, however, it is not designed for real-time transcription. In this paper, we build on top of Whisper and create Whisper-Streaming, an implementation of real-time speech transcription and translation of Whisper-like models. Whisper-Streaming uses local agreement policy with self-adaptive latency to enable streaming transcription. We show that Whisper-Streaming achieves high quality and 3.3 seconds latency on unsegmented long-form speech transcription test set, and we demonstrate its robustness and practical usability as a component in live transcription service at a multilingual conference.
|
| 9 |
|
| 10 |
|
| 11 |
+
[Paper PDF](https://aclanthology.org/2023.ijcnlp-demo.3.pdf), [Demo video](https://player.vimeo.com/video/840442741)
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
|
| 13 |
[Slides](http://ufallab.ms.mff.cuni.cz/~machacek/pre-prints/AACL23-2.11.2023-Turning-Whisper-oral.pdf) -- 15 minutes oral presentation at IJCNLP-AACL 2023
|
| 14 |
|
|
|
|
| 224 |
re-process confirmed sentence prefixes and skip them, making sure they don't
|
| 225 |
overlap, and we limit the processing buffer window.
|
| 226 |
|
|
|
|
|
|
|
| 227 |
### Performance evaluation
|
| 228 |
|
| 229 |
[See the paper.](http://www.afnlp.org/conferences/ijcnlp2023/proceedings/main-demo/cdrom/pdf/2023.ijcnlp-demo.3.pdf)
|
| 230 |
|
| 231 |
+
### Contributions
|
| 232 |
+
|
| 233 |
+
Contributions are welcome. We acknowledge especially:
|
| 234 |
+
|
| 235 |
+
- [The GitHub contributors](https://github.com/ufal/whisper_streaming/graphs/contributors) for their pull requests with new features and bugfixes.
|
| 236 |
+
- [The translation of this repo into Chinese.](https://github.com/Gloridust/whisper_streaming_CN)
|
| 237 |
+
- [Ondřej Plátek](https://opla.cz/) for the paper pre-review.
|
| 238 |
+
- [Peter Polák](https://ufal.mff.cuni.cz/peter-polak) for the original idea.
|
| 239 |
+
- The UEDIN team of the [ELITR project](https://elitr.eu) for the original line_packet.py.
|
| 240 |
+
|
| 241 |
|
| 242 |
## Contact
|
| 243 |
|