Spaces:
Running
on
CPU Upgrade
Prompting type of sentences
From what I see there are no exclamation or question marks in any of the harvard sentences.
Suggest adding those along with prompting type of sentences such as:
- He drew closer and quietly whispered in her ear: "I know what you did last summer."
- "Are you sure about that?" he said, confused.
- "Don’t test me!" he shouted angrily.
A side effect of narrator based datasets and a way to affect the emotional state.
https://elevenlabs.io/docs/speech-synthesis/prompting#emotion
IMHO a very powerful and underused feature by ElevenLabs, as there is still no way to remove the prompt.
Without this and without question and exclamation marks this evaluation test very basic TTS capabilities. 😕
Brilliant feedback, I think this requires a bit more thought on the best way to make this fit with other models.
Primarily because not all models support this kind of prompting and it’s not fair to them.
Creating evals is a hard problem - making the evals fair for all is an even harder one.
Hence, I recommend rolling back the change to the Harvard sentences list for now.
(It’s getting late for me here in CET timezone, I can respond with more thought tomorrow in my morning - till then it’d be great if we can rollback this change)
Indeed, so far when reviewing TTS Spaces, TorToiSe seems to be the only one impacted by the prompting that is described in aforementioned ElevenLabs Docs.
But IMHO TTS handling of questions and maybe even exclamation marks should be part of the evaluation.