Optimizing Multilingual Text-To-Speech with Accents & Emotions
State-of-the-art text-to-speech (TTS) systems realize high naturalness in
monolingual environments, synthesizing speech with correct multilingual accents
(especially for Indic languages) and context-relevant emotions still poses
difficulty owing to cultural nuance discrepancies in current frameworks. This
paper introduces a new TTS architecture integrating accent along with
preserving transliteration with multi-scale emotion modelling, in particularly
tuned for Hindi and Indian English accent. Our approach extends the Parler-TTS
model by integrating A language-specific phoneme alignment hybrid
encoder-decoder architecture, and culture-sensitive emotion embedding layers
trained on native speaker corpora, as well as incorporating a dynamic accent
code switching with residual vector quantization. Quantitative tests
demonstrate 23.7% improvement in accent accuracy (Word Error Rate reduction
from 15.4% to 11.8%) and 85.3% emotion recognition accuracy from native
listeners, surpassing METTS and VECL-TTS baselines. The novelty of the system
is that it can mix code in real time - generating statements such as "Namaste,
let's talk about <Hindi phrase>" with uninterrupted accent shifts while
preserving emotional consistency. Subjective evaluation with 200 users reported
a mean opinion score (MOS) of 4.2/5 for cultural correctness, much better than
existing multilingual systems (p<0.01). This research makes cross-lingual
synthesis more feasible by showcasing scalable accent-emotion disentanglement,
with direct application in South Asian EdTech and accessibility software.
6858c06ac0c8e29df8ea3c5c
Pranav Pawar
claimed_verified
2025-06-23T08:14:52.282Z
65d02414af9670557423df10
https://cdn-avatars.huggingface.co/v1/production/uploads/65d02414af9670557423df10/5SNGUGEEdkliGKNceo_gS.jpeg
Pranav Pawar
user
prnvpwr2612
6858c06ac0c8e29df8ea3c5d
Akshansh Dwivedi
claimed_verified
2025-06-24T08:10:03.090Z
64f2fdccf1b6c235aee14c3d
/avatars/a98156dd25c980cf2fd6c068aef16b6a.svg
Akshansh Dwivedi
user
En1gma02
6858c06ac0c8e29df8ea3c5e
Jenish Boricha
6858c06ac0c8e29df8ea3c5f
Himanshu Gohil
6858c06ac0c8e29df8ea3c60
Aditya Dubey
Optimizing Multilingual Text-To-Speech with Accents & Emotions
State-of-the-art text-to-speech (TTS) systems realize high naturalness in
monolingual environments, synthesizing speech with correct multilingual accents
(especially for Indic languages) and context-relevant emotions still poses
difficulty owing to cultural nuance discrepancies in current frameworks. This
paper introduces a new TTS architecture integrating accent along with
preserving transliteration with multi-scale emotion modelling, in particularly
tuned for Hindi and Indian English accent. Our approach extends the Parler-TTS
model by integrating A language-specific phoneme alignment hybrid
encoder-decoder architecture, and culture-sensitive emotion embedding layers
trained on native speaker corpora, as well as incorporating a dynamic accent
code switching with residual vector quantization. Quantitative tests
demonstrate 23.7% improvement in accent accuracy (Word Error Rate reduction
from 15.4% to 11.8%) and 85.3% emotion recognition accuracy from native
listeners, surpassing METTS and VECL-TTS baselines. The novelty of the system
is that it can mix code in real time - generating statements such as "Namaste,
let's talk about <Hindi phrase>" with uninterrupted accent shifts while
preserving emotional consistency. Subjective evaluation with 200 users reported
a mean opinion score (MOS) of 4.2/5 for cultural correctness, much better than
existing multilingual systems (p<0.01). This research makes cross-lingual
synthesis more feasible by showcasing scalable accent-emotion disentanglement,
with direct application in South Asian EdTech and accessibility software.
A new TTS architecture improves accent accuracy and emotion recognition for Hindi and Indian English by integrating phoneme alignment, culture-sensitive emotion embeddings, and dynamic accent code switching.
text-to-speech Parler-TTS phoneme alignment hybrid encoder-decoder emotion embedding residual vector quantization accent accuracy emotion recognition accent code switching cross-lingual synthesis accent-emotion disentanglement