title: EmoKnob: Enhance Voice Cloning with Fine-Grained Emotion Control
summary: While recent advances in Text-to-Speech (TTS) technology produce natural and
  expressive speech, they lack the option for users to select emotion and control
  intensity. We propose EmoKnob, a framework that allows fine-grained emotion
  control in speech synthesis with few-shot demonstrative samples of arbitrary
  emotion. Our framework leverages the expressive speaker representation space
  made possible by recent advances in foundation voice cloning models. Based on
  the few-shot capability of our emotion control framework, we propose two
  methods to apply emotion control on emotions described by open-ended text,
  enabling an intuitive interface for controlling a diverse array of nuanced
  emotions. To facilitate a more systematic emotional speech synthesis field, we
  introduce a set of evaluation metrics designed to rigorously assess the
  faithfulness and recognizability of emotion control frameworks. Through
  objective and subjective evaluations, we show that our emotion control
  framework effectively embeds emotions into speech and surpasses emotion
  expressiveness of commercial TTS services.
paper_authors._id: 66fed130001816b29e7429ff
paper_authors.name: Haozhe Chen
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-10-03T17:31:25.648Z
paper_authors._id: 6329cbaf74bc1990129c04e0
paper_authors.avatarUrl: /avatars/8977d458af6aa2fa84d722e3cb7aeba1.svg
paper_authors.fullname: Tony Chen
paper_authors.type: user
paper_authors.user: tonychenxyz
paper_authors._id: 66fed130001816b29e742a00
paper_authors.name: Run Chen
paper_authors._id: 66fed130001816b29e742a01
paper_authors.name: Julia Hirschberg
paper_title: EmoKnob: Enhance Voice Cloning with Fine-Grained Emotion Control
paper_summary: While recent advances in Text-to-Speech (TTS) technology produce natural and
  expressive speech, they lack the option for users to select emotion and control
  intensity. We propose EmoKnob, a framework that allows fine-grained emotion
  control in speech synthesis with few-shot demonstrative samples of arbitrary
  emotion. Our framework leverages the expressive speaker representation space
  made possible by recent advances in foundation voice cloning models. Based on
  the few-shot capability of our emotion control framework, we propose two
  methods to apply emotion control on emotions described by open-ended text,
  enabling an intuitive interface for controlling a diverse array of nuanced
  emotions. To facilitate a more systematic emotional speech synthesis field, we
  introduce a set of evaluation metrics designed to rigorously assess the
  faithfulness and recognizability of emotion control frameworks. Through
  objective and subjective evaluations, we show that our emotion control
  framework effectively embeds emotions into speech and surpasses emotion
  expressiveness of commercial TTS services.
paper_ai_summary: A framework called EmoKnob enables fine-grained emotion control in speech synthesis using few-shot samples, surpassing commercial TTS services in emotion expressiveness.
paper_ai_keywords: Text-to-Speech (TTS) expressive speaker representation foundation voice cloning models emotion control framework intuitive interface evaluation metrics faithfulness recognizability