title: Adapting General Disentanglement-Based Speaker Anonymization for
    Enhanced Emotion Preservation
summary: A general disentanglement-based speaker anonymization system typically
  separates speech into content, speaker, and prosody features using individual
  encoders. This paper explores how to adapt such a system when a new speech
  attribute, for example, emotion, needs to be preserved to a greater extent.
  While existing systems are good at anonymizing speaker embeddings, they are not
  designed to preserve emotion. Two strategies for this are examined. First, we
  show that integrating emotion embeddings from a pre-trained emotion encoder can
  help preserve emotional cues, even though this approach slightly compromises
  privacy protection. Alternatively, we propose an emotion compensation strategy
  as a post-processing step applied to anonymized speaker embeddings. This
  conceals the original speaker's identity and reintroduces the emotional traits
  lost during speaker embedding anonymization. Specifically, we model the emotion
  attribute using support vector machines to learn separate boundaries for each
  emotion. During inference, the original speaker embedding is processed in two
  ways: one, by an emotion indicator to predict emotion and select the
  emotion-matched SVM accurately; and two, by a speaker anonymizer to conceal
  speaker characteristics. The anonymized speaker embedding is then modified
  along the corresponding SVM boundary towards an enhanced emotional direction to
  save the emotional cues. The proposed strategies are also expected to be useful
  for adapting a general disentanglement-based speaker anonymization system to
  preserve other target paralinguistic attributes, with potential for a range of
  downstream tasks.
paper_authors._id: 66bc88c7780d735f1754c00b
paper_authors.name: Xiaoxiao Miao
paper_authors._id: 66bc88c7780d735f1754c00c
paper_authors.name: Yuxiang Zhang
paper_authors._id: 66bc88c7780d735f1754c00d
paper_authors.name: Xin Wang
paper_authors._id: 66bc88c7780d735f1754c00e
paper_authors.name: Natalia Tomashenko
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-14T11:11:14.807Z
paper_authors._id: 619d132b0facfda0c4ebc4e2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1637684705612-619d132b0facfda0c4ebc4e2.jpeg
paper_authors.fullname: Natalia Tomashenko
paper_authors.type: user
paper_authors.user: Natalia-T
paper_authors._id: 66bc88c7780d735f1754c00f
paper_authors.name: Donny Cheng Lock Soh
paper_authors._id: 66bc88c7780d735f1754c010
paper_authors.name: Ian Mcloughlin
paper_title: Adapting General Disentanglement-Based Speaker Anonymization for
    Enhanced Emotion Preservation
paper_summary: A general disentanglement-based speaker anonymization system typically
  separates speech into content, speaker, and prosody features using individual
  encoders. This paper explores how to adapt such a system when a new speech
  attribute, for example, emotion, needs to be preserved to a greater extent.
  While existing systems are good at anonymizing speaker embeddings, they are not
  designed to preserve emotion. Two strategies for this are examined. First, we
  show that integrating emotion embeddings from a pre-trained emotion encoder can
  help preserve emotional cues, even though this approach slightly compromises
  privacy protection. Alternatively, we propose an emotion compensation strategy
  as a post-processing step applied to anonymized speaker embeddings. This
  conceals the original speaker's identity and reintroduces the emotional traits
  lost during speaker embedding anonymization. Specifically, we model the emotion
  attribute using support vector machines to learn separate boundaries for each
  emotion. During inference, the original speaker embedding is processed in two
  ways: one, by an emotion indicator to predict emotion and select the
  emotion-matched SVM accurately; and two, by a speaker anonymizer to conceal
  speaker characteristics. The anonymized speaker embedding is then modified
  along the corresponding SVM boundary towards an enhanced emotional direction to
  save the emotional cues. The proposed strategies are also expected to be useful
  for adapting a general disentanglement-based speaker anonymization system to
  preserve other target paralinguistic attributes, with potential for a range of
  downstream tasks.
paper_ai_summary: A speaker anonymization system is adapted to preserve emotional cues while maintaining speaker privacy using integrated emotion embeddings or a post-processing emotion compensation strategy.
paper_ai_keywords: disentanglement-based speaker anonymization speaker embeddings emotion embeddings pre-trained emotion encoder support vector machines SVM emotion indicator