title: MulliVC: Multi-lingual Voice Conversion With Cycle Consistency
summary: Voice conversion aims to modify the source speaker's voice to resemble the
  target speaker while preserving the original speech content. Despite notable
  advancements in voice conversion these days, multi-lingual voice conversion
  (including both monolingual and cross-lingual scenarios) has yet to be
  extensively studied. It faces two main challenges: 1) the considerable
  variability in prosody and articulation habits across languages; and 2) the
  rarity of paired multi-lingual datasets from the same speaker. In this paper,
  we propose MulliVC, a novel voice conversion system that only converts timbre
  and keeps original content and source language prosody without multi-lingual
  paired data. Specifically, each training step of MulliVC contains three
  substeps: In step one the model is trained with monolingual speech data; then,
  steps two and three take inspiration from back translation, construct a
  cyclical process to disentangle the timbre and other information (content,
  prosody, and other language-related information) in the absence of
  multi-lingual data from the same speaker. Both objective and subjective results
  indicate that MulliVC significantly surpasses other methods in both monolingual
  and cross-lingual contexts, demonstrating the system's efficacy and the
  viability of the three-step approach with cycle consistency. Audio samples can
  be found on our demo page (mullivc.github.io).
paper_authors._id: 66b97e28a373e756edee2498
paper_authors.name: Jiawei Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-13T11:03:31.829Z
paper_authors._id: 63b2eaad677046a142862047
paper_authors.avatarUrl: /avatars/c6d6b06e7d815ab4214dc690ae4b401b.svg
paper_authors.fullname: Jiawei Huang
paper_authors.type: user
paper_authors.user: aweiqifei
paper_authors._id: 66b97e28a373e756edee2499
paper_authors.name: Chen Zhang
paper_authors._id: 66b97e28a373e756edee249a
paper_authors.name: Yi Ren
paper_authors._id: 66b97e28a373e756edee249b
paper_authors.name: Ziyue Jiang
paper_authors._id: 66b97e28a373e756edee249c
paper_authors.name: Zhenhui Ye
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-13T11:09:57.255Z
paper_authors._id: 6277fcd8c5c547c31b80cc77
paper_authors.avatarUrl: /avatars/7dab3f5c879b5c5eb86f822119a9b5bb.svg
paper_authors.fullname: Zhenhui Ye
paper_authors.type: user
paper_authors.user: yerfor
paper_authors._id: 66b97e28a373e756edee249d
paper_authors.name: Jinglin Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-13T11:10:06.161Z
paper_authors._id: 6205c6ad00e630c7a5d0fc2e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1645184905991-6205c6ad00e630c7a5d0fc2e.jpeg
paper_authors.fullname: Jinglin Liu
paper_authors.type: user
paper_authors.user: Silentlin
paper_authors._id: 66b97e28a373e756edee249e
paper_authors.name: Jinzheng He
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-13T11:10:12.247Z
paper_authors._id: 6594f06ac04427eb38444bce
paper_authors.avatarUrl: /avatars/b13fbf589b25eff038deb3fa12d95871.svg
paper_authors.fullname: Jinzheng He
paper_authors.type: user
paper_authors.user: jinzheng-he
paper_authors._id: 66b97e28a373e756edee249f
paper_authors.name: Xiang Yin
paper_authors._id: 66b97e28a373e756edee24a0
paper_authors.name: Zhou Zhao
paper_title: MulliVC: Multi-lingual Voice Conversion With Cycle Consistency
paper_summary: Voice conversion aims to modify the source speaker's voice to resemble the
  target speaker while preserving the original speech content. Despite notable
  advancements in voice conversion these days, multi-lingual voice conversion
  (including both monolingual and cross-lingual scenarios) has yet to be
  extensively studied. It faces two main challenges: 1) the considerable
  variability in prosody and articulation habits across languages; and 2) the
  rarity of paired multi-lingual datasets from the same speaker. In this paper,
  we propose MulliVC, a novel voice conversion system that only converts timbre
  and keeps original content and source language prosody without multi-lingual
  paired data. Specifically, each training step of MulliVC contains three
  substeps: In step one the model is trained with monolingual speech data; then,
  steps two and three take inspiration from back translation, construct a
  cyclical process to disentangle the timbre and other information (content,
  prosody, and other language-related information) in the absence of
  multi-lingual data from the same speaker. Both objective and subjective results
  indicate that MulliVC significantly surpasses other methods in both monolingual
  and cross-lingual contexts, demonstrating the system's efficacy and the
  viability of the three-step approach with cycle consistency. Audio samples can
  be found on our demo page (mullivc.github.io).
paper_ai_summary: MulliVC is a novel voice conversion system that separately converts timbre and preserves content and prosody in monolingual and cross-lingual scenarios without requiring multi-lingual paired data.
paper_ai_keywords: voice conversion timbre prosody articulation multi-lingual back translation cyclical process cycle consistency