title: Vox-Profile: A Speech Foundation Model Benchmark for Characterizing
    Diverse Speaker and Speech Traits
summary: We introduce Vox-Profile, a comprehensive benchmark to characterize rich
  speaker and speech traits using speech foundation models. Unlike existing works
  that focus on a single dimension of speaker traits, Vox-Profile provides
  holistic and multi-dimensional profiles that reflect both static speaker traits
  (e.g., age, sex, accent) and dynamic speech properties (e.g., emotion, speech
  flow). This benchmark is grounded in speech science and linguistics, developed
  with domain experts to accurately index speaker and speech characteristics. We
  report benchmark experiments using over 15 publicly available speech datasets
  and several widely used speech foundation models that target various static and
  dynamic speaker and speech properties. In addition to benchmark experiments, we
  showcase several downstream applications supported by Vox-Profile. First, we
  show that Vox-Profile can augment existing speech recognition datasets to
  analyze ASR performance variability. Vox-Profile is also used as a tool to
  evaluate the performance of speech generation systems. Finally, we assess the
  quality of our automated profiles through comparison with human evaluation and
  show convergent validity. Vox-Profile is publicly available at:
  https://github.com/tiantiaf0627/vox-profile-release.
paper_authors._id: 682e0839d8a198bf9e184ef2
paper_authors.name: Tiantian Feng
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-05-21T17:07:37.217Z
paper_authors._id: 64092a1ab6a334f53e278b3b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64092a1ab6a334f53e278b3b/tcueLWyyDL6WMUTw3Or4t.jpeg
paper_authors.fullname: Tiantian Feng
paper_authors.type: user
paper_authors.user: tiantiaf
paper_authors._id: 682e0839d8a198bf9e184ef3
paper_authors.name: Jihwan Lee
paper_authors._id: 682e0839d8a198bf9e184ef4
paper_authors.name: Anfeng Xu
paper_authors._id: 682e0839d8a198bf9e184ef5
paper_authors.name: Yoonjeong Lee
paper_authors._id: 682e0839d8a198bf9e184ef6
paper_authors.name: Thanathai Lertpetchpun
paper_authors._id: 682e0839d8a198bf9e184ef7
paper_authors.name: Xuan Shi
paper_authors._id: 682e0839d8a198bf9e184ef8
paper_authors.name: Helin Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T12:11:21.909Z
paper_authors._id: 63ecfb5ec5b3c734085db9ed
paper_authors.avatarUrl: /avatars/0b1d03dcd7997ad1daa764fb76f88993.svg
paper_authors.fullname: Helin Wang
paper_authors.type: user
paper_authors.user: westbrook
paper_authors._id: 682e0839d8a198bf9e184ef9
paper_authors.name: Thomas Thebaud
paper_authors._id: 682e0839d8a198bf9e184efa
paper_authors.name: Laureano Moro-Velazquez
paper_authors._id: 682e0839d8a198bf9e184efb
paper_authors.name: Dani Byrd
paper_authors._id: 682e0839d8a198bf9e184efc
paper_authors.name: Najim Dehak
paper_authors._id: 682e0839d8a198bf9e184efd
paper_authors.name: Shrikanth Narayanan
paper_title: Vox-Profile: A Speech Foundation Model Benchmark for Characterizing
    Diverse Speaker and Speech Traits
paper_summary: We introduce Vox-Profile, a comprehensive benchmark to characterize rich
  speaker and speech traits using speech foundation models. Unlike existing works
  that focus on a single dimension of speaker traits, Vox-Profile provides
  holistic and multi-dimensional profiles that reflect both static speaker traits
  (e.g., age, sex, accent) and dynamic speech properties (e.g., emotion, speech
  flow). This benchmark is grounded in speech science and linguistics, developed
  with domain experts to accurately index speaker and speech characteristics. We
  report benchmark experiments using over 15 publicly available speech datasets
  and several widely used speech foundation models that target various static and
  dynamic speaker and speech properties. In addition to benchmark experiments, we
  showcase several downstream applications supported by Vox-Profile. First, we
  show that Vox-Profile can augment existing speech recognition datasets to
  analyze ASR performance variability. Vox-Profile is also used as a tool to
  evaluate the performance of speech generation systems. Finally, we assess the
  quality of our automated profiles through comparison with human evaluation and
  show convergent validity. Vox-Profile is publicly available at:
  https://github.com/tiantiaf0627/vox-profile-release.
paper_ai_summary: Vox-Profile is a benchmark for evaluating multi-dimensional speaker and speech traits using speech foundation models, offering applications in ASR and speech generation performance analysis.
paper_ai_keywords: speech foundation models speaker traits speech properties emotion speech flow speaker characteristics ASR performance variability speech generation systems human evaluation convergent validity