Vox-Profile: A Speech Foundation Model Benchmark for Characterizing
  Diverse Speaker and Speech Traits
We introduce Vox-Profile, a comprehensive benchmark to characterize rich
speaker and speech traits using speech foundation models. Unlike existing works
that focus on a single dimension of speaker traits, Vox-Profile provides
holistic and multi-dimensional profiles that reflect both static speaker traits
(e.g., age, sex, accent) and dynamic speech properties (e.g., emotion, speech
flow). This benchmark is grounded in speech science and linguistics, developed
with domain experts to accurately index speaker and speech characteristics. We
report benchmark experiments using over 15 publicly available speech datasets
and several widely used speech foundation models that target various static and
dynamic speaker and speech properties. In addition to benchmark experiments, we
showcase several downstream applications supported by Vox-Profile. First, we
show that Vox-Profile can augment existing speech recognition datasets to
analyze ASR performance variability. Vox-Profile is also used as a tool to
evaluate the performance of speech generation systems. Finally, we assess the
quality of our automated profiles through comparison with human evaluation and
show convergent validity. Vox-Profile is publicly available at:
https://github.com/tiantiaf0627/vox-profile-release.
682e0839d8a198bf9e184ef2
Tiantian Feng
extracted_confirmed
2025-05-21T17:07:37.217Z
64092a1ab6a334f53e278b3b
https://cdn-avatars.huggingface.co/v1/production/uploads/64092a1ab6a334f53e278b3b/tcueLWyyDL6WMUTw3Or4t.jpeg
Tiantian Feng
user
tiantiaf
682e0839d8a198bf9e184ef3
Jihwan Lee
682e0839d8a198bf9e184ef4
Anfeng Xu
682e0839d8a198bf9e184ef5
Yoonjeong Lee
682e0839d8a198bf9e184ef6
Thanathai Lertpetchpun
682e0839d8a198bf9e184ef7
Xuan Shi
682e0839d8a198bf9e184ef8
Helin Wang
claimed_verified
2025-05-28T12:11:21.909Z
63ecfb5ec5b3c734085db9ed
/avatars/0b1d03dcd7997ad1daa764fb76f88993.svg
Helin Wang
user
westbrook
682e0839d8a198bf9e184ef9
Thomas Thebaud
682e0839d8a198bf9e184efa
Laureano Moro-Velazquez
682e0839d8a198bf9e184efb
Dani Byrd
682e0839d8a198bf9e184efc
Najim Dehak
682e0839d8a198bf9e184efd
Shrikanth Narayanan
Vox-Profile: A Speech Foundation Model Benchmark for Characterizing
  Diverse Speaker and Speech Traits
We introduce Vox-Profile, a comprehensive benchmark to characterize rich
speaker and speech traits using speech foundation models. Unlike existing works
that focus on a single dimension of speaker traits, Vox-Profile provides
holistic and multi-dimensional profiles that reflect both static speaker traits
(e.g., age, sex, accent) and dynamic speech properties (e.g., emotion, speech
flow). This benchmark is grounded in speech science and linguistics, developed
with domain experts to accurately index speaker and speech characteristics. We
report benchmark experiments using over 15 publicly available speech datasets
and several widely used speech foundation models that target various static and
dynamic speaker and speech properties. In addition to benchmark experiments, we
showcase several downstream applications supported by Vox-Profile. First, we
show that Vox-Profile can augment existing speech recognition datasets to
analyze ASR performance variability. Vox-Profile is also used as a tool to
evaluate the performance of speech generation systems. Finally, we assess the
quality of our automated profiles through comparison with human evaluation and
show convergent validity. Vox-Profile is publicly available at:
https://github.com/tiantiaf0627/vox-profile-release.
Vox-Profile is a benchmark for evaluating multi-dimensional speaker and speech traits using speech foundation models, offering applications in ASR and speech generation performance analysis.
speech foundation models speaker traits speech properties emotion speech flow speaker characteristics ASR performance variability speech generation systems human evaluation convergent validity