  In this work, we study the hypothesis that speaker identity embeddings
extracted from speech samples may be used for detection and classification of
emotion. In particular, we show that emotions can be effectively identified by
learning speaker identities by use of a 1-D Triplet Convolutional Neural
Network (CNN) & Global Style Token (GST) scheme (e.g., DeepTalk Network) and
reusing the trained speaker recognition model weights to generate features in
the emotion classification domain. The automatic speaker recognition (ASR)
network is trained with VoxCeleb1, VoxCeleb2, and Librispeech datasets with a
triplet training loss function using speaker identity labels. Using an Support
Vector Machine (SVM) classifier, we map speaker identity embeddings into
discrete emotion categories from the CREMA-D, IEMOCAP, and MSP-Podcast
datasets. On the task of speech emotion detection, we obtain 80.8% ACC with
acted emotion samples from CREMA-D, 81.2% ACC with semi-natural emotion samples
in IEMOCAP, and 66.9% ACC with natural emotion samples in MSP-Podcast. We also
propose a novel two-stage hierarchical classifier (HC) approach which
demonstrates +2% ACC improvement on CREMA-D emotion samples. Through this work,
we seek to convey the importance of holistically modeling intra-user variation
within audio samples

Audio and Speech Processing;Sound