  Dialogue enhancement (DE) plays a vital role in broadcasting, enabling the
personalization of the relative level between foreground speech and background
music and effects. DE has been shown to improve the quality of experience,
intelligibility, and self-reported listening effort (LE). A physiological
indicator of LE known from audiology studies is pupil size. The relation
between pupil size and LE is typically studied using artificial sentences and
background noises not encountered in broadcast content. This work evaluates the
effect of DE on LE in a multimodal manner that includes pupil size (tracked by
a VR headset) and real-world audio excerpts from TV. Under ideal listening
conditions, 28 normal-hearing participants listened to 30 audio excerpts
presented in random order and processed by conditions varying the relative
level between foreground and background audio. One of these conditions employed
a recently proposed source separation system to attenuate the background given
the original mixture as the sole input. After listening to each excerpt,
subjects were asked to repeat the heard sentence and self-report the LE. Mean
pupil dilation and peak pupil dilation were analyzed and compared with the
self-report and the word recall rate. The multimodal evaluation shows a
consistent trend of decreasing LE along with decreasing background level. DE,
also when enabled by source separation, significantly reduces the pupil size as
well as the self-reported LE. This highlights the benefit of personalization
functionalities at the user's end.

Audio and Speech Processing