  This paper gives an overview of the assessment and evaluation methods which
have been used to determine the quality of the INSPIRE smart home system. The
system allows different home appliances to be controlled via speech, and
consists of speech and speaker recognition, speech understanding, dialogue
management, and speech output components. The performance of these components
is first assessed individually, and then the entire system is evaluated in an
interaction experiment with test users. Initial results of the assessment and
evaluation are given, in particular with respect to the transmission channel
impact on speech and speaker recognition, and the assessment of speech output
for different system metaphors.

Human-Computer Interaction;Computation and Language