Furthermore , a number of performance features , largely based on the PARADISE dialogue evaluation scheme ( Walker et al. , 1997 ) , were automatically logged , derived , or manually annotated .
paradise a framework for evaluating spoken dialogue agents
Marilyn A Walker Diane J Litman Candance A Kamm Alicia Abella
User satisfaction was assessed by asking the subjects to fill out a questionnaire after interacting with each version of the system . Furthermore , a number of performance features , largely based on the PARADISE dialogue evaluation scheme ( Walker et al. , 1997 ) , were automatically logged , derived , or manually annotated . In addition , we logged the cues automatically detected in each user utterance , as well as the initiative distribution for each turn and the dialogue acts selected to generate each system response .
Furthermore , a number of performance features , largely based on the PARADISE dialogue evaluation scheme ( @@CITATION ) , were automatically logged , derived , or manually annotated .