Past research has demonstrated that there are cognitive processing costs associated with comprehension of speech generated by text-to-speech synthesizers, relative to comprehension of natural speech. This finding has important performance implications for the many applications that use such systems. The purpose of this study was to ascertain whether certain characteristics of synthetic speech slow on-line, real-time cognitive processing. Whereas past research has focused on the phonemic acoustic structure of synthetic speech, we manipulated prosodic, syntactic, and semantic cues in a task requiring participants to recall sentences spoken either by a human or by one of two speech synthesizers. The findings were interpreted to suggest that inappropriate prosodic modeling in synthetic speech was the major source of a performance differential between natural and synthetic speech. Prosodic cues, along with others, guide the parsing of speech and provide redundancy. When these cues are absent or inaccurate, the add itional burden placed on working memory may exceed its capacity, particularly in time-limited, demanding tasks. Actual or potential applications of this research include improvement of text-to-speech output systems in warning systems, feedback devices in aerospace vehicles, educational and training modules, aids for the handicapped, consumer products, and technologies designed to increase the functional independence of older adults. INTRODUCTION The field of machine speech generation has witnessed phenomenal growth over the past 30 years. Today speech output systems are critical components in military and industrial warning systems, feedback devices in aerospace vehicles, education and training modules, aids for the handicapped, consumer products, and technologies designed to increase the functional independence of older adults. As a consequence it is important to investigate whether task performance differs when computer speech is substituted for natural speech. This is a particularly critical question in the case of speech generated by text-to-speech (TTS) synthesizers. TTS systems produce words by combining stored representations of phonemes and other small units of speech according to a set of rules. Research has demonstrated that compared with natural voice, speech produced by TTS synthesizers places an increased burden on perceptual and cognitive resources during the comprehension process. Performance deficits have been discovered at many stages of processing, from phonemes to paragraphs, and have been detected on a variety of measures (see Duffy & Pisoni, 1992, for a review). One of the most basic yardsticks for comparison between natural and synthetic speech is single-word intelligibility. A typical procedure (Modified Rhyme Test) is to ask listeners to identify the word they heard from among a set of words differing by only one phonetic feature (House, Williams, Hecker, & Kryter, 1965). On this measure some TTS systems compare favorably with natural speech, with error rates only about 2% higher than natural speech (e.g., Logan, Greene, & Pisoni, 1989). However, measures of listeners' processing speed consistently reveal substantial differences between natural and synthetic voice input. That is, people take longer to comprehend computer-generated speech, even when single-word intelligibility is essentially equivalent. For example, the latency threshold for recognition of single words is higher for synthetically produced stimuli than for words produced by a human speaker (Manous & Pisoni, 1984). Also, data from sentence verification tasks reflect slower reaction times to judge th e truth of sentences spoken by a speech synthesizer than for natural speech sentences (Pisoni, Manous, & Dedina, 1987). Other measures of processing speed reveal similar differences. Paris, Gilson, Thomas, and Silver (1995) reported that when people were asked to shadow (repeat verbatim, as they listen to recorded passages, what a speaker is saying), they were less accurate when they listened to synthetic speech than when they listened to natural speech. …
['Carol R. Paris', 'Margaret H. Thomas', 'Richard D. Gilson', 'J. Peter Kincaid']
Linguistic cues and memory for synthetic and natural speech.