review massaro 1998 massaro w 1998 perceiving talking faces speech perception behavioral principle mit press cambridge mass 552 pp 212 illus 1 cdrom 55 00 reviewed byl noel nguyen laboratory psycholinguistics fpse university geneva switzerland nnguyen fapse unige ch 1 synopsis 1 1 general outline book concerned multiple sources information processed speech perception generally pattern recognition based upon important research programme conducted massaro colleagues last two decades book focuses perception socalled bimodal speech addressing wide range issues way visual information provided speaker face auditory information combined perceptual system scope book much larger however massaro purpose describe defend new psychological law relevant wide variety domains contrast already wellestablished laws kind e g weber law perception unidimensional new principle multidimensional describes several factors impact behaviour principle embodied computational model pattern recognition fuzzy logical model perception flmp whose latest version presented discussed detail flmp systematically contrasted alternative computational models using broad perceptual database benchmark throughout book separate part book also deals methods synthesizing talking faces experiments bimodal speech perception introduces baldi talking face developed massaro coworkers book accompanied cdrom contains series demonstrations relating many topics dealt book divided four main sections section 1 perceiving talking faces focuses perception speech ear eye massaro reviews significant empirical findings domain discusses main methodological issues presents general classification existing computational models bimodal speech perception central section idea speech perception obeys general behavioural principle integration different sources information section 2 broadening domain aims assessing well principle holds across broad individual situational variability author demonstrates interindividual variations bimodal speech perceived depending listener age native language instance accounted within flmp framework using examples taken different perceptual cognitive situations massaro also defends idea flmp adequately describes information processing irrespective situational differences section 3 broadening framework opens presentation extended explicit version flmp designed particular account dynamics speech processing section also includes detailed analysis methodological issues involved assessing quantitative predictions psychology along discussion critiques expressed investigators flmp years finally section 4 creating talking faces specifically dedicated synthesis visual speech 1 2 new behavioural principle although many readers may already familiar massaro fuzzy logical model perception shall assume contrary proceed present brief outline model central assumption flmp pattern recognition involves common set processes regardless specific nature patterns speech seen associated dedicated processing module motor theory speech perception liberman 1996 instance contrary sensory information assumed processed way whether brain busy recognizing speech sounds letters manual gestures take examples cases flmp postulates mapping stimulus unique perceptual category entails going three main stages processing feature evaluation stage feature integration stage decision stage evaluation stage consists converting available sources information set properties referred features feature given continuous fuzzy truth value represents degree stimulus corresponds set internal prototypical patterns along particular perceptual dimension thus one important visual feature perception cv syllables degree opening lips model therefore assumes internal prototypes available perceptual system specify lips open onset syllable da closed ba etc second stage features integrated determine overall degree match sensory input prototypes e g syllables known receiver third final stage decision taken basis relative goodness match input prototype flmp makes number specific assumptions stage process first hypothesizes available sources information simultaneously brought play pattern recognition thus visible speech auditory speech assumed influence bimodal speech perceived second different sources information assumed evaluated independently means example visible speech effect auditory speech converted set features two sources information combined later stage processing model also makes specific assumptions sources information integrated multiplicative rule decisions taken relative goodness rule major prediction model influence one source information greatest source neutral ambiguous 19 prediction best illustrated experiment whose results served database testing models pattern recognition several occasions book chapters 2 11 experiment synthetic auditory stimuli ranging continuum ba da crossed visual stimuli also varying ba da bimodal stimuli presented subjects forcedchoice identification task along unimodal stimuli expanded factorial design shown massaro appropriate experimental design determining two sources information combined pattern recognition bimodal stimuli main results typically depicted twofactor plot proportion da responses ordinate levels auditory source information abscissa different curve levels visual source information represented way results clearly show statistical interaction two sources information specifically influence one source information proves larger middle ambiguous range source interaction graphically takes shape american football reason presented throughout book hallmark fuzzy logical model perception summary massaro proposes universal principle perceptual cognitive performance explain pattern recognition according principle people influenced multiple sources information diverse set situations many cases sources information ambiguous particular source alone usually specify appropriate interpretation perceiver appears evaluate multiple sources information parallel degree supports various interpretations integrate together derive overall support interpretation assess support alternative based alternatives select appropriate response p 291 2 critical evaluation 2 1 general evaluation book clearly major contribution study speech perception generally cognitive psychology admirably clear written quite elegant manner doubt book read great interest research scientists many different fields work result ambitious intellectual endeavour aimed introducing new behavioural law placed massaro equal footing weber law perception power law learning speech scientists presented extensive series experiments perception bimodal speech whatever stance take domain find quite challenging massaro view speech perception constitutes one aspect much general form cognitive processing namely pattern recognition computer scientists working field speech technology particularly interested book final section synthesis visual speech regardless background readers also find book worth using tutorial experimental methods available investigating speech perception great variety experimental paradigms tasks discussed length massaro also extensively discusses methods assessing computational models pattern recognition particular fitting models observed results respect using results experiment described reference database quite good initiative view allows reader easily understand massaro point new issues raised without go details experimental design time book also prove invaluable resource teaching care taken select prototypical results well set work historical context number rather fascinating anecdotes historical references given going mcgurk personal account discovery mcgurk effect audiovisual rendition introduction george miller seminal article ubiquitousness number 7 plus minus 2 miller face texturemapped onto baldi wireframe head cdrom accompanies book enables reader directly experience psychological illusions associated perception bimodal speech constitutes useful research teaching tool negative side massaro use ba da experiment leading strand throughout obviously results book focused perception nonsense syllables although interaction visible speech audible speech word recognition mentioned number occasions e g pp 2123 pp 181182 book contains suggestions perceive isolated words let alone connected speech also surprised fact little place devoted presenting current theories models speech perception although models trace mentioned several occasions book think fair say flmp still given lion share book also minor defects absence list figures fact cdrom bands 1 4 1 5 1 6 referred incorrectly text list cdrom selections pointed pages band referred another domain would quite interesting perceptual database used book made available cdrom although would probably required substantial amount additional work also found useful provided interactive version main computational models discussed book flmp race model single channel model etc flmp model downloaded massaro laboratory web site santa cruz http mambo ucsc edu currently distributed fortran code modified recompiled new set data operation probably reach many students psychology linguistics 2 2 specific comments familiar areas dealt book hide fact review biased towards interests namely production perception auditory speech following comments specifically concentrate two issues relating area research role features speech processing time course speech processing 2 2 1 features useful extensive comments made massaro status features model see particular chapter 2 chapter 10 long found difficult determine close features classical phonetic features book makes clear direct relation former latter indicated flmp postulates three main stages processing pattern recognition feature evaluation stage feature integration stage decision stage specific assumptions made model features integrated decision taken depending outcome integration set feature values therefore model predict probability occurrence possible response e g ba da however attention paid fact feature values way derived stimulus actually determined posteriori manner subjects observed responses using algorithm stepit allows deviation responses predicted ones minimal features seen model free parameters whose values set basis actual performance subject pattern recognition task make model perform best e maximize goodness fit according massaro model predicting exact form results postdicting actual quantitative values make overall predictions p 294 emphasis words stimulus occasion explicitly mapped onto internal features flmp model respect features defined flmp look markedly different phonetic features let us take example opposition ba da much emphasis put book acoustically b said differ according feature graveacute b classified grave acute case flmp features grave acute viewed target values referring prototypical stops however graveacute feature explicitly defined acoustical terms e g slope shortterm spectrum release stop see stevens blumstein 1978 contrary exact nature flmp features remains undetermined values subject one main constraint make model account subjects responses accurately possible thus acoustic structure stimulus directly taken consideration estimation feature values experiments using audible speech flmp features lend acoustic interpretation ba da experiment example prototypes ba da assumed include one auditory feature namely variations frequency second f2 third f3 formants onset vowel slightly falling f2f3 da rising f2f3 ba however interpretation stems fact f2 f3 onset frequencies precisely acoustic parameters manipulated experimenters synthesize auditory continuum ba da words acoustic significance flmp features derived way experiment designed model rely particular system acoustic features see example stevens blumstein 1978 alternative system system embodied experimental design external model practice therefore issue speech sounds mapped onto features addressed model clear several occasions massaro suggests determining advance given individual convert given stimulus set feature values simply reach stimulusto feature mapping shows variability said analogous variability weather many previous contributions influences allow quantitative prediction 135 fundamental distinction fact established flmp intake information e stimulusto feature mapping information processing e features combined mapped response cf p 135 flmp predicts information processed way one individual regardless whether relates speech sounds facial movements manual gestures etc assumed way information extracted stimulus contrary subject many sources variations accurately characterized ahead time understanding means socalled evaluation stage accounted model least much accuracy however least one occasion massaro suggest limitation consubstantial every model perception pattern recognition could circumvented way according one could indeed easily hypothesize functions relating feature values stimulus levels although would represent model information addition one information processing 294 emphasis suggests building model information feasible whether possibility flmp completed model kind e explicit stimulusto feature mapping stage issue remains addressed 2 2 2 time course speech processing time plays quite central role different ways book first massaro shows flmp explicitly formalized account dynamics perceptual processing chap 9 formalization presented reply criticisms expressed number investigators e g mcclelland 1991 pointed flmp accurately characterizes asymptotic outcome perceptual system e g probability particular response occur little say time course processing dynamic version flmp intended address reactions version stimulusto feature mapping assumed take certain amount time interval information stimulus gradually accumulates becomes increasingly accurate assumed accuracy increases negatively accelerated function processing time information gleaned early late processing stimulus one assumption integration separate features updated continuously featural information evaluated similarly decision occur time stimulus presentation 259 thus partial temporal overlap different stages processing sense one process begin previous process finished see also figure 2 1 p 41 assumptions time course information processing supported number experiments concerned effect backward masking recognition pure tones recognition letters speech obviously raises number specific issues domain however unlike written words speech temporal phenomenon continuous e systematic acoustic boundaries phonemes syllables words furthermore time per se serves source information speech pointed massaro e g vowel duration major cue voicing following obstruent take one example somewhat regrettably indications given model could assessed speech domain see remarks p 194 p 263 addition discussing dynamics processing massaro examines temporal relations sources information dealt pattern recognition chapter 3 focuses sensitivity temporal asynchronies visible audible speech experiments reported chapter bimodal cv syllables various degrees onset asynchrony auditory synthetic speech visible synthetic speech presented subjects forcedchoice identification task results show integration two sources information still occurs sources information made asynchronous provided time shift exceed certain duration one major challenge phoneticians psycholinguists alike characterize relationship could called external dynamics speech e temporal organization speech signal internal time course speech processing play role perception speech difficult tell apart respective influences listener behaviour samuel 1996 example gating study investigating role vowel duration cue voicing postvocalic stop cvc syllables warren marslen wilson 1988 found proportion voicedcoda responses increased listeners presented increasingly long portions initial cv sequence one obvious interpretation longer vowels perceived associated voiced coda rather voiceless ones keeping massaro dynamical flmp however may also assumed evaluating information provided vowel takes time evidence pointing voiced coda gradually accumulates processing time made available listener things equal thus finding raises issue differentiate effect vowel duration per se listener response internal dynamics processing although issue directly addressed book doubt flmp would constitute appropriate framework investigations domain 2 3 general conclusion book provides us quite extensive review work carried author others use multiple cues speech perception generally pattern recognition aimed large audience constitutes useful tool teaching research purposes doubt soon become major reference researchers phonetics psycholinguistics cognitive psychology 4 bibliography liberman 1996 speech special code mit press cambridge mass mcclelland j l 1991 stochastic interactive processes effect context perception cognitive psychology 23 144 samuel g 1996 role time lexical access journal acoustical society america 100 4 2 2572 stevens k n blumstein e 1978 invariant cues place articulation stop consonants journal acoustical society america 64 13581368 warren p marslen wilson w 1988 cues lexical choice discriminating place voice perception psychophysics 43 2130 5 biographical details reviewer lecturer laboratory psycholinguistics fpse university geneva switzerland current research covers variety topics ranging dynamics articulatory movements speech production phonetic bases word recognition thanks due uli frauenfelder helpful comments latex version document available upon request nnguyen fapse unige ch