to successfully interact with the stimuli of our environment , we need to process selectively the information most relevant for our tasks .
when stimuli are attended to their processing become more rapid , more accurate , and more detailed ( posner et al . , 1980 ;
desimone and duncan , 1995 ; carrasco and mcelree , 2001 ; carrasco et al . , 2004 ; liu et al . , 2005 , 2009 ) .
attention improves performance on several visual tasks , such as contrast sensitivity , speed and orientation discrimination as well as spatial resolution ( lee et al . , 1999 ;
are limited , when the stimuli demanding attention for a perceptual task exceed system capacity , performance decreases .
for example , in a visual search task in which an object ( target ) has to be detected amongst irrelevant items ( distractors ) , reaction times increase directly with distractor number ( unless the difference between the stimuli is so striking to make the target pop out from the cluttered scene ) .
this correlation reflects the limited capacity of selective attention that prevents the observer from monitoring all items at the same time .
similarly , when more than one perceptual task is performed at the same time , overall performance decreases because of the underlying processing limitations .
this occurs even for simple tasks , such as naming a word or identifying the pitch of a tone ( pashler , 1992 ; pashler and o'brien , 1993 ; huang et al . ,
interference between concurrent perceptual tasks of the same sensory modality has been consistently reported in many psychological and psychophysical studies ( navon et al . , 1984 ; pashler , 1994 ; bonnel and prinzmetal , 1998 ; alais et al . , 2006b ) .
however , the evidence for audiovisual cross - modal interference is conflicting ( duncan et al .
bonnel and hafter ( 1998 ) found that in a identification task in which the sign of a change ( luminance in vision and intensity in audition ) had to be detected , performance in dual - task conditions were lower than in the single - task conditions regardless the interference was in the same or different modalities .
spence et al . ( 2000 ) found that selecting an auditory stream of words presented concurrently with a second ( distractor ) stream , it is more difficult if a video of moving lips mimicking the distracting sounds it is also displayed .
these psychophysical findings are not only congruent with some of the cognitive literature of the 1970s and 1980s ( taylor et al .
, 1967 ; tulving and lindsay , 1967 ; alais et al . , 2006b ) , but also with recent neurophysiological and imaging results .
2004 ) examined the electrophysiological correlates for auditory interference with vision by an identification task of non - ambiguous complex stimuli such as faces and voices .
their results suggest that cross - modal interactions occur at various different stages , involving brain areas such as fusiform gyrus , associative auditory areas ( ba 22 ) , and the superior frontal gyri .
hein et al . ( 2007 ) showed with a functional magnetic resonance ( fmri ) study , that even without competing motor responses , a simple auditory decision interferes with visual processing at neural levels including prefrontal cortex , middle temporal cortex , and other visual regions . taken together these results imply that limitations on resources for vision and audition operate at a central level of processing , rather than in the auditory and visual peripheral senses .
however , much evidence also supports the notion of independence of attentional resources for vision and audition ( allport et al .
, 1972 ; triesman and davies , 1973 ; shiffrin and grantham , 1974 ; alais et al . , 2006b ; santangelo et al . , 2010 ) .
for example , larsen et al . ( 2003 ) compared subjects accuracy for identification of two concurrent stimuli ( such as a visual and spoken letter ) relative to performance in a single - task .
they found that the proportion of correct response was almost the same for all experimental conditions and , furthermore , in the divided - attention condition the probability to correctly report a stimulus in one modality was independent of whether the stimulus was correctly reported in the other modality .
similarly , bonnel and hafter ( 1998 ) used an audiovisual dual - task paradigm to show that when identification of the direction of a stimulus change is capacity - limited ( see above ) , simple detection of visual and auditory patterns is governed by  capacity - free  processes , as in the detection task there was no performance drop compared with single - task controls .
visual thresholds were unaffected by concurrent pitch discrimination of chords and vice versa . however , when two tasks were performed within the same modality , thresholds increased by a factor of around two for visual discrimination and four for auditory discrimination . in line with these psychophysical results ,
a variety of imaging studies suggests that attention can act unimodally at early levels including the primary cortices such as a1 and v1 ( jancke et al .
, 1999a , b ; posner and gilbert , 1999 ; somers et al . , 1999 ) .
most of the studies mentioned deal with dual - task conditions where both tasks are brief ( hundreds of milliseconds ) stimuli to be detected or discriminated . very few consider conditions in which one of the tasks must be performed by continuously monitoring a specific pattern over a temporal scale of seconds , even though this is a typical requirement for many everyday activities , such as reading or driving .
we measure performance on the multiple object - tracking ( mot ) task of pylyshyn and storm ( 1988 ) , while asking subjects to perform simultaneously either a visual contrast discrimination task or an auditory pitch discrimination task .
the results show strong within - modality interference , but very little cross - modality interference , strongly supporting the idea that in sustained tasks each modality has access to a separate pool of attentional resources .
four naive subjects ( two males and two females , mean age 26 years ) , all with normal hearing and normal or corrected - to - normal visual acuity , served as subjects . all gave informed consent to participate to the study that was conducted in accordance with the guidelines of the university of florence .
all visual stimuli were presented on a sony trinitron crt monitor ( screen resolution of 1024   768  pixels , 32 bit color depth , refresh rate of 60  hz , and mean luminance 68.5  cd / m ) subtending ( 40    30 ) at the subjects view distance of 57  cm . to create visual stimuli we used psychophysics toolbox ( version 2 ) for matlab ( brainard , 1997 ; pelli , 1997 ) on a mac g4 running mac osx 9 .
auditory stimuli were digitized at a rate of 65  khz , and presented through two high quality loudspeakers ( creative mms 30 ) flanking the computer screen and lying in the same plane 60  cm from the subject .
speaker separation was around 80  cm and stimuli intensity was 75  db at the sound source .
the primary task was visual tracking of multiple moving objects ( mot ; pylyshyn and storm , 1988 ) .
the mot task consisted of 12 disks ( diameter 0.9 ) moving across a gray background at 5/s .
they moved in straight lines , and when colliding with other dots or the sides bounced appropriately ( obeying the laws of physics ) . at the start of each trial
35 disks were displayed in green ( xyy coordinates =  0.25,0.69 , 39.5 ) for 2  s to indicate that those were the targets whilst the remaining were displayed an isoluminant red ( xyy coordinates  =  0.61 , 0.33 , 39.5 ) .
the trial continued for 4  s ( tracking period ) , then disks stopped and four became orange ( xyy coordinates =   0.52,0.44 , 39.4 ; see movie s1 in supplementary material ) .
the subjects task was to choose which of these was the target ( only one valid target turned orange on each trial ) .
subjects were familiarized with the task during a training session of 50 trials before starting the experimental protocol .
each experimental session had five trials per condition ( varying in the number of dots to track ) for a total of 15 trials per session .
no feedback was provided , but subjects could check their overall performance at the end of each session .
stimuli for the secondary visual task were luminance - modulated gratings of 0.5  s duration with a spatial frequency of 3  c / deg covering the entire screen . on each trial
( 4  s duration , during the dot tracking ) subjects were presented with a sequence of three gratings , ramped in and out within a raised cosine envelope ( over 20  ms ) , with an inter stimulus interval randomly chosen between 0.5 and 1.3  s. two out of three gratings had the same contrast ( 50% ) while the target grating ( that subjects had to detect ) , randomly first , second or third in the sequence , had more or less contrast .
the size of the contrast difference (  ) was chosen from trial to trial by means of an adaptive staircase quest ( watson and pelli , 1983 ) that homed in on threshold ( 67% of correct responses ) .
the auditory secondary stimulus was of a sequence of three tones with the same presentation duration and temporal spacing as the visual version , two reference stimuli of 880  hz with the target frequency differing from trial to trial by  hz . in the dual - task condition subjects performed both the contrast or frequency discrimination task , and the mot task . to avoid possible biases for response order we counterbalanced subjects responding first to the mot task with those that responded for to the secondary task .
four naive subjects ( two males and two females , mean age 26 years ) , all with normal hearing and normal or corrected - to - normal visual acuity , served as subjects . all gave informed consent to participate to the study that was conducted in accordance with the guidelines of the university of florence .
all visual stimuli were presented on a sony trinitron crt monitor ( screen resolution of 1024   768  pixels , 32 bit color depth , refresh rate of 60  hz , and mean luminance 68.5  cd / m ) subtending ( 40    30 ) at the subjects view distance of 57  cm . to create visual stimuli we used psychophysics toolbox ( version 2 ) for matlab ( brainard , 1997 ; pelli , 1997 ) on a mac g4 running mac osx 9 .
auditory stimuli were digitized at a rate of 65  khz , and presented through two high quality loudspeakers ( creative mms 30 ) flanking the computer screen and lying in the same plane 60  cm from the subject .
speaker separation was around 80  cm and stimuli intensity was 75  db at the sound source .
the primary task was visual tracking of multiple moving objects ( mot ; pylyshyn and storm , 1988 ) .
the mot task consisted of 12 disks ( diameter 0.9 ) moving across a gray background at 5/s .
they moved in straight lines , and when colliding with other dots or the sides bounced appropriately ( obeying the laws of physics ) . at the start of each trial
35 disks were displayed in green ( xyy coordinates =  0.25,0.69 , 39.5 ) for 2  s to indicate that those were the targets whilst the remaining were displayed an isoluminant red ( xyy coordinates  =  0.61 , 0.33 , 39.5 ) .
the trial continued for 4  s ( tracking period ) , then disks stopped and four became orange ( xyy coordinates =   0.52,0.44 , 39.4 ; see movie s1 in supplementary material ) .
the subjects task was to choose which of these was the target ( only one valid target turned orange on each trial ) .
subjects were familiarized with the task during a training session of 50 trials before starting the experimental protocol .
each experimental session had five trials per condition ( varying in the number of dots to track ) for a total of 15 trials per session .
no feedback was provided , but subjects could check their overall performance at the end of each session .
stimuli for the secondary visual task were luminance - modulated gratings of 0.5  s duration with a spatial frequency of 3  c / deg covering the entire screen . on each trial
( 4  s duration , during the dot tracking ) subjects were presented with a sequence of three gratings , ramped in and out within a raised cosine envelope ( over 20  ms ) , with an inter stimulus interval randomly chosen between 0.5 and 1.3  s. two out of three gratings had the same contrast ( 50% ) while the target grating ( that subjects had to detect ) , randomly first , second or third in the sequence , had more or less contrast .
the size of the contrast difference (  ) was chosen from trial to trial by means of an adaptive staircase quest ( watson and pelli , 1983 ) that homed in on threshold ( 67% of correct responses ) .
the auditory secondary stimulus was of a sequence of three tones with the same presentation duration and temporal spacing as the visual version , two reference stimuli of 880  hz with the target frequency differing from trial to trial by  hz . in the dual - task condition subjects performed both the contrast or frequency discrimination task , and the mot task . to avoid possible biases for response order we counterbalanced subjects responding first to the mot task with those that responded for to the secondary task .
to evaluate the costs of dividing attention between sensory modalities , we measured subject performance for visual tracking alone , or with either an auditory or a visual secondary task . figure 1 shows the individual results for the three experimental conditions , plotting performance ( d ) in the dual - task conditions against single - task performance .
each small symbol indicates individual subject performance in a given condition defined by the number of dots to track whilst large circles indicate the data averaged across subjects and conditions .
it is quite clear that the concurrent visual task greatly reduced performance , shown by the average decrease in d from 2.48 to 1.50 , and also by the fact that all individual data lie below the equality line .
the difference was highly significant ( one - tailed paired t - test : t11  =  6.98 , p  <  0.001 ) . however , when the competing task was auditory rather than visual , there was no effect on tracking performance .
average d was virtually unchanged ( 2.48 vs 2.28 ) , certainly not significant ( one - tailed paired t - test ; t11  =  1.07 , p  =  0.30 ) .
sensitivities for the mot task performed alone ( on the abscissa ) plotted against sensitivities for dual - task conditions ( on the ordinate ) .
the 12 data points represent 4 subjects in 3 experimental conditions , defined by the number of dots to track ( from 3 to 5 ) .
green symbols refer to the intra - modal condition ( secondary task contrast discrimination ) , red to the cross - modal condition ( auditory secondary task ) .
small symbols refer to individual data ( different symbols shape indicates different number of dots to track : three dots  triangles , four dots  diamond , and five dots  pentagon ) whilst large symbols to averages .
there is a clear effect for intra - modal interference , but not for cross - modal interference .
we also measured sensitivity for both the visual and auditory secondary tasks when performed alone and matched these results with those achieved in the dual - task condition .
psychometric functions for auditory frequency discrimination ( left panel ) and visual contrast discrimination ( right panel ) for subjects f. g. performance in the auditory task was almost identical when frequency discrimination was performed alone ( black data points and lines ) or together with a visual mot task ( blue data points and lines ) as shown by the almost overlapping curves . however , when the two concurrent tasks were of the same sensory modality ( vision ) , subject performance was dramatically reduced by around a factor of 3 .
auditory frequency discrimination is shown on the left , visual contrast discrimination on the right .
it is obvious that the auditory discrimination was little affected by the concurrent visual tracking task .
the two psychometric functions ( best fitting cumulative gaussian functions ) are virtually identical , yielding thresholds (  frequency yielding 66% correct target identification ) close to 67  hz in both conditions .
however , visual contrast discrimination thresholds were much higher in the dual than in the single - task condition , 5.1 compared with 14.5 ( a factor of nearly three ) . figure 3 plots for all subjects the interference factor ( ratio of dual- to single - task thresholds ) for the within and between modality conditions .
it is clear that the auditory task is relatively immune to interference ( average factor 0.05 ) , while performance for luminance contrast discrimination thresholds increased by a factor of more than 2.5 .
subject performance on the secondary task , either auditory ( red bars ) or visual ( green bars ) .
the interference factor is defined as the ratio between dual - task and single - task thresholds ( a value of one meaning no interference between modalities .
in this paper we asked whether vision and audition share cognitive attentional resources in performing sustained tasks , particularly relevant for everyday functioning .
as most previous research has been restricted to tasks spanning only a few milliseconds ( larsen et al . , 2003 ; alais et al . , 2006b ) , or conditions with fast streams of simple auditory or visual patterns ( duncan et al . , 1997 )
, our study provides new knowledge about attentional mechanisms in ecological situations , where prolonged monitoring of information is necessary .
the results clearly indicate that under these conditions , vision and audition have access to separate cognitive resources .
performance on a sustained task , typical of everyday requirement , was completely unaffected by a concurrent auditory discrimination task .
the lack of interference did not reflect a bias in deploying attention on the visual primary task more than on the auditory task , as both tasks were performed as well as when they were presented alone . on the other hand ,
sharing attention between two tasks of the same sensory modality produces a robust decrease of performance for both primary and secondary tasks .
that vision and audition have access to separate cognitive resources is consistent with imaging studies showing that attention can modulate responses in primary and secondary visual and auditory cortexes ( gandhi et al .
if in both modalities attentional effects modulate neural responses at these early of sensory information processing , when the visual and auditory signals are relatively independent , it is reasonable that few interactions are seen between these two senses .
our results are important not only for the psychophysical data on the role of sustained attention between modalities , but also because they establish guidelines in designing audio  visual instrumentation .
information should be divided as much as possible between modalities , to maximize on the attentional resources available .
this becomes increasingly more important as more virtual - reality applications are developed and are used routinely in everyday life .
the authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest .
 our perceptual capacities are limited by attentional resources . 
 one important question is whether these resources are allocated separately to each sense or shared between them . 
 we addressed this issue by asking subjects to perform a double task , either in the same modality or in different modalities ( vision and audition ) . 
 the primary task was a multiple object - tracking task ( pylyshyn and storm , 1988 ) , in which observers were required to track between 2 and 5 dots for 4  s. concurrently , they were required to identify either which out of three gratings spaced over the interval differed in contrast or , in the auditory version of the same task , which tone differed in frequency relative to the two reference tones . 
 the results show that while the concurrent visual contrast discrimination reduced tracking ability by about 0.7  d , the concurrent auditory task had virtually no effect . 
 this confirms previous reports that vision and audition use separate attentional resources , consistent with fmri findings of attentional effects as early as v1 and a1 . 
 the results have clear implications for effective design of instrumentation and forms of audio  visual communication devices . 
Introduction
Materials and Methods
Subjects
Stimuli and procedure
Results
Discussion
Conflict of Interest Statement
Supplementary Material