Motion along the mental number line reveals shared representations for numerosity and space
Our perception of numerosity and space are tightly interrelated , as evidenced by the ‘mental number line’ , where small numbers are mapped to the left and large numbers are mapped to the right ( Dehaene et al . , 1993 ) . This spatial arrangement of numbers is evident in preverbal infants ( de Hevia and Spelke , 2010 ) , non-human primates ( Drucker and Brannon , 2014 ) , and even birds ( Rugani et al . , 2015 ) , suggesting a deep evolutionary heritage of the mental number line , although there is evidence that it can be influenced by cultural practice ( Shaki and Fischer , 2008 ) . Number-space interactions are attributed to the fact that the neural bases of numerical and space processing both localize to parietal cortex ( Hubbard et al . , 2005 ) , as evidenced by the fact that parietal lesions disrupt both numerical and spatial processing ( Zorzi et al . , 2002 ) . It has been suggested that this co-localization is the result of ‘cortical recycling’ during evolution or normal development ( Anderson , 2010; Dehaene and Cohen , 2007 ) , i . e . , numerical cognition builds upon circuits that originally process space , because the computations that are used to gauge target coordinates or to transform reference frames can also be used for operations on numbers . But to what extent do such recycled circuits retain their original functionality ? Does numerosity replace space , or does it augment it , allowing for shared computations across domains ? We describe a new phenomenon in which visual motion direction , a spatial dimension ubiquitous in dorsal stream areas , adapts numerosity , giving rise to a repulsive aftereffect: motion to the left adapts small numbers , leading to overestimation of numerosity , while motion to the right adapts large numbers , resulting in underestimation . Thus , motion can literally move us up and down the mental number line . This indicates that numerosity coopts circuits encoding space . Furthermore , we show that the reference frame of this cross-adaptation effect is spatiotopic , not retinotopic , which , together with the tuning profile of the adaptation effect , indicates that motion direction-numerosity cross-adaptation may occur in a homolog of area LIP . Thus , recycled numerosity processing circuits indeed remain sensitive to purely spatial features . This shows that ‘cortical recycling’ augments but does not obliterate the spatial capacities of parietal circuits .
A second characteristic of the neural circuitry in the IPS is that it serves to convert retinotopic coordinates prevalent in early visual areas into eye- , head- , and world-centered frames of reference . In particular , areas LIP and VIP both contain neurons with eye- and head-centered coordinates ( Duhamel et al . , 1997; Mullette-Gillman et al . , 2005 ) . However , the two areas can be distinguished based on their respective receptive field properties: while neurons in area LIP have spatially restricted receptive fields , each covering only a portion of the visual world , number-sensitive neurons in area VIP , the next stage of number processing , respond to stimuli distributed over the entire visual field ( Nieder and Dehaene , 2009 ) . We capitalized on these distinct properties of LIP and VIP and tested in Experiment 6 whether the cross-adaptation effect between motion direction and number occurs in a retinotopic ( early visual areas ) or spatiotopic ( IPS ) frame of reference , and whether it is spatially specific ( LIP ) or occurs over the entire visual field ( VIP ) , thus allowing us to infer the cortical area in which motion and number processing interact . To this end , in Experiment 6 we replicated Experiment 1 but now also manipulated the location of the test/probe stimuli on the screen relative to subjects’ gaze ( Figure 4 ) : Subjects made two saccades between the adaptor and the test/probe stimuli . The second saccade would either bring their gaze back to the initial fixation location ( Figure 4a and d ) , or to a new location 14 degrees visual angle ( dva ) away from the original center of gaze ( Figure 4b and c ) . We then presented the test/probe stimuli either at the same location on the screen as the adaptor ( Figure 4b ) , at the same location as the adapter relative to the subject’s current gaze ( Figure 4c ) , or at a new location ( Figure 4d ) . This allowed us to compare two frames of reference and the spatial specificity of the cross-adaptation effect . If the cross-adaptation effect was present only when subjects looked at a new location on the screen but the stimuli were presented at the same position relative to their current center of gaze ( Figure 4c ) , it must take place in areas with a retinotopic frame of reference , i . e . , early visual areas; if cross-adaptation was evident only when subjects fixated at a new location but the stimuli were presented at the original location on the screen ( Figure 4b ) , it must occur in an area with a spatiotopic frame of reference and with restricted receptive fields , such as LIP; and if it occurred when subjects looked at the original fixation spot but stimuli were presented at a location that neither matched retino- nor spatiotopic reference frames ( Figure 4d ) , the area in which cross-adaptation takes place must support effects across the entire visual field , as would be predicted for VIP . Figure 5 shows that cross-adaptation between motion direction and number occurs even in the presence of two intervening saccades , replicating Experiment 1 . More importantly , the results of Experiment 6 show that cross-adaptation takes place in a spatiotopic , not retinotopic frame of reference , but is restricted to the spatiotopic location of the stimulus and does not extend over the full field of view ( motion direction × reference frame interaction , F ( 3 , 15 ) =4 . 58 , p=0 . 018 , η2=0 . 478 ) . Only the ‘same location’ and the ‘spatiotopic’ condition were significantly different from zero ( both p=0 . 015 , Wilcoxon signed rank tests ) , while the ‘retinotopic’ and ‘full field’ condition were not ( p=0 . 71 and p=0 . 84 , respectively ) . Bayesian statistics ( Benavoli et al . , 2014 ) confirmed that there was overwhelming evidence for an effect in the ‘same location’ and ‘spatiotopic’ ( posterior probabilities >95% ) , but not in the retinotopic ( posterior probability <44 . 5% ) or full field ( posterior probability <29 . 8% ) conditions . Retinal eccentricity cannot explain our results , since it was matched in the ‘same location’ and the ‘retinotopic’ conditions ( 0 dva ) , and in the ‘spatiotopic’ and ‘full field’ conditions ( 14 dva ) , respectively , and thus orthogonal to the factor reference frame . Instead , spatiotopy allots the effect to a family of MAEs which are thought to tap into higher-order motion processing stages , such as the positional motion aftereffect ( Turi and Burr , 2012 ) , in contrast to the classical MAE , which occurs in a retinotopic frame of reference ( Knapen et al . , 2009 ) . This pattern of results is predicted by cross-adaptation occurring in a human LIP homolog . 10 . 7554/eLife . 10806 . 010Figure 4 . Paradigm to test cross-adaptation in different reference frames . The design of Experiment 6 closely followed that of Experiment 1 , but introduced two saccades ( arrows ) between the top-up adaptor and the test/probe stimuli . Initial stimulus and fixation locations are depicted in blue , final stimulus and fixation locations in red . ( a ) In the ‘same location’ condition , subjects made a 14 dva leftward or rightward saccade after the top-up adaptor had been presented , and then back to the original fixation dot . The location of the test/probe stimuli and adaptor was the same in space and on the retina , depicted here as blue ( initial ) and red ( final ) half-cycles . This condition served to replicate the cross-adaptation effect found in Experiment 1 . ( b ) In the ‘spatiotopic’ condition , subjects made two consecutive leftward or rightward saccades ( first 14 dva , second 28 dva ) , and test/probe stimuli were presented at the same screen position as the top-up adaptor but on a different retinal position . Here , a cross-adaptation effect would occur in an area that has spatiotopic but locally restricted receptive fields , such as LIP . ( c ) In the ‘retinotopic’ condition , subjects also made two consecutive leftward or rightward saccades ( first 14 dva , second 28 dva ) but test/probe stimuli were presented above and below the new fixation dot . Hence , stimuli occurred at the same retinal location ( relative to the subjects’ gaze ) , but at different location in spatiotopic coordinates . Here , the cross-adaptation effect would occur in early , retinotopically organized visual areas . ( d ) In the ‘full field’ condition subjects first made rightward or leftward saccaded ( 14 dva ) , and then back to the original fixation dot ( 14 dva , as in the ‘same location’ condition ) . Test/probe stimuli were however presented at a new location opposite to the second saccade direction . Here , a cross-adaptation effect would only occur if the effect takes place in an area with very large receptive fields , such as VIP . DOI: http://dx . doi . org/10 . 7554/eLife . 10806 . 01010 . 7554/eLife . 10806 . 011Figure 5 . The reference frame of motion direction-numerosity cross-adaptation . A non-parametric rmANOVA showed that there was a significant interaction between reference frame and motion direction ( F ( 3 , 15 ) =4 . 58 , p=0 . 018 , η2=0 . 478 ) . Only the ‘same location’ condition which replicated Experiment 1 with intervening saccades and the ‘spatiotopic’ condition with restricted receptive field sizes as predicted for area LIP showed a significant ( both p<0 . 05 ) cross-adaptation effect from motion-direction on numerosity ( see text for statistics ) . Together with the tuning function of the effect , this indicates that the influence of motion direction on numerosity takes place at an intermediate , analog stage of number processing , and not in early visual areas with a retinotopic frame of reference or in area VIP with receptive fields covering the entire visual field . Data are represented as mean ± SEM . All data shown here are publicly available at Figshare ( Schwiedrzik et al . , 2015 ) . DOI: http://dx . doi . org/10 . 7554/eLife . 10806 . 011
Taken together , our results show on purely behavioral grounds that the neural circuits recycled to process number retain sensitivity to a visual feature intimately linked to our sense of space , namely motion direction . This indicates that ‘cortical recycling’ expands the original functionality of a circuit instead of replacing it , thus establishing a functional link across domains . The combination of properties we find in our study , namely the tuning function , receptive field size , and frame of reference of the motion direction-numerosity cross-adaptation effect matches the known properties of area LIP neurons at the first stage of number processing , which are broadly tuned to numerosity ( Roitman et al . , 2007 ) , have relatively small receptive fields ( Ben Hamed et al . , 2001 ) , and a spatiotopic frame of reference ( Mullette-Gillman et al . , 2005 ) . In contrast , neurons in area VIP , the next stage of number processing , are narrowly tuned and have large receptive fields ( Nieder and Dehaene , 2009 ) , and neurons in area MT , a preceding stage of motion processing , have a retinotopic frame of reference ( Born and Bradley , 2005 ) . Hence , a human LIP homolog in the IPS seems to be the most likely origin of the motion direction-numerosity cross-adaptation effect . Motion direction-numerosity cross-adaptation is consistent with studies showing that transcranial magnetic stimulation ( TMS ) over parietal cortex can disturb number comparisons and motion detection ( Salillas et al . , 2009 ) , as well as those showing that the number of items in a display can be overestimated when they move at high speed ( Afraz et al . , 2004 ) . However , our studies extend these findings in two critical ways: First , by showing that the very same circuits , and not only the same brain area targeted with TMS , process number and motion; and second , by showing that sensitivity of number processing extends to a spatial feature that is orthogonal to magnitude , namely motion direction ( motion speed is a magnitude itself ) . The latter rules out that motion direction-numerosity cross-adaption is simply the consequence of the operation of a ‘generalized magnitude processing system’ ( Walsh , 2003 ) . Furthermore , as our results were not effector-dependent , they strongly suggest that number-space associations occur at the level of the representation , not sensorimotor mapping , which is another function of parietal cortex ( Fogassi and Luppino , 2005 ) and has been invoked as an alternative explanation for number-space associations ( Keus and Schwarz , 2005 ) . Motion direction-numerosity cross-adaption can also not be easily explained by attention . Because moving stimuli can attract attention in their direction of motion ( Mattingley et al . , 1994; Shi et al . , 2010 ) , it is thinkable that attention was drawn down the number line during leftward motion adaptation , and up the number line during rightward motion adaptation . The consequence of this would however be the opposite pattern of results from what we found , namely underestimation after leftward motion ( because attention is oriented down the number line ) and overestimation after rightward motion ( because attention is oriented up the number line ) . Experiment 4 also ruled out that attention was drawn to the left or to the right by illusory motion arising from a classical MAE ( Figure 2—figure supplement 1 and Figure 2—figure supplement 2 ) . Hence , attention is an unlikely explanation for motion direction-numerosity cross-adaptation . Our results are indicative of an opponent coding model such as the one proposed for the classical MAE ( Mather and Harris , 1998 ) , in which large and small numerosities compete ( Figure 6 ) . Here , adaptation causes broadly tuned neurons coding for large numbers to reduce their responsiveness , giving rise to an overshoot in the activity of neurons that code for small numbers , and vice versa , leading to under/overestimation . This is evident in the general rightward shift of PSEs reported in Experiment 1 and the leftward shift of PSEs reported in Experiment 2 , and consistent with previously reported adaptation effects of static numerosity ( Burr and Ross , 2008 ) . Our model also captures the co-tuning of leftward motion and small numbers/rightward motion and large numbers: leftward motion increases the activity in the small number pool , resulting in increased overestimation after adaptation , while rightward motion increases the activity in the large number pool , resulting in increased underestimation after adaptation . This causes the differential shifts of PSEs with motion direction along the mental number line . 10 . 7554/eLife . 10806 . 012Figure 6 . Hypothetical model of motion direction-numerosity cross-adaptation . We can explain the cross-adaptation effect using a two-pool opponent coding model similar to what is often invoked to explain the classical MAE and other aftereffects ( Mather and Harris , 1998 ) . Panel ( a ) exemplifies the effect of motion direction during adaptation with large numerosities ( Experiment 1 , Figure 2a ) , panel ( b ) exemplifies the effect of motion direction during adaptation with small numerosities ( Experiment 2 , Figure 2c ) . The two black lines reflect the activity level of two broadly tuned pools of neurons that respond maximally to small or large numbers , respectively . The tuning curves are overlapping such that any given stimulus produces activity in both pools . Adaptation to number causes each pool to reduce its firing rate proportionally to how well the stimulus activates the respective pool in the non-adapted state , e . g . , static or randomly moving dot clouds with large numbers of dots ( as in Experiment 1 , e . g . , in the lower cloud , or in Burr and Ross , 2008 ) lead to stronger adaptation in the pool for large numbers ( blue ) . After adaptation , this pool’s response is reduced , shifting the point at which the two tuning curves intersect to the either to the right ( a ) , i . e . , towards larger numbers , or to the left ( b ) , i . e . , towards smaller numbers . This also shifts the point of subjective equality ( PSE ) in a comparison task ( gray ) , leading to an underestimation ( a ) or overestimation ( b ) in how many dots are perceived . During adaptation with large numerosities , rightwards motion exaggerates underestimation ( a ) by activating the pool for large numbers more strongly , moving perception up the number line ( green , inset ) , subsequently causing a stronger cross-adaptation effect ( underestimation ) . In contrast , leftwards motion reduces the effect , moving perception down the number line ( orange , inset ) , subsequently causing a cross-adaption effect in the opposite direction . During adaptation with small numerosities , leftwards motion exaggerates overestimation ( b ) by activating the pool for small numbers more strongly , moving perception down the number line ( green , inset ) , subsequently causing a stronger overestimation effect . Rightwards motion reduces overestimation , moving perception up the number line ( orange , inset ) . Monotonically increasing and decreasing tuning curves are consistent with numerosity tuning curves measured in monkey LIP ( Roitman et al . , 2007 ) and with the fact that we could exert an effect on the perception dot numerosities far removed from the adaptor , e . g . , perception of 23–107 dots when adapting with 400 dots in Experiment 1 , which would not be expected if each number was represented by narrow tuning curves as those found in monkey VIP ( Nieder et al . , 2006 ) . This was also supported by Experiment 5 which showed that there was no motion direction-numerosity cross-adaption when the number of dots in the adapter and the probe were matched ( mean difference right vs . left −3 . 69 , p=0 . 92 , Wilcoxon signed rank test , one-sided; Bayesian posterior probability <0 . 18% ) , as predicted for two-pool opponent coding models ( Robbins et al . , 2007; Webster , 2011 ) . DOI: http://dx . doi . org/10 . 7554/eLife . 10806 . 012 Opponent coding further suggests that large adaptor – probe/test distances should lead to large repulsive aftereffects as they maximize the differential response ratio between the two pools , while models with multiple , narrowly tuned channels would predict the opposite ( Robbins et al . , 2007; Webster , 2011 ) . Specifically , if narrowly tuned neurons such as those found in area VIP ( Nieder et al . , 2006 ) are adapted , the strongest effects of adaptation occur directly around the adaptor , while effects fall off as a function of adaptor-test/probe distance . This is characteristic of multichannel models . In contrast , in two-pool opponent coding with broadly tuned neurons such as those found in area LIP ( Roitman et al . , 2007 ) , adaptation at the probe location leads to no or only very small effects ( assuming the probe stimulus serves as an explicit reference point ) , as adaptation influences the activity in both pools approximately equally , thus merely reinforcing the current level of overall adaptation . Indeed , we found that large adaptor – probe/test distances led to large repulsive aftereffects , while adapting and probing with the same number of dots did not lead to a significant motion direction-specific aftereffect . This further speaks for two-pool opponent coding and against multichannel models underlying numerosity adaptation . Together , this suggests that current theoretical models of number processing ( e . g . , Dehaene and Changeux , 1993 ) should be expanded to include such an opponent process at the analog summation stage , which would be compatible with the broad neural tuning curves found in monkey area LIP ( Roitman et al . , 2007 ) . Finally , it has been shown that the approximate number sense predicts later mathematical achievement ( Halberda et al . , 2008 ) . If the link between motion and number can be further substantiated , it may open the venue to utilizing motion processing as a marker for number skill development ( Sigmundsson et al . , 2010 ) , and thus to identify children at risk for dyscalculia early in life .
Fifty-five subjects participated in six experiments: In Experiment 1 we investigated motion direction-numerosity cross-adaptation with large numerosities in 11 subjects ( 4 female , mean age 22 . 8 years , range 19–33 years , 9 right handed ) . In Experiment 2 , we tested for motion direction-numerosity cross-adaptation with small numerosities in 11 subjects ( 4 female , mean age 27 . 1 years , range 22–35 years , 9 right handed ) ; one subject had to be excluded from data analyses because of an apparent failure to do the task . Experiment 3 addressed direct effects of motion direction on numerosity perception in the absence of adaptation in 10 subjects ( 5 female , mean age 32 . 2 years , range 18–48 years , 9 right handed ) . Experiment 4 served as a control for classical MAEs in 11 subjects ( 6 female , mean age 33 . 45 years , range 23–48 years , 10 right handed ) . Experiment 5 allowed us to contrast different coding schemes , i . e . , two-pool opponent process vs . multichannel models in 6 subjects ( 3 female , mean age 29 . 8 years , range 22–39 years , 5 right handed ) . Experiment 6 was conducted to probe the reference frame of motion direction numerosity cross-adaptation in 6 subjects ( 2 female , mean age 23 . 1 years , range 21–30 years , all right handed ) . All subjects had normal or corrected-to-normal vision , reported no history of neurological or psychiatric disease , and gave written informed consent before participation . All procedures were approved by The University Committee on Activities Involving Human Subjects at New York University . Subjects received monetary compensation for their participation . Stimulus presentation and response collection in all experiments were controlled using Presentation ( v17 . 2; Neurobehavioral Systems , Berkeley , CA , USA ) . Trials on which reaction times exceeded 2*median absolute deviation were excluded from the analyses . Psychometric functions were fitted in the Palamedes Toolbox ( v1 . 7 . 0 ) ( Prins and Kingdom , 2009 ) in Matlab ( R2012b , The Mathworks , Natick , MA , USA ) , using a logistic function to obtain threshold and slope estimates per subject ( fits using a cumulative Gaussian function yielded similar results ) . For Experiments 1 , 2 , and 3 threshold and slope estimates , respectively , were compared using paired t-tests ( two-sided ) in SPSS ( v22 , IBM , Armonk , NY , USA ) . Effect sizes were computed as Hedges’ d ( Hedges and Olkin , 1985 ) , where values >0 . 8 indicate a large effect and values <0 . 5 indicate a small effect . For Experiment 4 , we compared mean percentage ‘same’ responses using two-sided one-sample and paired t-tests; analyses of median percent ‘same’ responses using a percentile bootstrap procedure ( Wilcox , 2010 ) yielded the same results . Given the sample sizes in Experiments 5 and 6 , we resorted to non-parametric tests . Because we had a directional hypothesis ( PSE right>left ) in both experiments , planned comparisons were performed using one-sided exact Wilcoxon signed rank tests , but two-sided tests yielded identical patterns of results . In Experiment 6 , threshold estimates were also entered into a non-parametric two factor repeated measures analysis of variance ( rmANOVA ) based on the aligned rank transform ( Higgins and Tashtoush , 1994 ) using ARTool ( v1 . 5 . 1 ) ( Wobbrock et al . , 2011 ) to assess interaction effects . Effect size was computed as partial η2 , where values >0 . 14 indicate a large effect and values <0 . 06 indicate a small effect . Spatial reference frames in Experiment 6 were determined by testing each experimental condition against 0 . While it is standard in the literature to compare reference frames by contrasting conditions against 0 ( Arrighi et al . , 2014; Knapen et al . , 2009; Turi and Burr , 2012 ) , we note that such conclusions partially rest on non-significant results . We thus also performed Bayesian statistics , using an exact Wilcoxon signed rank test based on the prior ignorance Dirichlet process model ( 200000 samples , concentration parameter s=0 . 53 ) ( Benavoli et al . , 2014 ) to support our conclusions . All data are publicly available at Figshare ( Schwiedrzik et al . , 2015 ) .
Perception of number and space are tightly intertwined . It has been proposed that this is due to ‘cortical recycling’ , where numerosity processing takes over circuits originally processing space . Do such ‘recycled’ circuits retain their original functionality ? Here , we investigate interactions between numerosity and motion direction , two functions that both localize to parietal cortex . We describe a new phenomenon in which visual motion direction adapts nonsymbolic numerosity perception , giving rise to a repulsive aftereffect: motion to the left adapts small numbers , leading to overestimation of numerosity , while motion to the right adapts large numbers , resulting in underestimation . The reference frame of this effect is spatiotopic . Together with the tuning properties of the effect this suggests that motion direction-numerosity cross-adaptation may occur in a homolog of area LIP . ‘Cortical recycling’ thus expands but does not obliterate the functions originally performed by the recycled circuit , allowing for shared computations across domains .
Our sense of number is thought to have emerged from the circuits of cortical neurons in the brain that originally represent space , a process known as 'cortical recycling' . Accordingly , our perception of space and number are tightly intertwined: for example , people think about numbers on a mental number line , where smaller numbers are mapped to the left , and larger numbers are mapped to the right . Also , damage to a brain region called the parietal cortex disrupts both space and number processing . If number processing recycles the neurons that encode space , which form does this appropriation take ? Recycling could preserve the original behavior of the neurons ( processing space ) , thus enriching the neurons’ functional repertoire with a new capacity ( processing number ) . Alternatively , the newly developed role could replace the original one , such that space and number cohabitate the same brain area but use separate neurons . To disentangle these hypotheses , Schwiedrzik et al . used a technique called 'perceptual adaptation' . Here , continuously showing someone a particular feature eventually exhausts the neurons that respond to that feature . Neurons that respond to the opposite feature are however less exhausted and dominate perception . Consequently , people perceive the opposite of what they are adapted to . For example , after continuously seeing dots moving to the right , people perceive stationary dots as moving to the left . Similarly , after being exposed to large numbers of dots they underestimate how many dots they see . If the same neurons process numbers and space , then adapting to movement in a particular direction should influence number perception . During Schwiedrzik et al . ’s experiments , volunteers watched moving dots on a computer screen . After seeing dots move to the right , they underestimated the number of dots that then appeared on the screen . This is likely to be because larger numbers are mentally mapped to the right , and seeing rightward motion for a long time exhausted these neurons . This means that neurons that respond to smaller numbers ( mentally mapped to the left ) were more active when the new dots were presented , leading the volunteers to underestimate how many dots they saw . Adapting to leftward motion led to the opposite effect , with volunteers overestimating the number of dots . Thus , motion can literally move us up and down the number line . These results indicate that the same neurons encode both space and numbers . Cortical recycling does not erase the neurons’ original behavior: instead , neurons may carry out the same computations when processing numbers or space . This would allow the brain to add new functionality without sacrificing any of the computational resources for processing space .