## Linton Stereo Illusion

Paul Linton 1,2,3

- 1 NOMIS Foundation Fellow, Italian Academy for Advanced Studies, Columbia University
- 2 Presidential Scholar in Society and Neuroscience, Center for Science and Society, Columbia University
- 3 Visual Inference Lab, Zuckerman Mind Brain Behavior Institute, Columbia University

We present a new illusion that challenges our understanding of  stereo vision. The illusion consists of a larger circle at 50cm, and smaller circle in front of  it at 40cm, with constant angular sizes throughout. We move the larger circle forward by 10cm (to 40cm) and then back again (to 50cm). The question is, what distance should we move the smaller circle forward and back to maintain a constant perceived separation in depth between the circles? Constant physical distance (10cm) or constant retinal disparity (6.7cm)? Observers choose constant disparity. We therefore argue the 'Linton Stereo Illusion' appears to suggest that perceived stereo depth reflects retinal disparities rather than 3D geometry.

When do two circles appear to move rigidly in depth together? In the 'Linton Stereo Illusion' we test this question by moving a back circle (starting at 50cm) and a front circle (starting at 40cm) forwards and  backwards,  whilst  we  keep  their  angular  size  constant  throughout.  We  move  the  larger  circle forward by 10cm (to 40cm) and then back again (to 50cm). We move the smaller circle forward to one of  two  different  distances  -  30cm  in  a  'constant  physical  separation'  condition,  and  33.3cm  in  a 'constant retinal disparity' condition - and then back again. The question is, in which of  the two conditions do the two circles appear to move rigidly (parallel) in depth together?

[Insert Video 1 here: https://www.youtube.com/watch?v=krAfioCxJVQ]

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00770v2-figure-1f85d0ec-9212-4b24-8e57-8bcf23986250.jpg)

Video 1. The two conditions of the 'Linton Stereo Illusion'.

The two conditions reflect two very different ways of  thinking about stereo vision:

First, 'Triangulation' models of  stereo vision, that date back to (Kepler, 1604) and (Descartes, 1637) via (Wheatstone, 1838), suggest that stereo vision recovers the physical scene. In which case, we would expect to see the circles moving rigidly in depth together when we keep the physical distances between them constant as they move.

By contrast, a second approach ((Linton, 2023)(Linton, 2021b)), argues that perceived stereo depth simply reflects the disparities on the retina. In which case, we would expect to see the circles moving rigidly in depth together when we keep the retinal disparities between them constant, by compressing the physical distance between them as they move forwards in depth (as illustrated in Video 1).

The vast majority of observers pick the 'constant retinal disparity' condition as the one that moves rigidly.  And  you  can  try  the  'Linton  Stereo  Illusion'  for  yourself   in  Video  2  using  red-blue  stereo glasses. On the left is the 'constant physical separation' condition. On the right is the 'constant retinal disparity' condition. In which of  the two conditions do the circles appear to move rigidly?

[Insert Video 2 here: https://www.youtube.com/watch?v=18ONHln7mkI]

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00770v2-figure-8c3c0f01-6246-4b53-af70-fd98f5e72361.jpg)

Video 2. You can try the 'Linton Stereo Illusion' for yourself  with red-blue stereo glasses. 'Constant physical separation' condition (left) and 'constant retinal disparities' condition (right). Stimuli specified for a 27 inch display at a 40cm viewing distance.

In the 'constant physical separation' condition, the circles do not appear to move rigidly together. Instead, the near circle appears to move more than the far circle (Video 3).

[Insert Video 3 here: https://www.youtube.com/watch?v=jf3Q-I2fC8o]

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00770v2-figure-94a924af-77f1-4451-9ea4-243d5b4fdff8.jpg)

Video 3. Perceptual experience (right) in the 'constant physical separation' condition (left).

By contrast, in the 'constant retinal disparities' condition the circles do appear to move pretty much rigidly in depth together (Video 4).

[Insert Video 4 here: https://www.youtube.com/watch?v=\_\_3kRUrUS1o]

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00770v2-figure-1eeed502-ade7-4dc8-9e94-c27e820dd341.jpg)

Video 4. Perceptual experience (right) in the 'constant retinal disparities' condition (left).

The 'Linton  Stereo  Illusion'  was  presented  at  the  Applied  Vision  Association  (AVA)  Spring  2024 Meeting (Linton &amp; Kriegeskorte, 2024a) and European Conference on Visual Perception (ECVP) 2024 (Linton &amp; Kriegeskorte, 2024b), as well as at the Vision Sciences Society (VSS) 2024 Demo Night  and  the  ECVP  2024  Illusion  Night.  Roughly » 210  visitors  experienced  the  'Linton  Stereo Illusion'  across  the  two  conferences  ( » 160 at VSS, and » 150 at ECVP), with » 90% of observers picking the 'constant retinal disparities' condition as the rigid one . The 'Linton Stereo Illusion' has 1 also been widely discussed online, receiving 15,500 views on Twitter/X . 2

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00770v2-figure-dd475b03-a77c-486d-af96-33110e4eca12.jpg)

Figure  1.  'Linton  Stereo  Illusion'  at  VSS  Demo  Night  2024.  Here  being  viewed  by  Marty  Banks (University of  Berkeley) and Mike Landy (New York University). (Photos by Benjamin Peters).

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00770v2-figure-24b50edc-88d2-408f-a227-8d348e1f88a6.jpg)

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00770v2-figure-179320f1-4c09-4b44-a336-b817e9862576.jpg)

## What's the Significance of  the 'Linton Stereo Illusion'?

The  experience  of  the  'Linton  Stereo  Illusion'  is  undeniable.  The  question  then  turns  to  its significance. I will consider three potential explanations. First, my own explanation (that perceived stereo depth reflects retinal disparities rather than 3D geometry (Linton, 2023)(Linton, 2024a)), and then two alternative explanations that other vision scientists have suggested to me. Although I would estimate a significant majority of  vision scientists that I discussed the 'Linton Stereo Illusion' with at VSS and ECVP agreed with my interpretation, there were notable exceptions from the stereo vision community, and I cannot claim there was a clear consensus. Hopefully this article, along with the demos at VSS and ECVP, will spur an open, lively, and public exchange of ideas in print. And I very much look forward to - and indeed encourage - challenges to the explanation that I outline here.

That being said, the 'Linton Stereo Illusion'  is  a  clear  prediction  of   the  'Minimal'  (disparity-only) account of stereo vision advanced in (Linton, 2023). As (Popper, 1963) suggests, the predictive power of a theory has to seriously count in its favour. Indeed, the challenge facing alternative explanations

1 » 90% is higher than I expected, given roughly 1 in 8 people are thought to have some form of stereo vision deficits. Although when people mentioned they didn't see any depth, I didn't ask them to decide one way or another.

2 https://x.com/LintonVision/status/1788572440034947469 (as of the 8th October 2024).

is  significant.  Observers  in  the  'Linton  Stereo  Illusion'  act  as  if   they  have  direct  access  to  retinal disparities (I would argue, because they do). So alternative explanations of  the illusion have to explain why observers act as if  they have direct access to disparities, without admitting that they do.

Our field has benefitted immensely from the phenomenal exploration of illusions, from Wheatstone's original  papers  (Wheatstone,  1838)(Wheatstone,  1852),  to  (Helmholtz,  1866),  to  more  recent phenomena like the 'random-dot stereogram' (Julesz, 1960), the 'hollow-face illusion' (Gregory, 1970), and 'Adelson's chequerboard' (Adelson, 1995). No one would deny they have all had a profound effect on our field, even though they are simply phenomenal reports. And subsequent experimental study has only really served to parameterise the fundamental insights we already get from the illusions.

It  is  important to have this discussion now as a 'Phenomenal Report' for three reasons. First, the 'Linton Stereo Illusion' is already being widely discussed in the vision science community (at VSS, ECVP, and online ), and many vision scientists have asked me to fully describe the illusion and explain 3 my interpretation so they can better understand it. Second, I've been struck that when leading vision scientists have offered alternative explanations, they have been so divergent, and so it's important to explore  these  alternative  explanations  as  well.  Third,  this  dialogue  helps  to  clarify  what  actual experiments would be worth doing down the line in order to adjudicate between these competing interpretations. As you will see, each potential explanation rests on a number of  assumptions, and it would be good to clarify where they disagree in advance of  experimental investigation.

## 1. My Explanation: Perceived Stereo Depth Reflects Retinal Disparities

In the published abstracts of  the 'Linton Stereo Illusion' for the Applied Vision Association (AVA) Spring 2024 Meeting (Linton &amp; Kriegeskorte, 2024a) and European Conference on Visual Perception (ECVP) 2024 (Linton &amp; Kriegeskorte, 2024b) , I draw four specific conclusions: 4

## 1. 'First, perceived stereo depth appears to be best captured by the 'Minimal Model'.'

The 'Linton Stereo Illusion' is intended to adjudicate between traditional 'Triangulation' accounts of stereo vision and my own 'Minimal' (disparity-only) theory of  stereo vision (Linton, 2023).

You might argue that the two conditions only test 'full depth constancy' accounts like (Guan &amp; Banks, 2016)  vs.  'disparity-only'  accounts  (Linton,  2023),  but  leave  untouched  'partial  depth  constancy' accounts like (Johnston, 1991). In the next section, I will argue that this is a mistake. I argue that no plausible 'Triangulation' account predicts the pattern of  distortions that we experience in the 'constant physical separation' condition of  the 'Linton Stereo Illusion'.

- 2. 'Second, doubling disparity appears to double perceived depth, suggesting that perceived stereo depth is proportional to disparity.'

The point being that, if  the stimulus in 'constant retinal disparity' condition is perceived as roughly rigid, then the 33-40cm separation and the 40-50cm separation, which have the same disparity, must also be perceived as having the same depth, since the motion is rigid (there is no change in depth).

3 https://x.com/LintonVision/status/1788572440034947469 (as of the 8th October 2024).

4 Kriegeskorte (mentor) remains agnostic as to the different interpretations of  the 'Linton Stereo Illusion'.

But if  this is true, then 33-50cm (which is both separations added together, and therefore double the disparity) must have double the depth of  either of  them in isolation.

- 3. 'Third, changes in vergence appear to have no effect on perceived depth.'

If  vergence tracks the 'constant retinal disparity' stimulus as it moves in depth, but the circles appear rigid throughout, then it becomes difficult to argue, as (Banks, 2000) does, that:

'We have shown … that changes in the eyes' vergence can cause a compelling change in perceived shape even when the retinal disparities are completely constant.'

Instead, following my work on vergence as a distance (Linton, 2020) and size cue (Linton, 2021a), a new position is emerging in the literature. (Rogers, 2023) cites my work, and concludes that it:

- '…it raises the question of  whether there is any role for the vergence angle of  the eyes in the perception of  spatial layout in natural scenes (Linton, 2021a).'

Our experience of the 'constant retinal disparity' condition when vergence tracks the stimulus appears to provide additional evidence in favour of  this conclusion. Indeed, this is something that those who favour a (Johnston, 1991) interpretation of  the 'Linton Stereo Illusion' (below) largely agree with me on, since her account predicts only a small effect of  vergence on the scaling of  disparities.

- 4. 'Fourth, stereo 'depth constancy' appears to be a cognitive (not perceptual) phenomenon, reflecting our experience of a world distorted in perceived stereo depth.'

This argument is developed in the third section (below), and best articulated there.

- 5.  In  my  ECVP  poster  (https://osf.io/6wn7x),  I  further  argued  that  the  'Linton  Stereo  Illusion' highlights is that we've been trying to hold two incompatible commitments at the same time:
- a. On the one hand, the logic of  stereo 'depth constancy' (even partial 'depth constancy') implies that observers are blind to the changes in the relative disparities of  stimuli moving in depth.
- b. On the other hand, if  there's one thing we've know about stereo vision for well over a century (Howard, 1919), it's that observers are supremely sensitive to changes in relative disparities.

For me, sensitivity to changes in the relative disparities seems to be more fundamental of  the two. So, it's no surprise to me that the 'Linton Stereo Illusion' appears to demonstrate that observers are not blind to changes in relative disparities in the way that 'depth constancy' suggests.

However, against my interpretation, two alternative explanations have been raised:

## 2. Alternative Explanation One: Failure of  Depth Constancy?

A leading vision scientist has suggested to me  that the 'Linton Stereo Illusion' is just an instance of  a 5 well-known phenomenon - the failure of stereo 'depth constancy' - and that the 'Linton  Stereo Illusion'  can  be  explained  by  the  analysis  in  the  leading  paper  in  the  field ,  (Johnston,  1991).  By 6 contrast, in this Section, I argue that (Johnston, 1991)'s account makes predictions that are not only inconsistent with, but the exact opposite, of  what is observed in the 'Linton Stereo Illusion' . 7

Failure of  stereo 'depth constancy' is a claim that dates back at least to (Helmholtz, 1866), who argued that distortions in stereo vision are due to visual system scaling disparities using a faulty estimate of the viewing distance (which is typically thought to come from vergence):

'Owing to the uncertainty of our judgments as to the degree convergence of the eyes, we are liable to have illusions also about the forms of  things in space seen binocularly. The  interpretation  of   the  visual  phenomena  would  be  correct  if   the  amount  of convergence were different, but it is not correct for the convergence actually used.' (Helmholtz, 1866), pp.264-5 (trans. p.318).

For  a  representative  sample  of   restatements  of   this  over  the  last  seven  decades  see  (Wallach  &amp; Zuckerman, 1963) in the 1960s, (Ono &amp; Comerford, 1977) in the 1970s, (Foley, 1980) in the 1980s, (Johnston, 1991) in the 1990s, (Scarfe &amp; Hibbard, 2006) in the 2000s, (Volcic et al., 2013) in the 2010s, and (Yu et al., 2021) in the 2020s. It's fair to say (Johnston, 1991) has been the leading paper in the field for the last 33 years, and so it's understandable that this vision scientist focuses on it.

(Johnston, 1991) had participants adjust a stereo-defined cylinder so that it looked regular (dotted line in  Figure  2).  At  medium  distances  (107cm),  (Johnston,  1991)  found  close  to  veridical  3D  shape perception.  However,  at  close  distances  (53.5cm),  a  compressed  cylinder  looked  regular,  implying 'over-constancy'  (accentuated  depth)  at  near  distances.  Whilst  at  far  distances  (214cm),  a  cylinder accentuated in depth looked regular, implying 'under-constancy' (compressed depth) at far distances.

Figure 2: Disparity defined 3D shape of  cylinders perceived to be 'regular' (dotted line) by observers in (Johnston, 1991) at near (53.5cm) and far (214cm) viewing distances.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00770v2-figure-2b0a0f8f-1f56-46ea-a258-25a700daed1c.jpg)

5 Personal communication, 'There is no challenge to our understanding of stereo vision: Response to Linton and Kriegeskorte (ECVP 2024 and ArXiv, https://arxiv.org/abs/2408.00770)', 9th September 2024.

6 As evidenced by most citations for an article on failure of  stereo depth constancy (359 at time of  writing).

7 Since a I don't want to turn this paper on the 'Linton Stereo Illusion' into a paper on (Johnston, 1991), I have provided an extended version of the following analysis in (Linton, 2024b).

(Johnston,  1991)  concluded  that  binocular  disparities  are  'scaled'  by  the  visual  system  using  an inaccurate estimate ( ) of  the viewing distance ( y x ), where:

$$\text{Weak Triangulation} \stackrel { y } { \sim } \frac { 8 0 } { \sim } + \frac { 0. 2 6 5 ^ { * } ( x - 8 0 ) } { \sim } \text{Strong Triangulation}$$

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00770v2-figure-a9ecdac2-e1b3-4064-80ee-c828339bf3cb.jpg)

We can think of this formula as having two components : 8

- 1. Abathic  Distance  (Weak  Triangulation) :  A  default  internal  estimate  of   viewing  distance, 9 known as the 'abathic distance', that is roughly 80cm (where y = x ).
- 2. Vergence  (Strong  Triangulation):  A  refinement  of   the  'abathic  distance'  based  on  a compressed (gain of 0.265) estimate of x , thought to come from 'vergence'.

## a. 'Constant Physical Separation' Condition

You might think that (Johnston, 1991) provides an obvious explanation for the perceptual experience in the 'constant physical separation' condition of  the 'Linton Stereo Illusion' (Video 3), where we experience  accentuated  depth  as  the  stimulus  moves  towards  us.  Indeed,  20  years  ago,  (Scarfe  &amp; Hibbard, 2006) tested one of (Johnston, 1991)'s cylinders moving in depth, and found the stereodefined cylinder became noticeably accentuated depth as it moved closer, noting:

'A simple prediction from [(Johnston, 1991)] is that disparity-defined objects should appear to expand in depth when moving towards the observer, and compress when moving away.'

But the same stereo distortions are also predicted by my account (Linton, 2023), since the disparities are larger the closer the object is. So how can we differentiate between these two accounts? As we see in part c. (below) you can actually do it using (Johnston, 1991)'s original cylinder. But that's not the approach the 'Linton Stereo Illusion' adopts. Instead, it introduces two key innovations:

First, differentiating the two accounts is easier when we focus on more complex stimuli. Specifically, if  we have two sequential separations in depth (the red and blue arrows in Figure 3), we can explore how 'distance scaling' affects the perceived relative size of  these two separations in depth.

8 I  call  these  'Weak  Triangulation' and 'Strong Triangulation' because both are classic triangulation accounts, the only question is whether they use a default internal estimate of  the viewing distance or try to actively estimate it.

9 This is 'Triangulation' without 'depth constancy'.

Figure 3. Viewing geometry in the 'constant physical separation' condition.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00770v2-figure-9b5bafe0-391a-4206-b06e-d2651788ad98.jpg)

Second, using a scaling distance that is too far away will lead to a disproportionate scaling of  the red (far) separation over the blue (near) separation. So consider what happens when these disparities, that are produced by the 'constant physical separation' condition when vergence distance is fixed on the surface of  the display (40cm), are rescaled using (Johnston, 1991)'s estimate of  the viewing distance (70cm):

Figure 4. Disparities for the 'constant physical separation' condition of  the 'Linton Stereo Illusion' replotted for different scaling distances (with vergence fixed at 40cm).

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00770v2-figure-e41903f6-297a-4db4-a69c-cdcf98ef8c9f.jpg)

When we look at the predictions of  (Johnston, 1991)'s account for a scaling distance of  70cm in Figure 4, what we immediately notice is that the perceived stereo depth for the stimulus should be larger behind the display (red arrow) than it is in front of  the display (blue arrow). So as the stimulus moves behind the display it should expand (red arrow), and as the stimulus moves in front of  the display it

should contract (blue arrow). But this is the exact opposite of  what we actually observe in the 'constant physical separation' condition of  the 'Linton Stereo Illusion'.

[Insert Video 5 here: https://youtu.be/7chGl8JtPVE]

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00770v2-figure-92b7f10e-150f-47c1-9cdb-0a6a10b71e41.jpg)

Video 5. Perceived stereo distortions predicted by (Johnston, 1991) and (Linton, 2023) for the 'constant physical separation' condition of  the 'Linton Stereo Illusion' (vergence fixed at 40cm).

You might object that vergence was not fixed in my ECVP and VSS demonstrations. That is true. But you can test the illusion for yourself  using the anaglyph version. Set the screen at 40cm, fix your vergence on the screen, and watch the illusion. The separation between the circles appears to expand as it gets closer (as predicted by my account) not compress (as predicted by (Johnston, 1991)).

You might also object that (Johnston, 1991) just gets the 'scaling' ('abathic') distance wrong. But here's the dilemma for 'Triangulation' accounts. If  you bring the 'scaling' distance closer - to, say, 40cm then all you would predict is that the 'constant physical separation' condition should look undistorted as it moves in depth. To get the actual pattern of  distortions predicted by my account in Figure 4, and actually observed in the 'constant physical separation' condition, the 'scaling' distance would have to be much closer than 40cm. But arguing that stereo vision is 'optimized' for a viewing distance much closer than 40cm is hard to maintain, and the closest 'scaling' ('abathic') distances I have seen in the literature are 45cm in (Volcic et al., 2013) and 50cm in (Scarfe &amp; Hibbard, 2006).

Further objections are discussed in (Linton, 2024b). Consider, for instance, the massive perceived absolute distances predicted by (Johnston, 1991)'s account for the two 10cm separations. It predicts the near 10cm separation should be perceived as 26cm, and the further 10cm separation should be perceived as 37cm, neither of  which actually correspond to our experience of  the illusion.

## b. 'Constant Retinal Disparity' Condition

Next, let's model the 'constant retinal disparity' condition using (Johnston, 1991). The vision scientist argues that if  vergence is tracking the stimulus, then (Johnston, 1991) would predict something close to the rigid percept we see in the 'constant retinal disparity' condition. So, let's model (Johnston, 1991)'s prediction with vergence tracking the front circle from 40cm to 33cm, as they suggest:

Figure 5. (Johnston, 1991)'s prediction of  how the perceived stereo depth between the two circles should change as vergence tracks the near circle from 30cm to 40cm.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00770v2-figure-adaa661b-3eb6-440f-a08a-c21e82c3b8bf.jpg)

First, as we see in Figure 5, (Johnston, 1991)'s account doesn't predict a rigid separation between the two circles, but a compression in depth of  2.3cm. Since the physical compression in depth required to keep the disparities constant is 3.3cm, this means that our percept in the 'constant retinal disparity' should reflect most of  the physical compression of  the stimulus, but it doesn't appear to.

Second, you might look at Figure 5, and think the lines are roughly parallel, and therefore predicting something close to my account. But that's only because the 2.3cm compression in Figure 5 is masked by the astoundingly large absolute distances that (Johnston, 1991)'s account predicts. According to (Johnston, 1991), the 6.7cm-10cm separation should be perceived as 34.4cm-36.7cm. Not only does this not cohere with our experience of  perceived depth between the two circles in the  'constant retinal disparity' condition, it poses an impossible challenge for (Johnston, 1991): At what distance do we see the display that's 40cm away? If  we say 40cm, then (Johnston, 1991) implausibly entails the stereo percept is 5cm from our face. But if  we say the scaling distance (70cm), (Johnston, 1991) now struggles to explain how we are able to reach for the 'on' button of  the display without overshooting by 30cm.

## c. Single Depth Separation Stimulus

Finally,  you  should  come  to  the  same  conclusion  even  if   you  consider  a  single  depth  separation stimulus like (Johnston, 1991)'s cylinder (Figure 2). Ironically, the one thing that full 'depth constancy' 'Triangulation' accounts (Guan &amp; Banks, 2016) and my 'Minimal' (disparity-only) model (Linton, 2023) have  in  common,  is  that  our  stereo  depth  percept  is  largely 10 invariant  to  eye  movements,  either because there is no re-scaling of  disparities with each eye movement (my account), or because the rescaling of  disparities with each eye movement is veridical (Guan &amp; Banks, 2016). By contrast, on (Johnston, 1991)'s account, the rescaling of  disparities with each eye movement is not veridical, leading to the prediction of  massive distortions of  stereo depth as we looks around an object or scene.

The key problem is that the same disparity can be categorised as 'nearer than fixation' or 'further than fixation' depending on where we fixate. But if  our 'scaling' distance is » 80cm no matter where we

10 There will be some very subtle changes to the disparities on the retina with vergence eye movements ('gaze contingent disparities': (Turski, 2016), (Linton, 2019)) due to the slight offset of  the nodal point and the centre of  rotation of  the eye. But the key point is that retinal disparities are not being re-scaled as having different perceived depths.

fixate, then this is going to lead to massive stereo distortions as we fixate on the front or back of  an object. To convey the intuition, if  we have a large disparity that the visual system (falsely) thinks is coming from » 80cm, that disparity is going to correspond to a much larger distance behind fixation (&gt;80cm) than if the disparity corresponded to a distance in front of  fixation (&lt;80cm).

Let's apply this analysis to the 'Linton Stereo Illusion' at time t = 0 where (in both conditions) we have a back circle at 50cm and a front circle at 40cm. On (Johnston, 1991)'s account, this 10cm separation is going to be differently scaled depending on whether we're fixating near (40cm) or far (50cm). So, on (Johnston, 1991)'s account the 10cm separation should be perceived as 37cm in depth when we fixate near (40cm), but only 19cm when we fixate far (50cm). Put simply, the apparent stereo depth of the separation should expand by » 50% as we shift our fixation from far-to-near.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00770v2-figure-ef506b41-d0af-4b69-88f0-77d01775b992.jpg)

Figure 6. Illustration of  how the perception of  the very same physical distance distorts with near and far fixation under (Johnston, 1991)'s account. Note that the absolute size of  the percepts are not to scale. As I emphasise, (Johnston, 1991)'s account predicts implausibly large absolute distances. The key point being conveyed here is that (Johnston, 1991)'s account predicts that the stereo depth percept should expand by » 50% as we shift our fixation from far (50cm) to near (40cm).

## Alternative Explanation Two: Artefact of  Constant Angular Size?

A second objection that has been raised against the 'Linton Stereo Illusion' is that it's just an artefact of keeping angular size constant. First, we don't notice objects distorting in stereo depth in normal viewing conditions. Second, if  we add an angular size ('looming') cue to the 'Linton Stereo Illusion', now the 'constant physical separation' condition does seem to move rigidly in depth (Video 6, right):

[Insert Video 6 here: https://www.youtube.com/watch?v=QF1IjuveqqE]

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00770v2-figure-c7cb9eb0-e8e4-4791-8255-668a872645ae.jpg)

Video 6. 'Constant physical separation' condition of  the 'Linton Stereo Illusion' with angular size kept fixed (left) or veridically changing with distance (right). In the middle are three static stereo targets staggered at 30cm (in front of  screen), 40cm (on screen), and 50cm (behind screen).

By in large, I agree with this assessment. But I wouldn't count it as a criticism. Instead, I think it shows that the 'Linton Stereo Illusion' tells us something valuable about (a) stereo vision, (b) cue integration, and (c) 'depth constancy', even if  we don't notice its effects in everyday viewing conditions.

## a. Stereo Vision

When testing whether perceived stereo depth reflects retinal disparity or 3D geometry, it makes sense to control for angular size to make sure we are testing stereo vision alone, and not cue integration. But one objection that has been raised with me is that keeping angular size fixed isn't acting as a 'control' for angular size, so much as a cue against motion-in-depth, and so what it introduces is a 'cue-conflict' that (on traditional cue integration accounts) leads to a reduction in perceived stereo depth.

There are three responses to this objection:

First, we can test this and show that it's not true. The obvious test of  whether keeping angular size fixed leads to a reduction in perceived stereo motion-in-depth, is to place static stereo targets at the distances the motion should hit, and see if  the stimulus in fact hits these targets or falls short. Check the motion of the fixed-angular size stimulus in Video 6 (left), and you see it doesn't fall short of  the 30cm or 50cm static stereo targets in the middle of the display. Indeed, the original version of  the 'Linton Stereo Illusion' 11 had static targets at 30cm, 40cm, and 50cm, to address this very concern.

11 Version submitted to VSS Demo Night (17th March 2024): https://www.youtube.com/watch?v=El7YKaURsuQ

[Insert Video 7 here: https://youtu.be/z9V4waTmae8]

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00770v2-figure-b2c8c2da-94bc-4cdd-a8e8-44a53a47aafd.jpg)

Video 7. Original version of  the 'Linton Stereo Illusion' submitted to VSS Demo Night.

It's interesting that the stereo vision scientists who have objected to the illusion don't typically object on fixed-angular size grounds. Instead, they tend to agree with me that the illusion is a good test of stereo vision, but insist that it is fully explained by (Johnston, 1991).

Second, the 'Linton Stereo Illusion' appears to still work even if  don't control for angular size, so long as we use dots as the stimulus. This seems like an obvious solution, since the dots' angular size doesn't change much with distance. But the problem here is diplopia. I have a convincing demo in the lab the motion of the dots in front of  the screen looks much closer to the motion of  the dots behind the screen in the 'constant retinal disparity' condition than in the 'constant physical separation' condition - but I fully admit that the anaglyph version 12 does not fully capture this.

Third, the challenge for this 'cue conflict' explantion is how it accounts for our experience in the 'constant retinal disparity' condition (Video 2, right). In the 'constant retinal disparity' condition, a stimulus with constant retinal disparity is seen as moving rigidly in depth. On my account, this is because  constant  disparities  are  perceived  as  having  constant  stereo  depth.  But  the  'cue  conflict' objection can't appeal to this explanation. Indeed, it's the very thing they're rejecting. Otherwise, they wouldn't be objecting to my account. So instead, they must claim two things: First, that constant disparities are not perceived as having constant stereo depth for a stimulus moving in depth. Second, and this is the difficult part, the 'cue conflict' of  keeping angular size fixed just so happens to cancel out the change in perceived stereo depth predicted by their account by exactly the right amount so that constant disparities are perceived as having constant stereo depth. But why? On their account, this is entirely a coincidence, and nothing to do with constant disparities having constant stereo depth.

12 https://www.youtube.com/watch?v=uKbmoz5PGl8 Using dots to control for angular size. 'Constant physical separation' condition (left) and 'constant retinal disparity' condition (right). In which condition does the motion of  the dots in front of  the screen seem best matched to the motion of the dots behind the screen?

The  problem  is  that  our  experience  of   the  'constant  retinal  disparity'  condition  is  exactly  as  my 'Minimal' (disparity-only) account predicts. So, advocates of  this alternative explanation now have to explain why their account mimics my account, whilst also explicitly rejecting my account.

This discussion of stereo vision should be enough to convince you that the 'Linton Stereo Illusion' can't be written off  simply as an artefact of  keeping angular size constant. In part, because our leading theories of  stereo vision struggle to explain why such an 'artefact' should arise in the first place.

In the remainder of the paper, I also consider implications for 'cue integration' and 'depth constancy' that appear to arise from the 'Linton Stereo Illusion', but these are less essential to the core message.

## b. Cue Integration

It's typically assumed that 3D vision integrates the various different depth cues into a single coherent percept at the level of  our visual experience ('mandatory fusion')(Hillis et al., 2002). In which case, one tempting conclusion about the 'Linton Stereo Illusion' might be that whilst it teaches us something important about stereo vision, it doesn't really teach us anything interesting about 3D vision in normal viewing conditions. On this account, multiple depth cues bring us closer to the veridical percept, and the depth distortions inherent in stereo vision - whether they are (Johnston, 1991)'s or (Linton, 2023)'s - disappear (Johnston et al., 1994)(Scarfe &amp; Hibbard, 2013). This explains why, when we add changes in angular size to the 'constant physical separation' condition, the distortions suddenly disappear.

On  this  account,  then,  adding  the  angular  size  cue  leads  to  a  different  3D  percept.  Whilst  this interpretation  of   the  'Linton  Stereo  Illusion'  might  be  tempting,  I  don't  believe  it  is  correct.  The problem is that it predicts that the perceived stereo depth (i.e. the perceived real-world depth) of  the stimulus is changed by the addition of  the angular size cue. But this doesn't appear to be the case. The obvious test of  this, as we saw above, is to assess the perceived real-world motion-in-depth of  the changing-angular size stimulus against static disparity targets at 30cm, 40cm, and 50cm. Its perceived real-world motion-in-depth is completely circumscribed by the stereo targets. But so is the perceived real-world motion-in-depth of the fixed-angular size stimulus, leading to the conclusion that they both have the same perceived real-world motion-in-depth, even if  we can't directly judge this to be the case.

Instead, I treat the 'looming' (change in angular size) cue merely as a 'depicted' (pictorial) depth cue. For instance, when an object 'looms' (comes towards you) on TV , it doesn't come out of the screen in perceived real-world depth. This is because the perceived real-world depth in this scenario is dictated by the stereo disparities. And I would argue that the same principle applies here, too.

I agree that our judgements of  their perceived real-world motion-in-depth of  the two stimuli in Video 6 are different. And the challenge for my account is to explain this. But it's also a challenge for 'cue integration' accounts, because if  it were simply a case of  the two stimuli have different perceived realworld motion-in-depth, the static disparity targets couldn't be a good match for both stimuli.

My argument is that whilst we (cognitively) mistake 'depicted' motion-in-depth (from the 'looming cue')  for  perceived  real-world  motion-in-depth, our visual systems do not, and our visual systems continue to produce perceived real-world depth on the basis of stereo alone. In this case 'depicted' motion-in-depth is a 'cognitive bias' that we can't directly overcome when evaluating our own visual experience. And we can only overcome this failure of  introspection through 'structured introspection':

'scaffolding' visual space with static stereo targets to help us judge which depth cues are contributing to perceived real-world depth, and which ones are merely contributing to 'depicted' depth.

This requires a strong modularity thesis between 'perception' and 'cognition' ((Linton, 2017), Ch2; (Linton, 2023)): The idea that whilst we are (cognitively) fooled, our visual systems are not. Strong modularity is not necessarily an uncommon approach in the literature, but usually runs in the opposite direction: The idea that whilst we are not (cognitively) fooled, our visual systems are (for instance, by the 'Müller-Lyer illusion'). And, as (Block, 2023) notes, there is a lively debate where to draw the line:

'My usage of the terms 'cognition' and 'perception' is consonant with much of the recent literature on perception ((Firestone &amp; Scholl, 2016); (Pylyshyn, 1999)), but some restrict the term 'perception' to what I am calling low-level perception (Linton, 2017) and others  use  'cognition'  to  encompass  mid-level  and  high-level  perception  as  both perception and cognition (Cavanagh, 2011).'

(Block, 2023), p.13.

The  point  of   this  discussion  isn't  to  argue  the  'Linton  Stereo  Illusion'  in  Video  6  vindicates  my account, only to highlight the dilemma it appears to pose for traditional 'cue-integration' accounts: Why are both stimuli perceived to move veridically relative to the static stereo targets if  the difference in their judged motion-in-depth is attributed to differences in their perceived motion-in-depth?

## c. Depth Constancy

On my account, we experience a world distorted in perceived stereo depth, as predicted by the falloff   of   disparities  with  distance  squared,  but  we  typically  don't  notice  them  in  everyday  viewing conditions. One of the reasons is that we get a significant degree of 'depth constancy' for free in natural viewing conditions because of  perspective (changes in angular size with distance). Not because changes in angular size 'correct' distortions in stereo depth (the thesis I have been arguing against), but simply because they make them much less noticeable. To explain, 'depth constancy' focuses on distortions of  z-axis points with distance. And, since disparities fall off  with the viewing distance squared, you would expect to experience visual space as distorted in the same way. But what people are really concerned with is whether 3D objects look distorted with distance ('shape constancy'). Since x-axis size and y-axis size fall off  linearly with distance, full perceptual depth constancy would look weird: a 3D object would appear to expand in z-axis depth (relative to its x-axis and y-axis lengths) as it recedes. Conversely, the linear fall-off  of  angular size with distance makes the fall-off  of  disparities with distance squared appear to be merely a linear fall-off  of  stereo depth. And this is much closer to our actual everyday visual experience, where objects do seem flatter in stereo depth with distance.

## Acknowledgements

This research was conducted in Nikolaus Kriegeskorte's Visual Inference Lab at Columbia University's Zuckerman Mind Brain Behavior Institute, with support from the NOMIS Foundation (Grant 'New Approach to 3D Vision' to PL at the Italian Academy for Advanced Studies, Columbia University), from the Presidential Scholars in Society and Neuroscience (PSSN), Columbia University, and from the Italian Academy for Advanced Studies, Columbia University. I want to thank Prof. Kriegeskorte for  his  mentorship,  as  well  as  David  Freedberg  (Italian  Academy),  Christopher  Peacocke  (PSSN), Pamela Smith (PSSN), and Carol Mason (PSSN). I also want to thank audiences at Rochester Institute of Technology (Imaging Science and Cognitive Science) and the Center for Visual Science, University

of  Rochester  (February  2024),  Applied  Vision  Association  Spring  Meeting  (March  2024),  Royal Holloway, University of  London (March 2024), Vision Sciences Society Demo Night (May 2024), and European Conference on Visual Perception (August 2024) and ECVP Illusion Night (August 2024) where this work was presented, as well as the lively discussion on X.com (Twitter) with 15,500 views: https://x.com/LintonVision/status/1788572440034947469

In particular, I want to thank (in alphabetical order) Rob Allison, Marty Banks, Patrick Cavanagh, Aniruddha Das, Deborah Giaschi, Paul Hibbard, Mike Landy, Ken Nakayama, Andrew Parker, Jenny Read, Mike Shadlen, Shin Shimojo, Christopher Tyler, Laurie Wilcox, and Li Zhaoping, for helpful discussions at VSS, ECVP, and Columbia.

## References

Adelson, T. E. (1995). Checker shadow illusion: Http://persci.mit.edu/gallery/checkershadow .

Banks, M. (2000). Viewing Geometry and Stereoscopic Vision. NSF Award Abstract #9983387 .

https://www.nsf.gov/awardsearch/showAward?AWD\_ID=9983387

Block, N. (2023). The Border Between Seeing and Thinking . Oxford University Press.

Bruner, J. S., &amp; Goodman, C. C. (1947). Value and need as organizing factors in perception. The

Journal of  Abnormal and Social Psychology , 42 (1), 33-44. https://doi.org/10.1037/h0058484

Cavanagh, P. (2011). Visual cognition. Vision Research , 51

(13), 1538-1551.

https://doi.org/10.1016/j.visres.2011.01.015

Descartes, R. (1637). Dioptrique (Optics). In J. Cottingham, R. Stoothoff, &amp; D. Murdoch (Eds.), The

Philosophical Writings of  Descartes: Volume 1 (1985) . Cambridge University Press.

Firestone, C., &amp; Scholl, B. J. (2016). Cognition does not affect perception: Evaluating the evidence for 'top-down' effects. The Behavioral and Brain Sciences , 39

, e229.

https://doi.org/10.1017/S0140525X15000965

Foley, J. (1980). Binocular Distance Perception. Psychological Review , 87 , 411-434.

https://doi.org/10.1037//0033-295X.87.5.411

Gregory, R. L. (1970). The intelligent eye . Weidenfeld and Nicolson.

Guan, P., &amp; Banks, M. S. (2016). Stereoscopic depth constancy. Phil. Trans. R. Soc. B , 371 (1697),

20150253. https://doi.org/10.1098/rstb.2015.0253

Helmholtz, H. (1866). Handbuch der Physiologischen Optik, Vol.III (translated by J. P. C. Southall 1925 Opt. Soc. Am. Section 26, reprinted New York: Dover, 1962). Hillis, J. M., Ernst, M. O., Banks, M. S., &amp; Landy, M. S. (2002). Combining Sensory Information: Mandatory Fusion Within, but Not Between, Senses. Science , 298 (5598), 1627-1630. https://doi.org/10.1126/science.1075396 Howard, H. J. (1919). A Test for the Judgment of Distance. Transactions of  the American Ophthalmological Society , 17 , 195-235. Johnston, E. B. (1991). Systematic distortions of  shape from stereopsis. Vision Research , 31 (7), 13511360. https://doi.org/10.1016/0042-6989(91)90056-B Johnston, E. B., Cumming, B. G., &amp; Landy, M. S. (1994). Integration of  stereopsis and motion shape cues. Vision Research , 34 (17), 2259-2275. https://doi.org/10.1016/0042-6989(94)90106-6 Julesz, B. (1960). Binocular depth perception of  computer-generated patterns. Bell System Technical Journal , 39 , 1125-1162. https://doi.org/10.1002/j.1538-7305.1960.tb03954.x Kepler, J. (1604). Paralipomena to Witelo (W . H. Donahue, Trans.). In Optics: Paralipomena to Witelo and Optical Part of  Astronomy. Green Lion Press, 2000. Kerr, N. L. (1998). HARKing: Hypothesizing After the Results are Known. Personality and Social Psychology Review , 2 (3), 196-217. https://doi.org/10.1207/s15327957pspr0203\_4 Linton, P. (2017). The Perception and Cognition of  Visual Space . Palgrave Macmillan. Linton, P. (2019). Would Gaze-Contingent Rendering Improve Depth Perception in Virtual and Augmented Reality? https://arxiv.org/abs/1905.10366v1 Linton, P. (2020). Does vision extract absolute distance from vergence? Attention, Perception, &amp; Psychophysics , 82 (6), 3176-3195. https://doi.org/10.3758/s13414-020-02006-1 Linton, P. (2021a). Does Vergence Affect Perceived Size? Vision , 5 (3), Article 3. https://doi.org/10.3390/vision5030033

Linton, P. (2021b). V1 as an egocentric cognitive map. Neuroscience of  Consciousness , 7 (2), 1-19. https://doi.org/10.1093/nc/niab017 Linton, P. (2023). Minimal theory of  3D vision: New approach to visual scale and visual shape. Philosophical Transactions of  the Royal Society B: Biological Sciences , 378 (1869), 20210455. https://doi.org/10.1098/rstb.2021.0455 Linton, P. (2024a). Linton Stereo Illusion (arXiv:2408.00770). arXiv. https://doi.org/10.48550/arXiv.2408.00770 Linton, P. (2024b). Linton Stereo Illusion: Response on Johnston (1991) . OSF. https://doi.org/10.31234/osf.io/8njd5 Linton, P., &amp; Kriegeskorte, N. (2024a). Perceived Stereo Depth reflects Retinal Disparities, not 3D Geometry . OSF. https://doi.org/10.31234/osf.io/pc8kh Linton, P., &amp; Kriegeskorte, N. (2024b). Perceived Stereo Depth reflects Retinal Disparities, not 3D Geometry . OSF. https://doi.org/10.31234/osf.io/7z239 Ono, H., &amp; Comerford, J. (1977). Stereoscopic depth constancy. In W . Epstein (Ed.), Stability and constancy in visual perception: Mechanisms and process . Wiley. Popper, K. R. (1963). Conjectures and Refutations: The Growth of  Scientific Knowledge . Routledge &amp; K. Paul. Pylyshyn, Z. (1999). Is vision continuous with cognition?: The case for cognitive impenetrability of visual perception. Behavioral and Brain Sciences , 22 (3), 341-365. https://doi.org/10.1017/S0140525X99002022 Rogers, B. (2023). When is a disparity not a disparity? Toward an old theory of  three-dimensional vision. I-Perception , 14 (5), 20416695231202726.

https://doi.org/10.1177/20416695231202726

Scarfe, P ., &amp; Hibbard, P . B. (2006). Disparity-defined objects moving in depth do not elicit threedimensional shape constancy. Vision Research , 46 (10), 1599-1610.

https://doi.org/10.1016/j.visres.2005.11.002

Scarfe, P ., &amp; Hibbard, P . B. (2013). Reverse correlation reveals how observers sample visual information when estimating three-dimensional shape. Vision Research , 86 , 115-127. https://doi.org/10.1016/j.visres.2013.04.016

Turski, J. (2016). On binocular vision: The geometric horopter and Cyclopean eye. Vision Research , 119 , 73-81. https://doi.org/10.1016/j.visres.2015.11.001

Volcic, R., Fantoni, C., Caudek, C., Assad, J. A., &amp; Domini, F. (2013). Visuomotor Adaptation Changes Stereoscopic Depth Perception and Tactile Discrimination. Journal of  Neuroscience

, 33 (43), 17081-17088. https://doi.org/10.1523/JNEUROSCI.2936-13.2013

Wallach, H., &amp; Zuckerman, C. (1963). The constancy of stereoscopic depth. The American Journal of Psychology , 76 , 404-412.

Wheatstone, C. (1838). Contributions to the physiology of  vision. -Part the first. On some remarkable, and hitherto unobserved, phenomena of binocular vision. Philosophical Transactions of  the Royal Society of  London , 128 , 371-394.

https://doi.org/10.1098/rstl.1838.0019

Wheatstone, C. (1852). The Bakerian Lecture.-Contributions to the physiology of vision.-Part the second. On some remarkable, and hitherto unobserved, phenomena of binocular vision (continued). Philosophical Transactions of  the Royal Society of  London , 142 , 1-17.

https://doi.org/10.1098/rstl.1852.0001

Yu, Y., Todd, J. T., &amp; Petrov, A. A. (2021). Failures of  stereoscopic shape constancy over changes of viewing distance and size for bilaterally symmetric polyhedra. Journal of  Vision , 21 (6), 5. https://doi.org/10.1167/jov.21.6.5
Paul Linton
Linton Stereo Illusion
We present a new illusion that challenges our understanding of stereo vision.
The illusion consists of a larger circle at 50cm, and smaller circle in front
of it at 40cm, with constant angular sizes throughout. We move the larger
circle forward by 10cm (to 40cm) and then back again (to 50cm). The question
is, what distance should we move the smaller circle forward and back to
maintain a constant perceived separation in depth between the circles? Constant
physical distance (10cm) or constant retinal disparity (6.7cm)? Observers
choose constant disparity. We therefore argue the 'Linton Stereo Illusion'
appears to suggest that perceived stereo depth reflects retinal disparities
rather than 3D geometry.