Optimal integration of visual speed across different spatiotemporal frequency channels
How do humans perceive the speed of a coherent motion stimulus that contains motion energy in multiple spatiotemporal frequency bands? Here we tested the idea that perceived speed is the result of an integration process that optimally combines speed information across independent spatiotemporal frequency channels. We formalized this hypothesis with a Bayesian observer model that combines the likelihood functions provided by the individual channel responses (cues). We experimentally validated the model with a 2AFC speed discrimination experiment that measured subjects' perceived speed of drifting sinusoidal gratings with different contrasts and spatial frequencies, and of various combinations of these single gratings. We found that the perceived speeds of the combined stimuli are independent of the relative phase of the underlying grating components. The results also show that the discrimination thresholds are smaller for the combined stimuli than for the individual grating components, supporting the cue combination hypothesis. The proposed Bayesian model fits the data well, accounting for the full psychometric functions of both simple and combined stimuli. Fits are improved if we assume that the channel responses are subject to divisive normalization. Our results provide an important step toward a more complete model of visual motion perception that can predict perceived speeds for coherent motion stimuli of arbitrary spatial structure.
COMMENTS BASED ON REVIEWER DISCUSSIONS AND AUTHOR 
      REBUTTAL: I agree with the other reviewers that more could be done 
      to constrain the specifics of the cue integration mechanism. However, I 
      believe that if the data set is expanded, allowing the models to be better 
      constrained, then the paper is appropriate and interesting for the NIPS 
      community. I have left my quality score as it was, but I agree 
      with the other reviewers that the paper merits a ``1'' rather than a ``2'' 
      for impact score. ORIGINAL REVIEW: Summary: This paper 
      extends an existing model for the perception of visual speed that uses a 
      Bayesian observer model acting on the activity of independent 
      spatiotemporal frequency channels. Previously, the model accounted for 
      illusions of perceived speed by postulating the Bayes-optimal combination 
      of noisy sensory representations with a prior for slow speeds. Here this 
      model is extended to operate on spatiotemporal filter outputs, allowing 
      the model to account for changes in perceived speed as a function of 
      spatial frequency. The revised model is shown to provide a good fit to 
      human speed discrimination data. Strengths: The model is 
      elegantly simple and provides a good fit to data. The model is a useful 
      extension of previous work, providing a more general and 
      neurophysiologically plausible version. The behavioral experiment is 
      straightforward and clearly makes the case for the addition of the 
      normalization mechanism. The paper is concise and clearly written. 
      Weaknesses: -- ``The perception of combined stimuli 
      can therefore be fully predicted by optimal combining responses from 
      individual spatiotemporal frequency channels.'' This statement 
      (``fully predicted'') seems too strong to me for two reasons: 1) while 
      bias is predicted really well, threshold fits are less impressive and 2) 
      no new data has been predicted (all data were used in the model fits). It 
      would be interesting to see the cross-validated area under the ROC between 
      the two models and the Gaussian fits to give the reader a better sense of 
      the scale of the prediction improvement. On thresholds, while it's 
      hard to say from three data points, is there a plausible reason for my 
      impression that the discrimination thresholds for combined stimuli seem to 
      decline more steeply before plateauing (e.g. power law or exponential) 
      rather than the more linear predictions of the model? That is, the 
      threshold predictions at high contrasts are good, but thresholds at low 
      contrasts might be under-predicted whereas thresholds at medium contrasts 
      are over-predicted. Would the ``Foley function'' with the exponent of the 
      numerator set to 2.4 (instead of 2 in your CRF) provide a better fit to 
      those data due to the accelerating contrast response early in the 
      function? If I understand correctly, the channel variances 
      \sigma^2 correspond to (or can be transformed to give) the bandwidths of 
      the hypothetical channels. Can the authors comment on whether the 
      bandwidths estimated in their model fits correspond to bandwidths of 
      spatiotemporally-oriented filters measured elsewhere (e.g. physiology, 
      psychophysical masking paradigms)? Line 367: ``The model also 
      fully supports previous accounts on the shape of the prior for speed: the 
      slope of the linear approximation of the prior probability density is 
      negative and conforms to previously reported values.'' Where in the 
      manuscript is the slope of the prior reported (or any fitted model 
      parameters for that matter)? A small table of best-fitting model 
      parameters and their standard errors would be of use here. -- How 
      easily can the two-channel model derivation be extended? Will 
      multi-channel (> 3) responses require numerical approximations? 
      Minor points: Line 85: grammar... "proposing that the 
      visual system treats the responses in individual spatiotemporal frequency 
      channel as independent cues". Line 103: grammar... "only those unit" 
      should be "units" Line 107: grammar... "An moving stimulus" should be 
      "A moving stimulus" Line 131: typo "r2," Line 242: typo "agains 
      data" I found the paragraph beginning Line 256 unclear, and I 
      didn't really understand the stimulus combinations until reading the 
      results. I think it would be clearer to write something like: ``There 
      were four simple stimuli: a 0.5 c/deg grating presented at 22.5% contrast, 
      and three 1.5 c/deg gratings presented at 7.5, 22.5 and 67.5% contrast 
      respectively. Combined stimuli were superpositions of the lower and higher 
      frequency simple stimuli, in either a ``peaks-add'' or ``peaks-subtract'' 
      configuration.'' Line 261:``We used a broadband stimulus regulated 
      by an adaptive staircase as reference.'': it would be useful to explicitly 
      state here that it is the speed of the reference that is under adaptive 
      control. Line 315: ``we plot the peaks-add conditions as black 
      squares, and the peaks-subtract conditions as black triangles.'' I see no 
      black triangles in the figure; I assume these were separated into separate 
      panels. Line 320: ``independent on phase'' should be ``independent 
      of phase'' Line 375: ``This decrease is nicely reflected in the 
      measured discrimination thresholds for the combined stimuli where the 
      threshold for both individual gratings were of the same size (Fig. 4b)'': 
      I think this figure reference must be wrong because 4b refers to bias, not 
      thresholds. Line 377: ``predicts that the perceived speed are'' 
      should be ``speeds are'' or ``speed is''. Line 378: ``sensory 
      system in general processes'' should be ``systems in general processes'' 
      or ``system in general process''. Same for ``question'' on line 380.