Third-Order Edge Statistics: Contour Continuation, Curvature, and Cortical Connections
Association field models have attempted to explain human contour grouping performance, and to explain the mean frequency of long-range horizontal connections across cortical columns in V1. However, association fields only depend on the pairwise statistics of edges in natural scenes. We develop a spectral test of the sufficiency of pairwise statistics and show there is significant higher order structure. An analysis using a probabilistic spectral embedding reveals curvature-dependent components.
( Required, Visible To Authors During Feedback and 
      After Decision Notification ) Synopsis. This paper examines 
      third-order co-occurrence statistics of edges in natural images. The main 
      finding is that triples of edges co-occur in ways consistent with smooth 
      flows, elongated along the tangent direction. Better statistical 
      models of the geometry of natural image structure are crucial to 
      understanding biological vision as well as making progress in computer 
      vision, and the authors are correct in their claim that to date most 
      models have focused on 2nd-order relationships, particularly the joint 
      distribution of pairs of edges. Thus the investigation of higher-order 
      statistics is of great interest. Quality. Overall the methods 
      seem sound. Clarity. The paper is understandable in broad 
      strokes, but there are many parts that are unclear, and in general the 
      paper is a bit sloppy and hurried in presentation. Here are some examples: 
      • Line 74. I don’t think this equation is what the authors intend. 
      • Line 106. What data were used for this analysis? Later in Section 4, 
      the authors indicate that they use the van Hataren dataset, but it’s not 
      clear whether this is also used here. • Line 205 What is the K matrix? 
      Looks to me like the phis are just the eigenvectors of the P(i,j|0) 
      matrix. • Line 232 Do you mean Fig 4 • Line 234 Do you mean Fig 5? 
      • Fig 5. This figure is very unclear. What are the dimensions in the 
      top panels? The first two eigenvectors? • Lines 264. From the text I 
      cannot understand the color coding scheme. “The colors along each column 
      correspond, so similar colors map to nearby points along the dimension 
      corresponding to the row” – Columns of what? Rows of what? Clearly the 
      colors are somehow dependent on phi_2, phi_3 and phi_4 in the left, middle 
      and right columns. I assume then, that the color is determined by the 
      value of the projection of each point on these eigenvectors? • Line 
      249. The authors suggest that the projection on phi_2 reveals colinearity 
      and the projection on phi_4 indicates cocircularity. What then does the 
      projection on phi_3 represent? • …”each small cluster in diffusion 
      space corresponds to half of a cocircular field”. What is a cocircular 
      field? Where is the proof of this statement? • Line 274 “Shown 
      above..” above where? • Line 280 Why complex cells and not simple 
      cells? The assumed circuit model should be specified. • “Such 
      curvature-based facilitation can explain the non-monotonic variance in 
      excitatory long-range horizontal connections in V1”. This will be 
      completely impenetrable except for a very small group of cognoscenti. 
      • Line 288. We find that the embedding is only one dimensional, with 
      the significant eigenvector a monotonic function…” This is an example of 
      methodological sloppiness. What are the criteria for determining 
      dimensionality and significance? • Line 298? What does the figure on 
      the left represent, and what am I supposed to learn from it? • Line 
      312, “the distribution of these directions in 3D shape are an excellent 
      proxy for measuring the response of orientation filters to dense textured 
      objects”. Why do I want a proxy for measuring orientation filter 
      responses? Aren’t these the proximal measurements I can actually measure, 
      to make inference about the distal 3D shape variables I cannot directly 
      measure? • Line 346. Why are their two clusters (one cylinder, one 
      spot)? Originality. There isn’t that much work on higher-order 
      geometry statistics, so in this sense the paper is fairly novel. 
      Significance. Although the effort here is worthwhile, I do 
      think significance is lessened a bit because a) the results are not 
      particularly surprising or counter to current thinking, b) there is not a 
      clear advance in terms of hypothesis testing, improved quantitative model 
      or algorithm and c) the paper is unclear in places. On the other hand, 
      perhaps the novel spectral embedding method could in the future be used to 
      achieve a more clear contribution. Also, when judging 
      significance, it’s important to make clear that most prior work on natural 
      edge statistics does not really make the claim that 2nd-order statistics 
      are sufficient models for joint edge statistics in natural images. 
      For example, Geisler et al and Elder & Goldberg were not 
      measuring co-occurrence statistics over images. Geisler was measuring 
      co-occurrence statistics over object boundaries, and Elder & Goldberg 
      were measuring sequential statistics along object boundaries. Thus the 
      Elder & Goldberg data, for example, provide the statistics required 
      for a first-order Markov model of contours, not a Markov model of the 2D 
      edge (association) field. Thus in Fig 2, the conditional distribution for 
      contours is being misapplied to the problem of modeling the edge field. 
      To make this clearer, note that generally the E&G model would 
      predict the existence of higher-order co-occurrence statistics over the 
      edge field. For example, suppose this model is used to generate a number 
      of contours sparsely distributed over the image. The edge elements 
      belonging to the same contour would generate strong third-order 
      co-occurrences explainable by second-order statistics along the contour 
      but NOT explainable by second order statistics in the 2D association 
      field. This does not mean that the present effort to understand 
      these third-order co-occurrences in the edge field is not worthwhile, but 
      it’s important to put it in the right context. In particular, given this 
      presentation it is still not clear whether a model like the E&G model 
      might be sufficient to explain the 2D edge field. (Although we know this 
      is unlikely: Ren & Malik (2002) have in fact shown that contours are 
      not 1st-order Markov – no surprise.) X. F. Ren and J. Malik. A 
      probabilistic multi-scale model for contour completion based on image 
      statistics. In Lecture Notes in Computer Science, Proc. ECCV, volume 2350, 
      pages 312–327, Berlin, 2002. Springer-Verlag.