Answers.Body: <p>The estimation of optical flow by the <a href="https://en.wikipedia.org/wiki/Horn%E2%80%93Schunck_method" rel="nofollow noreferrer">Horn-Schunk method</a> can be written in matrix form as (for the sake of clarity, assume no regularization, that is, <span class="math-container">$\alpha = 0$</span>)</p>
  <p><span class="math-container">$$
  \left(
  \begin{array}{ll}
  I_x^2 &amp; I_xI_y \\
  I_xI_y &amp; I_y^2 \\
  \end{array}
  \right)
  \left(
  \begin{array}{ll}
  u \\ v
  \end{array}
  \right) = 
  \left(
  \begin{array}{ll}
  -I_x I_t \\ -I_y I_t
  \end{array}
  \right)
  $$</span></p>
  <p>where <span class="math-container">$(u, v)$</span> is the estimated displacement between two frames.</p>
  <p>You can solve this linear system of equations if the matrix on the left is not singular, i.e. its eigenvalues are not zero. But this means that gradients in both directions are not zero. And this is what the <a href="https://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/OWENS/LECT12/node4.html" rel="nofollow noreferrer">aperture problem</a> is about: when the matrix is singular.</p>
  <p>As an example, consider a black square aligned along the coordinates axis, moving horizontally (along x axis) on a white background, and you want to estimate the displacement of the square from the optical flow. Furthermore, you do not consider the whole image, but just some rectangular area of the image.</p>
  <p>Now, if your rectangle does not contain any of the edges perpendicular to the movement direction, you will not be able to estimate the movement because your gradient estimates will be <span class="math-container">$I_x = I_y = 0$</span>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-25T15:38:51.957Z
Answers.LastActivityDate: 2021-12-25T19:20:01.393Z
Answers.LastEditDate: 2021-12-25T19:20:01.393Z
Answers.UserDisplayName: jpmuc
Body: <p>This is regarding the details stated in <a href="https://en.wikipedia.org/wiki/Horn%E2%80%93Schunck_method" rel="nofollow noreferrer">Wikipedia</a>.</p>
  <p>I am reading optical flow in Computer Vision. I understood the Horn–Schunck method as such, but did not get how it is related to the aperture problem, and how it is solved using Horn–Schunck method.</p>
  <p>Also, why is Horn–Schunck method invented/used where a simpler &quot;Lucas–Kanade method&quot; is already there (<a href="https://en.wikipedia.org/wiki/Lucas%E2%80%93Kanade_method" rel="nofollow noreferrer">Reference</a>)?</p>
  
Tags: <computer-vision><optical-flow>
Title: How does Horn–Schunck method for Optical Flow solve the aperture problem?
UserDisplayName: Sukti Sen
ThreadText: # How does Horn–Schunck method for Optical Flow solve the aperture problem?
  
  (Asked by: username_0 on 2021-12-23)
  
  This is regarding the details stated in [Wikipedia](https://en.wikipedia.org/wiki/Horn%E2%80%93Schunck_method).
  
  I am reading optical flow in Computer Vision. I understood the Horn–Schunck method as such, but did not get how it is related to the aperture problem, and how it is solved using Horn–Schunck method.
  
  Also, why is Horn–Schunck method invented/used where a simpler "Lucas–Kanade method" is already there ([Reference](https://en.wikipedia.org/wiki/Lucas%E2%80%93Kanade_method))?
  
  **Tags:** computer-vision, optical-flow
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-12-25. Score: 0
  
  The estimation of optical flow by the [Horn-Schunk method](https://en.wikipedia.org/wiki/Horn%E2%80%93Schunck_method) can be written in matrix form as (for the sake of clarity, assume no regularization, that is, $\alpha = 0$)
  
  $$
  \left(
  \begin{array}{ll}
  I\_x^2 & I\_xI\_y \\
  I\_xI\_y & I\_y^2 \\
  \end{array}
  \right)
  \left(
  \begin{array}{ll}
  u \\ v
  \end{array}
  \right) =
  \left(
  \begin{array}{ll}
  -I\_x I\_t \\ -I\_y I\_t
  \end{array}
  \right)
  $$
  
  where $(u, v)$ is the estimated displacement between two frames.
  
  You can solve this linear system of equations if the matrix on the left is not singular, i.e. its eigenvalues are not zero. But this means that gradients in both directions are not zero. And this is what the [aperture problem](https://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/OWENS/LECT12/node4.html) is about: when the matrix is singular.
  
  As an example, consider a black square aligned along the coordinates axis, moving horizontally (along x axis) on a white background, and you want to estimate the displacement of the square from the optical flow. Furthermore, you do not consider the whole image, but just some rectangular area of the image.
  
  Now, if your rectangle does not contain any of the edges perpendicular to the movement direction, you will not be able to estimate the movement because your gradient estimates will be $I\_x = I\_y = 0$.
  
  
  ---
  
  