Body: <p>I am completely new to computer vision and I am working on a small hobby project.
  The goal is to use camera footage of a foosball table to map the image to already well defined object geometry with few moving parts.</p>
  <p>To illustrate the exact problem the input could be this:
  <a href="https://i.sstatic.net/yinR8.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/yinR8.png" alt="Video footage of a foos match" /></a></p>
  <p>I am looking to identify all the features of the image and map them to an exact model that could be rendered like so:
  <a href="https://i.sstatic.net/O0LZ0.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/O0LZ0.png" alt="Geometry features mapped on the model" /></a>
  It shows the position of the ball and player rods.</p>
  <p>I would already have an exact definition of what the image contains like:</p>
  <ul>
  <li>Exact size of the playing field</li>
  <li>Exact positions of player rods relative to the length of the field</li>
  <li>The player sizes and positions on the bar</li>
  <li>Every detail of the table is described already in the model and can not deviate from it</li>
  </ul>
  <p>Some limitations:</p>
  <ul>
  <li>The image would always be at an angle. So part of the playing field is cropped and would have to be extrapolated.</li>
  <li>The lighting can vary from one set-up to another.</li>
  </ul>
  <p>There are only a few moving parts that need to pinned pointed- the position of the rods (can for now disregard the rotation) and the position of the ball.</p>
  <p>It sounds quite complicated but given that I already can tell exactly what I am looking for maybe it can make the task easier.</p>
  <p>What would be the most straight-forward approach to solving this problem? Any suggestions or further reading would be greatly appreciated.</p>
  <p>My current ideas would be to maybe identify a few anchor points from the image. For example if I would identify the top edge of the playing field - where the green playing pitch ends. And If I were to then be able to identify the position of the top player rod - the goalie rod. Measuring the distance between these lines would allow me to calculate the height of the camera angle because I know the exact spatial relationship between these two features in the pre-defined table model/geometry. With this data I would be able to calculate the positions where all the other features in the image could be located. If I could narrow down areas in the picture that need to be looked at for special features of the table I could maybe add additional constraints on the things that need to be looked at and simplify the task further.</p>
  <p>Another idea I had was using the table surface image to identify the table orientation. For instance given the above input picture I might also have a hardcoded reference picture like this:
  <a href="https://i.sstatic.net/LhxVM.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/LhxVM.jpg" alt="Reference image of the playing area" /></a></p>
  <p>Can this reference picture be used to find it in the input image in a skewed / cropped way and infer the table position from this overlaying of the two images?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-26T18:48:30.117Z
Comments.Text: I would first think of some more details concerning the data: Do you already have labeled data samples?  If so how many? Would camera position or the angle change?  Would the design of the table change (like figures, grass or ball colors)?
Comments.UserDisplayName: GKozinski
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-26T18:54:05.010Z
Comments.Text: @GKozinski, thanks for the comment as I am new to this it helps me formulate the question better. As for data - nothing. I was hoping to be able to identify some anchor points of the image and then use geometry to transform everything to the 2D model. For example: if I can detect the very top green edge of the table and also the position of the top goalie rod. I know exactly their relative distances from the pre-defined table definitions. So I should be able to calculate the view angle from those relative values.
  
  Nothing can change as far the table set-up. All is fixed and well defined.
Comments.UserDisplayName: apriede
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-26T18:58:18.757Z
Comments.Text: Will there always be this one table design or it can change in future, I mean for example will the players figures always have the same shape and yellow/black colors, or you plan using your model on different tables? Will camera stay always at exactly same position? You can edit the  question to deliver those details so it will be easier to understand your entire question
Comments.UserDisplayName: GKozinski
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-26T19:10:02.950Z
Comments.Text: @GKozinski, the camera can change. For simplicity I would assume it will always be centered but the height can change in small variations. For simplicity it can be assumed that the players and everything will also always remain the same.
Comments.UserDisplayName: apriede
Tags: <image-recognition><object-recognition><opencv>
Title: Mapping an image to a well defined object using OpenCV
UserDisplayName: apriede
ThreadText: # Mapping an image to a well defined object using OpenCV
  
  (Asked by: username_0 on 2022-08-26)
  
  I am completely new to computer vision and I am working on a small hobby project.
  The goal is to use camera footage of a foosball table to map the image to already well defined object geometry with few moving parts.
  
  To illustrate the exact problem the input could be this:
  [![Video footage of a foos match](https://i.sstatic.net/yinR8.png)](https://i.sstatic.net/yinR8.png)
  
  I am looking to identify all the features of the image and map them to an exact model that could be rendered like so:
  [![Geometry features mapped on the model](https://i.sstatic.net/O0LZ0.png)](https://i.sstatic.net/O0LZ0.png)
  It shows the position of the ball and player rods.
  
  I would already have an exact definition of what the image contains like:
  
  * Exact size of the playing field
  * Exact positions of player rods relative to the length of the field
  * The player sizes and positions on the bar
  * Every detail of the table is described already in the model and can not deviate from it
  
  Some limitations:
  
  * The image would always be at an angle. So part of the playing field is cropped and would have to be extrapolated.
  * The lighting can vary from one set-up to another.
  
  There are only a few moving parts that need to pinned pointed- the position of the rods (can for now disregard the rotation) and the position of the ball.
  
  It sounds quite complicated but given that I already can tell exactly what I am looking for maybe it can make the task easier.
  
  What would be the most straight-forward approach to solving this problem? Any suggestions or further reading would be greatly appreciated.
  
  My current ideas would be to maybe identify a few anchor points from the image. For example if I would identify the top edge of the playing field - where the green playing pitch ends. And If I were to then be able to identify the position of the top player rod - the goalie rod. Measuring the distance between these lines would allow me to calculate the height of the camera angle because I know the exact spatial relationship between these two features in the pre-defined table model/geometry. With this data I would be able to calculate the positions where all the other features in the image could be located. If I could narrow down areas in the picture that need to be looked at for special features of the table I could maybe add additional constraints on the things that need to be looked at and simplify the task further.
  
  Another idea I had was using the table surface image to identify the table orientation. For instance given the above input picture I might also have a hardcoded reference picture like this:
  [![Reference image of the playing area](https://i.sstatic.net/LhxVM.jpg)](https://i.sstatic.net/LhxVM.jpg)
  
  Can this reference picture be used to find it in the input image in a skewed / cropped way and infer the table position from this overlaying of the two images?
  
  **Tags:** image-recognition, object-recognition, opencv
  > **Comments:**
  > **username_1**: I would first think of some more details concerning the data: Do you already have labeled data samples?  If so how many? Would camera position or the angle change?  Would the design of the table change (like figures, grass or ball colors)?  
  > **username_0**: @GKozinski, thanks for the comment as I am new to this it helps me formulate the question better. As for data - nothing. I was hoping to be able to identify some anchor points of the image and then use geometry to transform everything to the 2D model. For example: if I can detect the very top green edge of the table and also the position of the top goalie rod. I know exactly their relative distances from the pre-defined table definitions. So I should be able to calculate the view angle from those relative values.  Nothing can change as far the table set-up. All is fixed and well defined.  
  > **username_1**: Will there always be this one table design or it can change in future, I mean for example will the players figures always have the same shape and yellow/black colors, or you plan using your model on different tables? Will camera stay always at exactly same position? You can edit the  question to deliver those details so it will be easier to understand your entire question  
  > **username_0**: @GKozinski, the camera can change. For simplicity I would assume it will always be centered but the height can change in small variations. For simplicity it can be assumed that the players and everything will also always remain the same.  
  
  ---
  
  ## 0 answers
  
  