Answers.Body: <p>This is a really cool problem. You already have a working model here are a few different ways of going forward with the project.</p>
  
  <ul>
  <li><p>Grouping text based on locality. "no segmentation" <a href="https://www.sciencedirect.com/science/article/abs/pii/S0031320302000821" rel="nofollow noreferrer" > Text region extraction in a document image based on the Delaunay tessellation </a></p></li>
  <li><p>Segmentation <a href="https://ieeexplore.ieee.org/abstract/document/4036951" rel="nofollow noreferrer" > Multiscale Edge-Based Text Extraction from Complex Images </a></p></li>
  <li><p>Training a map of the image then extracting text form the map. <a href="http://openaccess.thecvf.com/content_cvpr_2016/html/Zhang_Multi-Oriented_Text_Detection_CVPR_2016_paper.html" rel="nofollow noreferrer" > Multi Oriented Text Detection </a></p></li>
  </ul>
  
  <p>Training the model to detect boxes followed by extracting the text from each box seems like a very smart direction to move in. This paper talks about the former. <a href="https://ieeexplore.ieee.org/abstract/document/995125" rel="nofollow noreferrer" > automatic image segmentation and edge detection </a> Good luck!</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-17T23:58:01.967Z
Answers.Text: Thank you for the directions! I'll look at them and return the check answer as soon as I've figured it out.
Answers.UserDisplayName: Tiago Duque
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-15T22:14:24.633Z
Answers.LastActivityDate: 2019-11-15T22:14:24.633Z
Answers.UserDisplayName: Michael Hearn
Body: <p>I'm working on a project where I need to extract text from grocery discount flyers like the Costco announcement below (retrieved in a random google search, Costco is not the deal here):</p>
  
  <p><a href="https://i.sstatic.net/PcVxY.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/PcVxY.jpg" alt="example sample image"></a></p>
  
  <p>If I just run OCR (like with Tesseract in python):</p>
  
  <pre><code>import cv2
  import pytesseract
  img = cv2.imread('costco.jpg')
  text = pytesseract.image_to_string(img)
  print(text)
  </code></pre>
  
  <p>I get:</p>
  
  <blockquote>
    <p>Cadbury Chocolate</p>
    
    <p>variety pack packet</p>
    
    <p>ere <span class="math-container">$12.99 i rom hagst 31026 2012
  &gt; 
  &gt; Je laa
  &gt; + a
  &gt; 
  &gt; Wrigley’s Excel Gum variety
  &gt; 
  &gt; Backol 24
  &gt; 
  &gt; $</span>13.79 fom agus 26.202</p>
    
    <p>OFF</p>
    
    <p>Solon Extra virgin olive oil [...]</p>
  </blockquote>
  
  <p>Which is a lot noisy.</p>
  
  <p>My guess is that splitting the image to its base squares enchances the recognition.</p>
  
  <p>However, I'm confused on how to do it. I can classify images using a CNN, but am not sure about object recognition.</p>
  
  <p>Should I have a sliding window and train several "grid box" objects on a generic CNN and then provide this window data to be classified? How to adapt to distinct object window sizes?</p>
  
Tags: <convolutional-neural-networks><image-segmentation>
Title: Best approach for 2D Grid Image Segmentation
UserDisplayName: Tiago Duque
ThreadText: # Best approach for 2D Grid Image Segmentation
  
  (Asked by: username_0 on 2019-11-15)
  
  I'm working on a project where I need to extract text from grocery discount flyers like the Costco announcement below (retrieved in a random google search, Costco is not the deal here):
  
  [![example sample image](https://i.sstatic.net/PcVxY.jpg)](https://i.sstatic.net/PcVxY.jpg)
  
  If I just run OCR (like with Tesseract in python):
  
  ```
  import cv2
  import pytesseract
  img = cv2.imread('costco.jpg')
  text = pytesseract.image_to_string(img)
  print(text)
  
  ```
  
  I get:
  
  > Cadbury Chocolate
  >
  > variety pack packet
  >
  > ere $12.99 i rom hagst 31026 2012
  > >
  > > Je laa
  > > + a
  > >
  > > Wrigley’s Excel Gum variety
  > >
  > > Backol 24
  > >
  > > $13.79 fom agus 26.202
  >
  > OFF
  >
  > Solon Extra virgin olive oil [...]
  
  Which is a lot noisy.
  
  My guess is that splitting the image to its base squares enchances the recognition.
  
  However, I'm confused on how to do it. I can classify images using a CNN, but am not sure about object recognition.
  
  Should I have a sliding window and train several "grid box" objects on a generic CNN and then provide this window data to be classified? How to adapt to distinct object window sizes?
  
  **Tags:** convolutional-neural-networks, image-segmentation
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-11-15. Score: 2
  
  This is a really cool problem. You already have a working model here are a few different ways of going forward with the project.
  
  * Grouping text based on locality. "no segmentation"  [Text region extraction in a document image based on the Delaunay tessellation](https://www.sciencedirect.com/science/article/abs/pii/S0031320302000821)
  * Segmentation  [Multiscale Edge-Based Text Extraction from Complex Images](https://ieeexplore.ieee.org/abstract/document/4036951)
  * Training a map of the image then extracting text form the map.  [Multi Oriented Text Detection](http://openaccess.thecvf.com/content_cvpr_2016/html/Zhang_Multi-Oriented_Text_Detection_CVPR_2016_paper.html)
  
  Training the model to detect boxes followed by extracting the text from each box seems like a very smart direction to move in. This paper talks about the former.  [automatic image segmentation and edge detection](https://ieeexplore.ieee.org/abstract/document/995125)  Good luck!
  
  > **Comments:**
  > **username_0**: Thank you for the directions! I'll look at them and return the check answer as soon as I've figured it out.  
  
  ---
  
  