<p>This is a really cool problem. You already have a working model here are a few different ways of going forward with the project.</p>

<ul>
<li><p>Grouping text based on locality. "no segmentation" <a href="https://www.sciencedirect.com/science/article/abs/pii/S0031320302000821" rel="nofollow noreferrer" > Text region extraction in a document image based on the Delaunay tessellation </a></p></li>
<li><p>Segmentation <a href="https://ieeexplore.ieee.org/abstract/document/4036951" rel="nofollow noreferrer" > Multiscale Edge-Based Text Extraction from Complex Images </a></p></li>
<li><p>Training a map of the image then extracting text form the map. <a href="http://openaccess.thecvf.com/content_cvpr_2016/html/Zhang_Multi-Oriented_Text_Detection_CVPR_2016_paper.html" rel="nofollow noreferrer" > Multi Oriented Text Detection </a></p></li>
</ul>

<p>Training the model to detect boxes followed by extracting the text from each box seems like a very smart direction to move in. This paper talks about the former. <a href="https://ieeexplore.ieee.org/abstract/document/995125" rel="nofollow noreferrer" > automatic image segmentation and edge detection </a> Good luck!</p>

CC BY-SA 4.0
2019-11-17T23:58:01.967Z
Thank you for the directions! I'll look at them and return the check answer as soon as I've figured it out.
Tiago Duque
CC BY-SA 4.0
2019-11-15T22:14:24.633Z
2019-11-15T22:14:24.633Z
Michael Hearn
<p>I'm working on a project where I need to extract text from grocery discount flyers like the Costco announcement below (retrieved in a random google search, Costco is not the deal here):</p>

<p><a href="https://i.sstatic.net/PcVxY.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/PcVxY.jpg" alt="example sample image"></a></p>

<p>If I just run OCR (like with Tesseract in python):</p>

<pre><code>import cv2
import pytesseract
img = cv2.imread('costco.jpg')
text = pytesseract.image_to_string(img)
print(text)
</code></pre>

<p>I get:</p>

<blockquote>
  <p>Cadbury Chocolate</p>
  
  <p>variety pack packet</p>
  
  <p>ere <span class="math-container">$12.99 i rom hagst 31026 2012
&gt; 
&gt; Je laa
&gt; + a
&gt; 
&gt; Wrigley’s Excel Gum variety
&gt; 
&gt; Backol 24
&gt; 
&gt; $</span>13.79 fom agus 26.202</p>
  
  <p>OFF</p>
  
  <p>Solon Extra virgin olive oil [...]</p>
</blockquote>

<p>Which is a lot noisy.</p>

<p>My guess is that splitting the image to its base squares enchances the recognition.</p>

<p>However, I'm confused on how to do it. I can classify images using a CNN, but am not sure about object recognition.</p>

<p>Should I have a sliding window and train several "grid box" objects on a generic CNN and then provide this window data to be classified? How to adapt to distinct object window sizes?</p>

<convolutional-neural-networks><image-segmentation>
Best approach for 2D Grid Image Segmentation
Tiago Duque
# Best approach for 2D Grid Image Segmentation

(Asked by: username_0 on 2019-11-15)

I'm working on a project where I need to extract text from grocery discount flyers like the Costco announcement below (retrieved in a random google search, Costco is not the deal here):

If I just run OCR (like with Tesseract in python):
import cv2
import pytesseract
img = cv2.imread('costco.jpg')
text = pytesseract.image_to_string(img)
print(text)

I get:

Cadbury Chocolate
variety pack packet
ere $12.99 i rom hagst 31026 2012
> 
> Je laa
> + a
> 
> Wrigley’s Excel Gum variety
> 
> Backol 24
> 
> $13.79 fom agus 26.202
OFF
Solon Extra virgin olive oil [...]

Which is a lot noisy.
My guess is that splitting the image to its base squares enchances the recognition.
However, I'm confused on how to do it. I can classify images using a CNN, but am not sure about object recognition.
Should I have a sliding window and train several "grid box" objects on a generic CNN and then provide this window data to be classified? How to adapt to distinct object window sizes?

**Tags:** convolutional-neural-networks, image-segmentation


---

## 1 answers

### Answer by username_1 on 2019-11-15. Score: 2

This is a really cool problem. You already have a working model here are a few different ways of going forward with the project.

* Grouping text based on locality. "no segmentation"  [Text region extraction in a document image based on the Delaunay tessellation](https://www.sciencedirect.com/science/article/abs/pii/S0031320302000821)
* Segmentation  [Multiscale Edge-Based Text Extraction from Complex Images](https://ieeexplore.ieee.org/abstract/document/4036951)
* Training a map of the image then extracting text form the map.  [Multi Oriented Text Detection](http://openaccess.thecvf.com/content_cvpr_2016/html/Zhang_Multi-Oriented_Text_Detection_CVPR_2016_paper.html)

Training the model to detect boxes followed by extracting the text from each box seems like a very smart direction to move in. This paper talks about the former.  [automatic image segmentation and edge detection](https://ieeexplore.ieee.org/abstract/document/995125)  Good luck!

> **Comments:**
> **username_0**: Thank you for the directions! I'll look at them and return the check answer as soon as I've figured it out.  

---

