Answers.Body: <p>After a long wait and some digging, I accidently found what I was looking for. In 2015, polish researcher <strong>Dominika Traczyk</strong> publish an article presenting CERMINE, a solution for the posted problem.</p>
  <p>His solution is SVM-based, but the article gives good insights for an alternate Neural Network version.</p>
  <p>The article is open access and can be found on the <a href="https://link.springer.com/article/10.1007/s10032-015-0249-8" rel="nofollow noreferrer">Springer website</a>, while all the source code is available on <a href="https://github.com/CeON/CERMINE" rel="nofollow noreferrer">GitHub</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-19T14:29:07.463Z
Answers.LastActivityDate: 2021-10-19T15:17:35.303Z
Answers.LastEditDate: 2021-10-19T15:17:35.303Z
Answers.UserDisplayName: AlexSC
Answers.Body: <blockquote>
  <p>How to extract the main content text from a formated text file?</p>
  </blockquote>
  <h3>I am not sure that just a neural network is the best approach to your problem.</h3>
  <p>Traditional <a href="https://en.wikipedia.org/wiki/Natural_language_processing" rel="nofollow noreferrer">natural language processing</a> software are using something else, and generally using a complex mix of several techniques. I am supposing you are processing written text available as some file (in a file format you are <em>very</em> familiar with, e.g. <a href="https://en.wikipedia.org/wiki/Office_Open_XML" rel="nofollow noreferrer">OOXML</a> or <a href="https://en.wikipedia.org/wiki/PDF" rel="nofollow noreferrer">PDF</a> or <a href="https://en.wikipedia.org/wiki/HTML5" rel="nofollow noreferrer">HTML5</a>).</p>
  <p>Read the <a href="https://en.wikipedia.org/wiki/Natural-language_understanding" rel="nofollow noreferrer">wikipage on natural-language understanding</a> and the one on <a href="https://en.wikipedia.org/wiki/Parse_tree" rel="nofollow noreferrer">parse trees</a> (or concrete syntax trees).</p>
  <p>BTW, you might use <a href="https://www.latex-project.org/" rel="nofollow noreferrer">LaTeX</a> or the <a href="https://en.wikipedia.org/wiki/Lout_(software)" rel="nofollow noreferrer">Lout</a> formatter to produce some PDF file. Both are open-source software (easily available on most Linux distributions, including <a href="https://debian.org/" rel="nofollow noreferrer">Debian</a> or <a href="https://ubuntu.com/" rel="nofollow noreferrer">Ubuntu</a>). I recommend you to try generating some PDF file using them, and experiment on the generated PDF file. And a lot of AI papers are available (as preprints) in PDF form.</p>
  <p>You could also use, as a PDF input to experiment your software, <a href="http://starynkevitch.net/Basile/bismon-chariot-doc.pdf" rel="nofollow noreferrer">this</a> or <a href="http://starynkevitch.net/Basile/refpersys-design.pdf" rel="nofollow noreferrer">that</a> draft reports (you might enjoy reading them too...). <strong>If in 2021 your software is capable of &quot;understanding&quot; and &quot;abstracting/summarizing&quot; these PDF files, please send me an email</strong> to <code>basile@starynkevitch.net</code> explaining (in written English) how you did build your neural network and what is the output of your software.</p>
  <p>There are several issues:</p>
  <ul>
  <li><p>extracting the non-textual things (e.g. HTML tags from HTML input, or strings from a PDF file, or some <a href="https://en.wikipedia.org/wiki/LaTeX" rel="nofollow noreferrer">LaTeX</a> one).</p>
  </li>
  <li><p>detecting the human language used in your text (e.g. French or English or Russian or Chinese). <a href="https://en.wikipedia.org/wiki/N-gram" rel="nofollow noreferrer">N-gram</a> based techniques come to mind.</p>
  </li>
  <li><p>having a data structure or database representing a dictionnary of a thousand (at least) of significant words (in English or Russian or whatever human language you are interested in) related to the domain you want to handle (that dictionary would be different if you want to parse weather forecasts or documentation related to the automotive industry, since the word <code>pressure</code> or <code>speed</code> relates to different concepts. Notice also that &quot;weather&quot; and &quot;time&quot; are expressed in French by the <em>same</em> word: &quot;temps&quot; - as in &quot;le temps qu'il fait&quot; for ongoing weather and &quot;le temps qui passe&quot; for the flow of time). A &quot;Queen&quot; is not the same for a chess player and an historian. A program translating -or just analyzing- chess comments from English won't use the same word for translating / understanding &quot;<a href="https://en.wikipedia.org/wiki/Bishop_(chess)" rel="nofollow noreferrer">bishop</a>&quot; (in chess, &quot;fou&quot; in French, literally the crazy guy, unrelated to religion; in Russian chess books it would be &quot;слон&quot;,  literally an elephant) than another program translating / analyzing historical comments from English (e.g. about <a href="https://en.wikipedia.org/wiki/Mary,_Queen_of_Scots" rel="nofollow noreferrer">Mary Stuart</a>).</p>
  </li>
  <li><p>modeling inside your software some domain-specific knowledge related to your analyzed text, since you would handle differently weather forecast text to textual comments of chess competitions, or textual exercises in any computer science or programming book (like <a href="https://en.wikipedia.org/wiki/Introduction_to_Algorithms" rel="nofollow noreferrer">CLRS</a>). You could use some <a href="https://en.wikipedia.org/wiki/Frame_(artificial_intelligence)" rel="nofollow noreferrer">frame</a>-based representations, like in <a href="http://refpersys.org/" rel="nofollow noreferrer">RefPerSys</a> or in <a href="https://en.wikipedia.org/wiki/Cyc" rel="nofollow noreferrer">CyC</a>.</p>
  </li>
  <li><p>building a <a href="https://en.wikipedia.org/wiki/Semantic_network" rel="nofollow noreferrer">semantic network</a> representing the input text. I believe you might need some prior one representing domain-specific knowledge in the area of the analyzed text (e.g. a program analyzing comments on chess games needs to know the rule of chess; another program analyzing <a href="https://stackoverflow.com/">StackOverflow</a> answers probably needs to know something about operating systems in general). In think that in English &quot;overflow&quot; or &quot;overheating&quot; means <em>very different concepts</em> to software developers and to weather forecasters or climate experts.</p>
  </li>
  </ul>
  <p>Look also for inspiration into <a href="http://bootstrappingartificialintelligence.fr/WordPress3/" rel="nofollow noreferrer">this blog</a> of the late Jacques Pitrat. He did wrote an interesting book on your topic.</p>
  <p>You might look inside the <a href="https://www.decoder-project.eu/" rel="nofollow noreferrer">DECODER</a> European project, and read more about <a href="https://en.wikipedia.org/wiki/Expert_system" rel="nofollow noreferrer">expert systems</a> and their <a href="https://en.wikipedia.org/wiki/Inference_engine" rel="nofollow noreferrer">inference engine</a> and <a href="https://en.wikipedia.org/wiki/Knowledge_base" rel="nofollow noreferrer">knowledge bases</a>.</p>
  <h2>Your project could give you some PhD.</h2>
  <p>You certainly need several years of work to achieve your goals. I suggest contacting some academic in your area to be your advisor.</p>
  <p>Notice that on Linux the <a href="https://linux.die.net/man/1/pdftotext" rel="nofollow noreferrer"><code>pdf2text</code> software</a> is extracting text from PDF files. It is open-source, but I won't say it is an AI software. However, you could use it thru <a href="https://man7.org/linux/man-pages/man3/popen.3.html" rel="nofollow noreferrer">popen(3)</a>. See also <a href="https://man7.org/linux/man-pages/man7/regex.7.html" rel="nofollow noreferrer">regex(7)</a>.</p>
  <p>BTW, the <a href="https://en.wikipedia.org/wiki/PDF" rel="nofollow noreferrer">PDF</a> specification is public as <a href="https://www.iso.org/standard/63534.html" rel="nofollow noreferrer">ISO 32000-2:2017</a> (and is related to <a href="https://en.wikipedia.org/wiki/PostScript" rel="nofollow noreferrer">PostScript</a>). Get it and read it, and see also this <a href="https://youtu.be/KmP7pbcAl-8" rel="nofollow noreferrer">youtube video</a> or <a href="https://www.adobe.com/content/dam/acom/en/devnet/pdf/pdfs/pdf_reference_archives/PDFReference.pdf" rel="nofollow noreferrer">this 978 pages document</a>. On Linux, most PDF files can usually be inspected with <a href="https://man7.org/linux/man-pages/man1/od.1.html" rel="nofollow noreferrer">od(1)</a> or <a href="https://man7.org/linux/man-pages/man1/less.1.html" rel="nofollow noreferrer">less(1)</a>.</p>
  <p>My HP Office Pro 8610 printer (connected to a Linux desktop) is capable of printing some PDF and of scanning into a PDF file. But if I print on paper some PDF file and scan it, the PDF file did change a lot, even if visually it looks the same.</p>
  <p>Notice that some drawings -or photos- could be embedded in a PDF file, and appear to a non-blind human reader as letters.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-19T12:08:30.663Z
Answers.Text: AFAIK, when using [LaTeX](https://www.latex-project.org/) to produce some PDF file, the generated PDF does not know about header and footers and columns. Of course, the LaTeX source file does know about them.
Answers.UserDisplayName: Basile Starynkevitch
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-19T12:20:45.530Z
Answers.Text: BTW, if your software is capable of using NLP techniques on PDF, could you please send me an email explaining that. I am *not* asking for your software, just for you to explain me one of its successful runs. Honestly, I am skeptical about you being able to achieve your goal in one year of full time work.
Answers.UserDisplayName: Basile Starynkevitch
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-19T18:04:57.440Z
Answers.Text: Please do so. When it works, send me an email. My intuition is that a full year of work would be needed, but I hope for you that is is less, or that you have enough money to live during that while working on your interesting problem.
Answers.UserDisplayName: Basile Starynkevitch
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-21T12:57:53.017Z
Answers.Text: Sorry, but nota really what I'm trying to do. Mu problema hás nothing to do with the information inside the texto, just identifying the text portions.
Answers.UserDisplayName: AlexSC
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-22T11:57:27.733Z
Answers.Text: @AlexSC: please edit your question to define in several paragraphs of written English what exactly is the text portion of a PDF file. In particular, a PDF file can contain photos  of objects having some text of them, or drawings (i.e. figures) with text. An example is  [this draft report](http://starynkevitch.net/Basile/bismon-chariot-doc.pdf). What do you think are the text portions of it? Is the figure 1 on page 10 some text of that file?
Answers.UserDisplayName: Basile Starynkevitch
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-24T15:05:22.880Z
Answers.Text: @BasileStarynkevitch, I just added some clarifications.
Answers.UserDisplayName: AlexSC
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-07T11:03:58.957Z
Answers.Text: It seems similar to extracting a support email question and support email provided answer leaving all the previous responses in those emails and email signatures out. Imagine 1000s of questions and replies in the support box and you want to extract only actual questions and answers leaving all the noise to train ChatGPT. Maybe solving this problem is an image recognition problem as humans we can look at documents and point out where the main content is. So first you teach your model to "see" the data blocks it needs to extract and after you extract what was seen by your model as a target text?
Answers.UserDisplayName: Igor Kryltsov
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-19T07:38:15.723Z
Answers.LastActivityDate: 2020-11-19T18:49:24.133Z
Answers.LastEditDate: 2020-11-19T18:49:24.133Z
Answers.UserDisplayName: Basile Starynkevitch
Body: <p>My idea is to model and train a neural network that receives a text version of a PDF file as the input and gives the content text as output.</p>
  <p>Take the scenario:</p>
  <ol>
  <li><p>One prints a PDF file to a text file (the text file does not have images, but has the main text, headings, page numbers, some other footer text, and so on, and keeps the same number of columns  - two for instance - of text);</p>
  </li>
  <li><p>This text file is submitted to a tool that strips everything that is not the main content of the text in one single text column (one text stream), keeping the section titles, paragraphs, and the text in a readable form (does not mix columns);</p>
  </li>
  <li><p>The tool generates a new version of the original text file containing only the main text portion, ready to be used for other purposes where the striped parts would be considered noise.</p>
  </li>
  </ol>
  <p>How to model this problem in a way a neural network can handle it?</p>
  <h2>Update 1</h2>
  Here are some clarifications on the problem.
  <h3>PDF file</h3>
  <p>The picture below shows two pages of a pdf version of a scientific paper. This is just to set the context, the PDF file is not the input for this problem, it is just to understand where the actual input data comes from.</p>
  <p><a href="https://i.sstatic.net/SwBQl.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/SwBQl.jpg" alt="PDF version of a scientific paper with parts of interest in colored blocks" /></a></p>
  <p>The color boxes show some parts of interest for this discussion. Red boxes are headers and footers. <strong>We are not interested in them</strong>. Blue and green boxes are content text blocks. Different colors were used to emphasize the text is organized in columns and that is part of the problem. <strong>Those blue and green boxes are what we actually want</strong>.</p>
  <h3>Text file</h3>
  If a use the "save as text file" feature of my free PDF reader, I get a text file similar to the image below.
  <p><a href="https://i.sstatic.net/P8XwL.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/P8XwL.jpg" alt="Text version of the PDF file" /></a></p>
  <p>The text file is continuous, but I put the equivalent of the first two pages of the PDF file side-by-side just to make things easier to compare. We can see the very same colored boxes. In terms of words, those boxes contain the same text as in the PDF version.</p>
  <h3>Understanding the problem</h3>
  When we read a paper, we are usually not very interested in footers or headers. The main content is what we actually read and that will provide us with the knowledge we are looking for. In this case, the text is inside blue and green boxes.
  <p>So, what we want here is to generate a new version of the input (text) file organized in one single text stream (one column if you will), with the text laid-out in a form someone can read it, which means, alternating the blue and the green boxes.</p>
  <p>However, if the original PDF has no footers, it should work in the same way, providing the main text content. If the text comes in three of four columns, the final product must be a text in good condition to be read without losing any information.</p>
  <p>Any pictures will be simply stripped off the text version of the paper and we are fine with that.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-11-19T07:33:07.367Z
Comments.Text: How many years can you afford spending on such a project?
Comments.UserDisplayName: Basile Starynkevitch
Tags: <neural-networks>
Title: How to extract the main text from a formatted text file?
UserDisplayName: AlexSC
ThreadText: # How to extract the main text from a formatted text file?
  
  (Asked by: username_0 on 2020-11-17)
  
  My idea is to model and train a neural network that receives a text version of a PDF file as the input and gives the content text as output.
  
  Take the scenario:
  
  1. One prints a PDF file to a text file (the text file does not have images, but has the main text, headings, page numbers, some other footer text, and so on, and keeps the same number of columns - two for instance - of text);
  2. This text file is submitted to a tool that strips everything that is not the main content of the text in one single text column (one text stream), keeping the section titles, paragraphs, and the text in a readable form (does not mix columns);
  3. The tool generates a new version of the original text file containing only the main text portion, ready to be used for other purposes where the striped parts would be considered noise.
  
  How to model this problem in a way a neural network can handle it?
  
  ## Update 1
  
  Here are some clarifications on the problem.
  
  ### PDF file
  
  The picture below shows two pages of a pdf version of a scientific paper. This is just to set the context, the PDF file is not the input for this problem, it is just to understand where the actual input data comes from.
  
  [![PDF version of a scientific paper with parts of interest in colored blocks](https://i.sstatic.net/SwBQl.jpg)](https://i.sstatic.net/SwBQl.jpg)
  
  The color boxes show some parts of interest for this discussion. Red boxes are headers and footers. **We are not interested in them**. Blue and green boxes are content text blocks. Different colors were used to emphasize the text is organized in columns and that is part of the problem. **Those blue and green boxes are what we actually want**.
  
  ### Text file
  
  If a use the "save as text file" feature of my free PDF reader, I get a text file similar to the image below.
  
  [![Text version of the PDF file](https://i.sstatic.net/P8XwL.jpg)](https://i.sstatic.net/P8XwL.jpg)
  
  The text file is continuous, but I put the equivalent of the first two pages of the PDF file side-by-side just to make things easier to compare. We can see the very same colored boxes. In terms of words, those boxes contain the same text as in the PDF version.
  
  ### Understanding the problem
  
  When we read a paper, we are usually not very interested in footers or headers. The main content is what we actually read and that will provide us with the knowledge we are looking for. In this case, the text is inside blue and green boxes.
  
  So, what we want here is to generate a new version of the input (text) file organized in one single text stream (one column if you will), with the text laid-out in a form someone can read it, which means, alternating the blue and the green boxes.
  
  However, if the original PDF has no footers, it should work in the same way, providing the main text content. If the text comes in three of four columns, the final product must be a text in good condition to be read without losing any information.
  
  Any pictures will be simply stripped off the text version of the paper and we are fine with that.
  
  **Tags:** neural-networks
  
  > **Comments:**
  > **username_1**: How many years can you afford spending on such a project?  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_0 on 2021-10-19. Score: 2
  
  After a long wait and some digging, I accidently found what I was looking for. In 2015, polish researcher **Dominika Traczyk** publish an article presenting CERMINE, a solution for the posted problem.
  
  His solution is SVM-based, but the article gives good insights for an alternate Neural Network version.
  
  The article is open access and can be found on the [Springer website](https://link.springer.com/article/10.1007/s10032-015-0249-8), while all the source code is available on [GitHub](https://github.com/CeON/CERMINE).
  
  
  ---
  
  ### Answer by username_1 on 2020-11-19. Score: 1
  
  > How to extract the main content text from a formated text file?
  
  ### I am not sure that just a neural network is the best approach to your problem.
  
  Traditional [natural language processing](https://en.wikipedia.org/wiki/Natural_language_processing) software are using something else, and generally using a complex mix of several techniques. I am supposing you are processing written text available as some file (in a file format you are *very* familiar with, e.g. [OOXML](https://en.wikipedia.org/wiki/Office_Open_XML) or [PDF](https://en.wikipedia.org/wiki/PDF) or [HTML5](https://en.wikipedia.org/wiki/HTML5)).
  
  Read the [wikipage on natural-language understanding](https://en.wikipedia.org/wiki/Natural-language_understanding) and the one on [parse trees](https://en.wikipedia.org/wiki/Parse_tree) (or concrete syntax trees).
  
  BTW, you might use [LaTeX](https://www.latex-project.org/) or the [Lout](https://en.wikipedia.org/wiki/Lout_(software)) formatter to produce some PDF file. Both are open-source software (easily available on most Linux distributions, including [Debian](https://debian.org/) or [Ubuntu](https://ubuntu.com/)). I recommend you to try generating some PDF file using them, and experiment on the generated PDF file. And a lot of AI papers are available (as preprints) in PDF form.
  
  You could also use, as a PDF input to experiment your software, [this](http://username_1.net/username_1/bismon-chariot-doc.pdf) or [that](http://username_1.net/username_1/refpersys-design.pdf) draft reports (you might enjoy reading them too...). **If in 2021 your software is capable of "understanding" and "abstracting/summarizing" these PDF files, please send me an email** to `username_1@username_1.net` explaining (in written English) how you did build your neural network and what is the output of your software.
  
  There are several issues:
  
  * extracting the non-textual things (e.g. HTML tags from HTML input, or strings from a PDF file, or some [LaTeX](https://en.wikipedia.org/wiki/LaTeX) one).
  * detecting the human language used in your text (e.g. French or English or Russian or Chinese). [N-gram](https://en.wikipedia.org/wiki/N-gram) based techniques come to mind.
  * having a data structure or database representing a dictionnary of a thousand (at least) of significant words (in English or Russian or whatever human language you are interested in) related to the domain you want to handle (that dictionary would be different if you want to parse weather forecasts or documentation related to the automotive industry, since the word `pressure` or `speed` relates to different concepts. Notice also that "weather" and "time" are expressed in French by the *same* word: "temps" - as in "le temps qu'il fait" for ongoing weather and "le temps qui passe" for the flow of time). A "Queen" is not the same for a chess player and an historian. A program translating -or just analyzing- chess comments from English won't use the same word for translating / understanding "[bishop](https://en.wikipedia.org/wiki/Bishop_(chess))" (in chess, "fou" in French, literally the crazy guy, unrelated to religion; in Russian chess books it would be "слон", literally an elephant) than another program translating / analyzing historical comments from English (e.g. about [Mary Stuart](https://en.wikipedia.org/wiki/Mary,_Queen_of_Scots)).
  * modeling inside your software some domain-specific knowledge related to your analyzed text, since you would handle differently weather forecast text to textual comments of chess competitions, or textual exercises in any computer science or programming book (like [CLRS](https://en.wikipedia.org/wiki/Introduction_to_Algorithms)). You could use some [frame](https://en.wikipedia.org/wiki/Frame_(artificial_intelligence))-based representations, like in [RefPerSys](http://refpersys.org/) or in [CyC](https://en.wikipedia.org/wiki/Cyc).
  * building a [semantic network](https://en.wikipedia.org/wiki/Semantic_network) representing the input text. I believe you might need some prior one representing domain-specific knowledge in the area of the analyzed text (e.g. a program analyzing comments on chess games needs to know the rule of chess; another program analyzing [StackOverflow](https://stackoverflow.com/) answers probably needs to know something about operating systems in general). In think that in English "overflow" or "overheating" means *very different concepts* to software developers and to weather forecasters or climate experts.
  
  Look also for inspiration into [this blog](http://bootstrappingartificialintelligence.fr/WordPress3/) of the late Jacques Pitrat. He did wrote an interesting book on your topic.
  
  You might look inside the [DECODER](https://www.decoder-project.eu/) European project, and read more about [expert systems](https://en.wikipedia.org/wiki/Expert_system) and their [inference engine](https://en.wikipedia.org/wiki/Inference_engine) and [knowledge bases](https://en.wikipedia.org/wiki/Knowledge_base).
  
  ## Your project could give you some PhD.
  
  You certainly need several years of work to achieve your goals. I suggest contacting some academic in your area to be your advisor.
  
  Notice that on Linux the [`pdf2text` software](https://linux.die.net/man/1/pdftotext) is extracting text from PDF files. It is open-source, but I won't say it is an AI software. However, you could use it thru [popen(3)](https://man7.org/linux/man-pages/man3/popen.3.html). See also [regex(7)](https://man7.org/linux/man-pages/man7/regex.7.html).
  
  BTW, the [PDF](https://en.wikipedia.org/wiki/PDF) specification is public as [ISO 32000-2:2017](https://www.iso.org/standard/63534.html) (and is related to [PostScript](https://en.wikipedia.org/wiki/PostScript)). Get it and read it, and see also this [youtube video](https://youtu.be/KmP7pbcAl-8) or [this 978 pages document](https://www.adobe.com/content/dam/acom/en/devnet/pdf/pdfs/pdf_reference_archives/PDFReference.pdf). On Linux, most PDF files can usually be inspected with [od(1)](https://man7.org/linux/man-pages/man1/od.1.html) or [less(1)](https://man7.org/linux/man-pages/man1/less.1.html).
  
  My HP Office Pro 8610 printer (connected to a Linux desktop) is capable of printing some PDF and of scanning into a PDF file. But if I print on paper some PDF file and scan it, the PDF file did change a lot, even if visually it looks the same.
  
  Notice that some drawings -or photos- could be embedded in a PDF file, and appear to a non-blind human reader as letters.
  
  > **Comments:**
  > **username_1**: AFAIK, when using [LaTeX](https://www.latex-project.org/) to produce some PDF file, the generated PDF does not know about header and footers and columns. Of course, the LaTeX source file does know about them.  
  > **username_1**: BTW, if your software is capable of using NLP techniques on PDF, could you please send me an email explaining that. I am *not* asking for your software, just for you to explain me one of its successful runs. Honestly, I am skeptical about you being able to achieve your goal in one year of full time work.  
  > **username_1**: Please do so. When it works, send me an email. My intuition is that a full year of work would be needed, but I hope for you that is is less, or that you have enough money to live during that while working on your interesting problem.  
  > **username_0**: Sorry, but nota really what I'm trying to do. Mu problema hás nothing to do with the information inside the texto, just identifying the text portions.  
  > **username_1**: @username_0: please edit your question to define in several paragraphs of written English what exactly is the text portion of a PDF file. In particular, a PDF file can contain photos  of objects having some text of them, or drawings (i.e. figures) with text. An example is  [this draft report](http://username_1.net/username_1/bismon-chariot-doc.pdf). What do you think are the text portions of it? Is the figure 1 on page 10 some text of that file?  
  > **username_0**: @username_1, I just added some clarifications.  
  > **username_2**: It seems similar to extracting a support email question and support email provided answer leaving all the previous responses in those emails and email signatures out. Imagine 1000s of questions and replies in the support box and you want to extract only actual questions and answers leaving all the noise to train ChatGPT. Maybe solving this problem is an image recognition problem as humans we can look at documents and point out where the main content is. So first you teach your model to "see" the data blocks it needs to extract and after you extract what was seen by your model as a target text?  
  
  ---
  
  