Help Make the 19th Century Searchable
Commercially available OCR is amazingly bad. The errors it makes are crazy. For example, if the top of a 'd' is missing it might read it as an 'a', while a human can see that it's identical to all the other 'd's on the page except that a bit is missing, while it looks nothing at all like an 'a' <i>in any of the fonts used in that book</i>. Or the OCR sees a random blotch and guesses it might be a comma, although it's in completely the wrong place and has the wrong size to be part of the text.<p>I think perhaps the developers took a wrong turn when they started trying to improve OCR with language models rather than font models. Humans can accurately transcribe a <i>printed</i> text without knowing the language. They do that with a mental model of font metrics and so on. In fact, a human transcribes more accurately (though more slowly) when they don't know the language because they don't erroneously "autocorrect".<p>I wonder if Google has in-house OCR that works much better than commercially available OCR. Google has OCR-ed at least 25 million books. You can't download the complete texts, but you can see snippets. Perhaps someone would like to publish a paper assessing the quality of Google's OCR and comparing it with commercial software. (Probably someone has done that already; I'm just bad at finding papers.)