General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model
Traditional OCR systems (OCR-1.0) are increasingly unable to meet people's
usage due to the growing demand for intelligent processing of man-made optical
characters. In this paper, we collectively refer to all artificial optical
signals (e.g., plain texts, math/molecular formulas, tables, charts, sheet
music, and even geometric shapes) as "characters" and propose the General OCR
Theory along with an excellent model, namely GOT, to promote the arrival of
OCR-2.0. The GOT, with 580M parameters, is a unified, elegant, and end-to-end
model, consisting of a high-compression encoder and a long-contexts decoder. As
an OCR-2.0 model, GOT can handle all the above "characters" under various OCR
tasks. On the input side, the model supports commonly used scene- and
document-style images in slice and whole-page styles. On the output side, GOT
can generate plain or formatted results (markdown/tikz/smiles/kern) via an easy
prompt. Besides, the model enjoys interactive OCR features, i.e., region-level
recognition guided by coordinates or colors. Furthermore, we also adapt dynamic
resolution and multi-page OCR technologies to GOT for better practicality. In
experiments, we provide sufficient results to prove the superiority of our
model.
66d8f9357765323b81bc6c53
Haoran Wei
admin_assigned
2024-09-13T12:10:14.107Z
6436618aeef1f55654a9f458
https://cdn-avatars.huggingface.co/v1/production/uploads/6436618aeef1f55654a9f458/OvxGtuDg2GAFG9As-2hzW.jpeg
Haoran Wei
user
HaoranWei
66d8f9357765323b81bc6c54
Chenglong Liu
admin_assigned
2024-09-13T12:10:23.238Z
64749d70bb0e9dd77646a1e7
/avatars/37db75d7e5f50f80909f98321d6f2d36.svg
liu
user
chenglongliu
66d8f9357765323b81bc6c55
Jinyue Chen
66d8f9357765323b81bc6c56
Jia Wang
66d8f9357765323b81bc6c57
Lingyu Kong
claimed_verified
2024-09-27T06:59:58.205Z
650d23b69169ea733178f508
/avatars/5f9a7ab6b6a44e74c3eb6a59da2e21ed.svg
Lingyu Kong
user
kppkkp
66d8f9357765323b81bc6c58
Yanming Xu
66d8f9357765323b81bc6c59
Zheng Ge
66d8f9357765323b81bc6c5a
Liang Zhao
66d8f9357765323b81bc6c5b
Jianjian Sun
66d8f9357765323b81bc6c5c
Yuang Peng
admin_assigned
2024-09-13T12:11:40.254Z
631ee086c1a8269da39265c6
https://cdn-avatars.huggingface.co/v1/production/uploads/631ee086c1a8269da39265c6/wUa1epGtTGcUv2mvrLUcD.png
Yuang Peng
user
yuangpeng
66d8f9357765323b81bc6c5d
Chunrui Han
66d8f9357765323b81bc6c5e
Xiangyu Zhang
General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model
Traditional OCR systems (OCR-1.0) are increasingly unable to meet people's
usage due to the growing demand for intelligent processing of man-made optical
characters. In this paper, we collectively refer to all artificial optical
signals (e.g., plain texts, math/molecular formulas, tables, charts, sheet
music, and even geometric shapes) as "characters" and propose the General OCR
Theory along with an excellent model, namely GOT, to promote the arrival of
OCR-2.0. The GOT, with 580M parameters, is a unified, elegant, and end-to-end
model, consisting of a high-compression encoder and a long-contexts decoder. As
an OCR-2.0 model, GOT can handle all the above "characters" under various OCR
tasks. On the input side, the model supports commonly used scene- and
document-style images in slice and whole-page styles. On the output side, GOT
can generate plain or formatted results (markdown/tikz/smiles/kern) via an easy
prompt. Besides, the model enjoys interactive OCR features, i.e., region-level
recognition guided by coordinates or colors. Furthermore, we also adapt dynamic
resolution and multi-page OCR technologies to GOT for better practicality. In
experiments, we provide sufficient results to prove the superiority of our
model.
GOT, a unified OCR model with 580M parameters, handles various types of characters and OCR tasks and offers advanced features like region-level recognition and dynamic resolution.
high-compression encoder long-contexts decoder OCR-2.0 region-level recognition dynamic resolution multi-page OCR