title: KITAB-Bench: A Comprehensive Multi-Domain Benchmark for Arabic OCR and
    Document Understanding
summary: With the growing adoption of Retrieval-Augmented Generation (RAG) in document
  processing, robust text recognition has become increasingly critical for
  knowledge extraction. While OCR (Optical Character Recognition) for English and
  other languages benefits from large datasets and well-established benchmarks,
  Arabic OCR faces unique challenges due to its cursive script, right-to-left
  text flow, and complex typographic and calligraphic features. We present
  KITAB-Bench, a comprehensive Arabic OCR benchmark that fills the gaps in
  current evaluation systems. Our benchmark comprises 8,809 samples across 9
  major domains and 36 sub-domains, encompassing diverse document types including
  handwritten text, structured tables, and specialized coverage of 21 chart types
  for business intelligence. Our findings show that modern vision-language models
  (such as GPT-4, Gemini, and Qwen) outperform traditional OCR approaches (like
  EasyOCR, PaddleOCR, and Surya) by an average of 60% in Character Error Rate
  (CER). Furthermore, we highlight significant limitations of current Arabic OCR
  models, particularly in PDF-to-Markdown conversion, where the best model
  Gemini-2.0-Flash achieves only 65% accuracy. This underscores the challenges in
  accurately recognizing Arabic text, including issues with complex fonts,
  numeral recognition errors, word elongation, and table structure detection.
  This work establishes a rigorous evaluation framework that can drive
  improvements in Arabic document analysis methods and bridge the performance gap
  with English OCR technologies.
paper_authors._id: 67bc4ced7727595ca5b108f1
paper_authors.name: Ahmed Heakl
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-24T10:58:23.973Z
paper_authors._id: 656864e12d73834278a8dea7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/656864e12d73834278a8dea7/sfAWS2eyPtFHb_2GZIypp.jpeg
paper_authors.fullname: Ahmed Heakl
paper_authors.type: user
paper_authors.user: ahmedheakl
paper_authors._id: 67bc4ced7727595ca5b108f2
paper_authors.name: Abdullah Sohail
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-24T13:05:04.733Z
paper_authors._id: 672e4574b60c3a27d783a1ac
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/aut4W4hJcOT8jvQnlWs-y.png
paper_authors.fullname: Muhammad Abdullah
paper_authors.type: user
paper_authors.user: mabdullahsohail
paper_authors._id: 67bc4ced7727595ca5b108f3
paper_authors.name: Mukul Ranjan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-24T11:54:56.690Z
paper_authors._id: 65262a396b41932089fd7bae
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65262a396b41932089fd7bae/6YIEoAfJojuTW1UOKlwZT.png
paper_authors.fullname: Mukul Ranjan
paper_authors.type: user
paper_authors.user: mukul54
paper_authors._id: 67bc4ced7727595ca5b108f4
paper_authors.name: Rania Hossam
paper_authors._id: 67bc4ced7727595ca5b108f5
paper_authors.name: Ghazi Ahmed
paper_authors._id: 67bc4ced7727595ca5b108f6
paper_authors.name: Mohamed El-Geish
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-22T10:16:22.469Z
paper_authors._id: 5e796e1230dc073f817a2b92
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5e796e1230dc073f817a2b92/3i3MVu2XezcCpEOOB-OQH.jpeg
paper_authors.fullname: Mohamed El-Geish
paper_authors.type: user
paper_authors.user: elgeish
paper_authors._id: 67bc4ced7727595ca5b108f7
paper_authors.name: Omar Maher
paper_authors._id: 67bc4ced7727595ca5b108f8
paper_authors.name: Zhiqiang Shen
paper_authors._id: 67bc4ced7727595ca5b108f9
paper_authors.name: Fahad Khan
paper_authors._id: 67bc4ced7727595ca5b108fa
paper_authors.name: Salman Khan
paper_title: KITAB-Bench: A Comprehensive Multi-Domain Benchmark for Arabic OCR and
    Document Understanding
paper_summary: With the growing adoption of Retrieval-Augmented Generation (RAG) in document
  processing, robust text recognition has become increasingly critical for
  knowledge extraction. While OCR (Optical Character Recognition) for English and
  other languages benefits from large datasets and well-established benchmarks,
  Arabic OCR faces unique challenges due to its cursive script, right-to-left
  text flow, and complex typographic and calligraphic features. We present
  KITAB-Bench, a comprehensive Arabic OCR benchmark that fills the gaps in
  current evaluation systems. Our benchmark comprises 8,809 samples across 9
  major domains and 36 sub-domains, encompassing diverse document types including
  handwritten text, structured tables, and specialized coverage of 21 chart types
  for business intelligence. Our findings show that modern vision-language models
  (such as GPT-4, Gemini, and Qwen) outperform traditional OCR approaches (like
  EasyOCR, PaddleOCR, and Surya) by an average of 60% in Character Error Rate
  (CER). Furthermore, we highlight significant limitations of current Arabic OCR
  models, particularly in PDF-to-Markdown conversion, where the best model
  Gemini-2.0-Flash achieves only 65% accuracy. This underscores the challenges in
  accurately recognizing Arabic text, including issues with complex fonts,
  numeral recognition errors, word elongation, and table structure detection.
  This work establishes a rigorous evaluation framework that can drive
  improvements in Arabic document analysis methods and bridge the performance gap
  with English OCR technologies.
paper_ai_summary: KITAB-Bench, a comprehensive Arabic OCR benchmark, highlights the strengths and limitations of modern vision-language models in recognizing Arabic text, especially in complex scenarios like PDF-to-Markdown conversion.
paper_ai_keywords: Retrieval-Augmented Generation RAG text recognition KITAB-Bench OCR Arabic OCR cursive script right-to-left text character error rate CER GPT-4 Gemini Qwen EasyOCR PaddleOCR Surya vision-language models PDF-to-Markdown complex fonts numeral recognition word elongation table structure detection