title: MOLE: Metadata Extraction and Validation in Scientific Papers Using LLMs
summary: Metadata extraction is essential for cataloging and preserving datasets,
  enabling effective research discovery and reproducibility, especially given the
  current exponential growth in scientific research. While Masader (Alyafeai et
  al.,2021) laid the groundwork for extracting a wide range of metadata
  attributes from Arabic NLP datasets' scholarly articles, it relies heavily on
  manual annotation. In this paper, we present MOLE, a framework that leverages
  Large Language Models (LLMs) to automatically extract metadata attributes from
  scientific papers covering datasets of languages other than Arabic. Our
  schema-driven methodology processes entire documents across multiple input
  formats and incorporates robust validation mechanisms for consistent output.
  Additionally, we introduce a new benchmark to evaluate the research progress on
  this task. Through systematic analysis of context length, few-shot learning,
  and web browsing integration, we demonstrate that modern LLMs show promising
  results in automating this task, highlighting the need for further future work
  improvements to ensure consistent and reliable performance. We release the
  code: https://github.com/IVUL-KAUST/MOLE and dataset:
  https://huggingface.co/datasets/IVUL-KAUST/MOLE for the research community.
paper_authors._id: 68356e736bb42c7e99d2d266
paper_authors.name: Zaid Alyafeai
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-27T08:47:10.858Z
paper_authors._id: 5f04bd384ec31d33a72116d1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1594145966049-noauth.jpeg
paper_authors.fullname: Zaid Alyafeai
paper_authors.type: user
paper_authors.user: Zaid
paper_authors._id: 68356e736bb42c7e99d2d267
paper_authors.name: Maged S. Al-Shaibani
paper_authors._id: 68356e736bb42c7e99d2d268
paper_authors.name: Bernard Ghanem
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:42:05.974Z
paper_authors._id: 6808bf97ffadd78ec71cb721
paper_authors.avatarUrl: /avatars/9adca3142c06b8f69889fcbe85fa374d.svg
paper_authors.fullname: Bernard Ghanem
paper_authors.type: user
paper_authors.user: bernardghanem
paper_title: MOLE: Metadata Extraction and Validation in Scientific Papers Using LLMs
paper_summary: Metadata extraction is essential for cataloging and preserving datasets,
  enabling effective research discovery and reproducibility, especially given the
  current exponential growth in scientific research. While Masader (Alyafeai et
  al.,2021) laid the groundwork for extracting a wide range of metadata
  attributes from Arabic NLP datasets' scholarly articles, it relies heavily on
  manual annotation. In this paper, we present MOLE, a framework that leverages
  Large Language Models (LLMs) to automatically extract metadata attributes from
  scientific papers covering datasets of languages other than Arabic. Our
  schema-driven methodology processes entire documents across multiple input
  formats and incorporates robust validation mechanisms for consistent output.
  Additionally, we introduce a new benchmark to evaluate the research progress on
  this task. Through systematic analysis of context length, few-shot learning,
  and web browsing integration, we demonstrate that modern LLMs show promising
  results in automating this task, highlighting the need for further future work
  improvements to ensure consistent and reliable performance. We release the
  code: https://github.com/IVUL-KAUST/MOLE and dataset:
  https://huggingface.co/datasets/IVUL-KAUST/MOLE for the research community.
paper_ai_summary: A framework leveraging Large Language Models (LLMs) is introduced for automatic metadata extraction from scientific papers, with a focus on datasets from languages other than Arabic.
paper_ai_keywords: Large Language Models (LLMs) schema-driven methodology benchmark few-shot learning web browsing integration