{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d3dd7178-8337-44f0-a468-bc1af5c0e811",
   "metadata": {},
   "source": [
    "# How to load PDFs\n",
    "\n",
    "[Portable Document Format (PDF)](https://en.wikipedia.org/wiki/PDF), standardized as ISO 32000, is a file format developed by Adobe in 1992 to present documents, including text formatting and images, in a manner independent of application software, hardware, and operating systems.\n",
    "\n",
    "This guide covers how to load `PDF` documents into the LangChain [Document](https://api.python.langchain.com/en/latest/documents/langchain_core.documents.base.Document.html#langchain_core.documents.base.Document) format that we use downstream.\n",
    "\n",
    "LangChain integrates with a host of PDF parsers. Some are simple and relatively low-level; others will support OCR and image-processing, or perform advanced document layout analysis. The right choice will depend on your application. Below we enumerate the possibilities.\n",
    "\n",
    "## Using PyPDF\n",
    "\n",
    "Here we load a PDF using `pypdf` into array of documents, where each document contains the page content and metadata with `page` number."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "35c08d82-8b0a-45e2-8167-73e70f88208a",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install pypdf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "7d8ccd0b-8415-4916-af32-0e6d30b9496b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Document(page_content='LayoutParser : A Uniﬁed Toolkit for Deep\\nLearning Based Document Image Analysis\\nZejiang Shen1( \\x00), Ruochen Zhang2, Melissa Dell3, Benjamin Charles Germain\\nLee4, Jacob Carlson3, and Weining Li5\\n1Allen Institute for AI\\nshannons@allenai.org\\n2Brown University\\nruochen zhang@brown.edu\\n3Harvard University\\n{melissadell,jacob carlson }@fas.harvard.edu\\n4University of Washington\\nbcgl@cs.washington.edu\\n5University of Waterloo\\nw422li@uwaterloo.ca\\nAbstract. Recent advances in document image analysis (DIA) have been\\nprimarily driven by the application of neural networks. Ideally, research\\noutcomes could be easily deployed in production and extended for further\\ninvestigation. However, various factors like loosely organized codebases\\nand sophisticated model conﬁgurations complicate the easy reuse of im-\\nportant innovations by a wide audience. Though there have been on-going\\neﬀorts to improve reusability and simplify deep learning (DL) model\\ndevelopment in disciplines like natural language processing and computer\\nvision, none of them are optimized for challenges in the domain of DIA.\\nThis represents a major gap in the existing toolkit, as DIA is central to\\nacademic research across a wide range of disciplines in the social sciences\\nand humanities. This paper introduces LayoutParser , an open-source\\nlibrary for streamlining the usage of DL in DIA research and applica-\\ntions. The core LayoutParser library comes with a set of simple and\\nintuitive interfaces for applying and customizing DL models for layout de-\\ntection, character recognition, and many other document processing tasks.\\nTo promote extensibility, LayoutParser also incorporates a community\\nplatform for sharing both pre-trained models and full document digiti-\\nzation pipelines. We demonstrate that LayoutParser is helpful for both\\nlightweight and large-scale digitization pipelines in real-word use cases.\\nThe library is publicly available at https://layout-parser.github.io .\\nKeywords: Document Image Analysis ·Deep Learning ·Layout Analysis\\n·Character Recognition ·Open Source library ·Toolkit.\\n1 Introduction\\nDeep Learning(DL)-based approaches are the state-of-the-art for a wide range of\\ndocument image analysis (DIA) tasks including document image classiﬁcation [ 11,arXiv:2103.15348v2  [cs.CV]  21 Jun 2021', metadata={'source': '../../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf', 'page': 0})"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_community.document_loaders import PyPDFLoader\n",
    "\n",
    "file_path = (\n",
    "    \"../../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf\"\n",
    ")\n",
    "loader = PyPDFLoader(file_path)\n",
    "pages = loader.load_and_split()\n",
    "\n",
    "pages[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78ce6d1d-86cc-45e3-8259-e21fbd2c7e6c",
   "metadata": {},
   "source": [
    "An advantage of this approach is that documents can be retrieved with page numbers.\n",
    "\n",
    "### Vector search over PDFs\n",
    "\n",
    "Once we have loaded PDFs into LangChain `Document` objects, we can index them (e.g., a RAG application) in the usual way:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c3b932bb",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install faiss-cpu \n",
    "# use `pip install faiss-gpu` for CUDA GPU support"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7ba35f1c-0a85-4f2f-a56e-3a994c69180d",
   "metadata": {},
   "outputs": [],
   "source": [
    "import getpass\n",
    "import os\n",
    "\n",
    "os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"OpenAI API Key:\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e0eaec77-f5cf-4172-8e39-41e1520eabba",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "13: 14 Z. Shen et al.\n",
      "6 Conclusion\n",
      "LayoutParser provides a comprehensive toolkit for deep learning-based document\n",
      "image analysis. The oﬀ-the-shelf library is easy to install, and can be used to\n",
      "build ﬂexible and accurate pipelines for processing documents with complicated\n",
      "structures. It also supports hi\n",
      "0: LayoutParser : A Uniﬁed Toolkit for Deep\n",
      "Learning Based Document Image Analysis\n",
      "Zejiang Shen1( \u0000), Ruochen Zhang2, Melissa Dell3, Benjamin Charles Germain\n",
      "Lee4, Jacob Carlson3, and Weining Li5\n",
      "1Allen Institute for AI\n",
      "shannons@allenai.org\n",
      "2Brown University\n",
      "ruochen zhang@brown.edu\n",
      "3Harvard University\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from langchain_community.vectorstores import FAISS\n",
    "from langchain_openai import OpenAIEmbeddings\n",
    "\n",
    "faiss_index = FAISS.from_documents(pages, OpenAIEmbeddings())\n",
    "docs = faiss_index.similarity_search(\"What is LayoutParser?\", k=2)\n",
    "for doc in docs:\n",
    "    print(str(doc.metadata[\"page\"]) + \":\", doc.page_content[:300])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ac123ca-386f-4b06-b3a7-9205ea3d6da7",
   "metadata": {},
   "source": [
    "### Extract text from images\n",
    "\n",
    "Some PDFs contain images of text-- e.g., within scanned documents, or figures. Using the `rapidocr-onnxruntime` package we can extract images as text as well:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "347f67fb-67f3-4be7-9af3-23a73cf00f71",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install rapidocr-onnxruntime"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "babc138a-2188-49f7-a8d6-3570fa3ad802",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'LayoutParser : A Uniﬁed Toolkit for DL-Based DIA 5\\nTable 1: Current layout detection models in the LayoutParser model zoo\\nDataset Base Model1Large Model Notes\\nPubLayNet [38] F / M M Layouts of modern scientiﬁc documents\\nPRImA [3] M - Layouts of scanned modern magazines and scientiﬁc reports\\nNewspaper [17] F - Layouts of scanned US newspapers from the 20th century\\nTableBank [18] F F Table region on modern scientiﬁc and business document\\nHJDataset [31] F / M - Layouts of history Japanese documents\\n1For each dataset, we train several models of diﬀerent sizes for diﬀerent needs (the trade-oﬀ between accuracy\\nvs. computational cost). For “base model” and “large model”, we refer to using the ResNet 50 or ResNet 101\\nbackbones [ 13], respectively. One can train models of diﬀerent architectures, like Faster R-CNN [ 28] (F) and Mask\\nR-CNN [ 12] (M). For example, an F in the Large Model column indicates it has a Faster R-CNN model trained\\nusing the ResNet 101 backbone. The platform is maintained and a number of additions will be made to the model\\nzoo in coming months.\\nlayout data structures , which are optimized for eﬃciency and versatility. 3) When\\nnecessary, users can employ existing or customized OCR models via the uniﬁed\\nAPI provided in the OCR module . 4)LayoutParser comes with a set of utility\\nfunctions for the visualization and storage of the layout data. 5) LayoutParser\\nis also highly customizable, via its integration with functions for layout data\\nannotation and model training . We now provide detailed descriptions for each\\ncomponent.\\n3.1 Layout Detection Models\\nInLayoutParser , a layout model takes a document image as an input and\\ngenerates a list of rectangular boxes for the target content regions. Diﬀerent\\nfrom traditional methods, it relies on deep convolutional neural networks rather\\nthan manually curated rules to identify content regions. It is formulated as an\\nobject detection problem and state-of-the-art models like Faster R-CNN [ 28] and\\nMask R-CNN [ 12] are used. This yields prediction results of high accuracy and\\nmakes it possible to build a concise, generalized interface for layout detection.\\nLayoutParser , built upon Detectron2 [ 35], provides a minimal API that can\\nperform layout detection with only four lines of code in Python:\\n1import layoutparser as lp\\n2image = cv2. imread (\" image_file \") # load images\\n3model = lp. Detectron2LayoutModel (\\n4 \"lp :// PubLayNet / faster_rcnn_R_50_FPN_3x / config \")\\n5layout = model . detect ( image )\\nLayoutParser provides a wealth of pre-trained model weights using various\\ndatasets covering diﬀerent languages, time periods, and document types. Due to\\ndomain shift [ 7], the prediction performance can notably drop when models are ap-\\nplied to target samples that are signiﬁcantly diﬀerent from the training dataset. As\\ndocument structures and layouts vary greatly in diﬀerent domains, it is important\\nto select models trained on a dataset similar to the test samples. A semantic syntax\\nis used for initializing the model weights in LayoutParser , using both the dataset\\nname and model name lp://<dataset-name>/<model-architecture-name> .'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loader = PyPDFLoader(\"https://arxiv.org/pdf/2103.15348.pdf\", extract_images=True)\n",
    "pages = loader.load()\n",
    "pages[4].page_content"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eaf6c92e-ad2f-4157-ad35-9a2dc4dd1b66",
   "metadata": {},
   "source": [
    "## Using PyMuPDF\n",
    "\n",
    "This is the fastest of the PDF parsing options, and contains detailed metadata about the PDF and its pages, as well as returns one document per page."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1be9463c-e08b-432e-be46-dc41f6d0ec28",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders import PyMuPDFLoader\n",
    "\n",
    "loader = PyMuPDFLoader(\"example_data/layout-parser-paper.pdf\")\n",
    "data = loader.load()\n",
    "data[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7839a181-f042-4b30-a31f-4ae8631fba42",
   "metadata": {},
   "source": [
    "Additionally, you can pass along any of the options from the [PyMuPDF documentation](https://pymupdf.readthedocs.io/en/latest/app1.html#plain-text/) as keyword arguments in the `load` call, and it will be pass along to the `get_text()` call.\n",
    "\n",
    "## Using MathPix\n",
    "\n",
    "Inspired by Daniel Gross's [https://gist.github.com/danielgross/3ab4104e14faccc12b49200843adab21](https://gist.github.com/danielgross/3ab4104e14faccc12b49200843adab21)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b5f17610-2b24-43a0-908b-8144a5a79916",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders import MathpixPDFLoader\n",
    "\n",
    "file_path = (\n",
    "    \"../../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf\"\n",
    ")\n",
    "loader = MathpixPDFLoader(file_path)\n",
    "data = loader.load()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17c40629-09b8-42d0-a3de-3a43939c4cd8",
   "metadata": {},
   "source": [
    "## Using Unstructured\n",
    "\n",
    "[Unstructured](https://unstructured-io.github.io/unstructured/) supports a common interface for working with unstructured or semi-structured file formats, such as Markdown or PDF. LangChain's [UnstructuredPDFLoader](https://api.python.langchain.com/en/latest/document_loaders/langchain_community.document_loaders.pdf.UnstructuredPDFLoader.html) integrates with Unstructured to parse PDF documents into LangChain [Document](https://api.python.langchain.com/en/latest/documents/langchain_core.documents.base.Document.html) objects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "c6a15bd3-aaa4-49dc-935a-f18617a7dbdd",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders import UnstructuredPDFLoader\n",
    "\n",
    "file_path = (\n",
    "    \"../../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf\"\n",
    ")\n",
    "loader = UnstructuredPDFLoader(file_path)\n",
    "data = loader.load()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4263ba1f-4ccc-413c-9644-46a3ab3ae6fb",
   "metadata": {},
   "source": [
    "### Retain Elements\n",
    "\n",
    "Under the hood, Unstructured creates different \"elements\" for different chunks of text. By default we combine those together, but you can easily keep that separation by specifying `mode=\"elements\"`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "efd80620-0bb8-4298-ab3b-07d7ef9c0085",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Document(page_content='1 2 0 2', metadata={'source': '../../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf', 'coordinates': {'points': ((16.34, 213.36), (16.34, 253.36), (36.34, 253.36), (36.34, 213.36)), 'system': 'PixelSpace', 'layout_width': 612, 'layout_height': 792}, 'file_directory': '../../../docs/integrations/document_loaders/example_data', 'filename': 'layout-parser-paper.pdf', 'languages': ['eng'], 'last_modified': '2024-03-18T13:22:22', 'page_number': 1, 'filetype': 'application/pdf', 'category': 'UncategorizedText'})"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "file_path = (\n",
    "    \"../../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf\"\n",
    ")\n",
    "loader = UnstructuredPDFLoader(file_path, mode=\"elements\")\n",
    "\n",
    "data = loader.load()\n",
    "data[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b269d2a-2385-48a0-95c0-07202e1dff5f",
   "metadata": {},
   "source": [
    "See the full set of element types for this particular document:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "3c40d9e8-5bf7-466d-b2bb-ce2ae08bea35",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ListItem', 'NarrativeText', 'Title', 'UncategorizedText'}"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "set(doc.metadata[\"category\"] for doc in data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90fa9e65-6b00-456c-a0ee-23056f7dacdf",
   "metadata": {},
   "source": [
    "### Fetching remote PDFs using Unstructured\n",
    "\n",
    "This covers how to load online PDFs into a document format that we can use downstream. This can be used for various online PDF sites such as https://open.umn.edu/opentextbooks/textbooks/ and https://arxiv.org/archive/\n",
    "\n",
    "Note: all other PDF loaders can also be used to fetch remote PDFs, but `OnlinePDFLoader` is a legacy function, and works specifically with `UnstructuredPDFLoader`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "54737607-072e-4eb9-aac8-6615472fefc1",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders import OnlinePDFLoader\n",
    "\n",
    "loader = OnlinePDFLoader(\"https://arxiv.org/pdf/2302.03803.pdf\")\n",
    "data = loader.load()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c7199f9-bbc5-4b03-873a-3d54c1bf4f68",
   "metadata": {},
   "source": [
    "## Using PyPDFium2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f209821b-1fe9-402b-adf7-d472c8a24939",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders import PyPDFium2Loader\n",
    "\n",
    "file_path = (\n",
    "    \"../../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf\"\n",
    ")\n",
    "loader = PyPDFium2Loader(file_path)\n",
    "data = loader.load()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "885a8c0e-25e4-4f3b-bb84-9db3f2c9367d",
   "metadata": {},
   "source": [
    "## Using PDFMiner"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4f465592-15be-4b8f-8f8c-0ffe207d0e4d",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders import PDFMinerLoader\n",
    "\n",
    "file_path = (\n",
    "    \"../../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf\"\n",
    ")\n",
    "loader = PDFMinerLoader(file_path)\n",
    "data = loader.load()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9345c37-b0ba-4803-813c-f1c344a90a7c",
   "metadata": {},
   "source": [
    "### Using PDFMiner to generate HTML text\n",
    "\n",
    "This can be helpful for chunking texts semantically into sections as the output html content can be parsed via `BeautifulSoup` to get more structured and rich information about font size, page numbers, PDF headers/footers, etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "2d39159e-61a5-4ac2-a6c2-3981c3aa6f4d",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders import PDFMinerPDFasHTMLLoader\n",
    "\n",
    "file_path = (\n",
    "    \"../../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf\"\n",
    ")\n",
    "loader = PDFMinerPDFasHTMLLoader(file_path)\n",
    "data = loader.load()[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "2f18fc1e-988f-4778-ab79-4fac739bec8f",
   "metadata": {},
   "outputs": [],
   "source": [
    "from bs4 import BeautifulSoup\n",
    "\n",
    "soup = BeautifulSoup(data.page_content, \"html.parser\")\n",
    "content = soup.find_all(\"div\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "0b40f5bd-631e-4444-b79e-ef55e088807e",
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "\n",
    "cur_fs = None\n",
    "cur_text = \"\"\n",
    "snippets = []  # first collect all snippets that have the same font size\n",
    "for c in content:\n",
    "    sp = c.find(\"span\")\n",
    "    if not sp:\n",
    "        continue\n",
    "    st = sp.get(\"style\")\n",
    "    if not st:\n",
    "        continue\n",
    "    fs = re.findall(\"font-size:(\\d+)px\", st)\n",
    "    if not fs:\n",
    "        continue\n",
    "    fs = int(fs[0])\n",
    "    if not cur_fs:\n",
    "        cur_fs = fs\n",
    "    if fs == cur_fs:\n",
    "        cur_text += c.text\n",
    "    else:\n",
    "        snippets.append((cur_text, cur_fs))\n",
    "        cur_fs = fs\n",
    "        cur_text = c.text\n",
    "snippets.append((cur_text, cur_fs))\n",
    "# Note: The above logic is very straightforward. One can also add more strategies such as removing duplicate snippets (as\n",
    "# headers/footers in a PDF appear on multiple pages so if we find duplicates it's safe to assume that it is redundant info)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "953b168f-4ae1-4279-b370-c21961206c0a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.documents import Document\n",
    "\n",
    "cur_idx = -1\n",
    "semantic_snippets = []\n",
    "# Assumption: headings have higher font size than their respective content\n",
    "for s in snippets:\n",
    "    # if current snippet's font size > previous section's heading => it is a new heading\n",
    "    if (\n",
    "        not semantic_snippets\n",
    "        or s[1] > semantic_snippets[cur_idx].metadata[\"heading_font\"]\n",
    "    ):\n",
    "        metadata = {\"heading\": s[0], \"content_font\": 0, \"heading_font\": s[1]}\n",
    "        metadata.update(data.metadata)\n",
    "        semantic_snippets.append(Document(page_content=\"\", metadata=metadata))\n",
    "        cur_idx += 1\n",
    "        continue\n",
    "\n",
    "    # if current snippet's font size <= previous section's content => content belongs to the same section (one can also create\n",
    "    # a tree like structure for sub sections if needed but that may require some more thinking and may be data specific)\n",
    "    if (\n",
    "        not semantic_snippets[cur_idx].metadata[\"content_font\"]\n",
    "        or s[1] <= semantic_snippets[cur_idx].metadata[\"content_font\"]\n",
    "    ):\n",
    "        semantic_snippets[cur_idx].page_content += s[0]\n",
    "        semantic_snippets[cur_idx].metadata[\"content_font\"] = max(\n",
    "            s[1], semantic_snippets[cur_idx].metadata[\"content_font\"]\n",
    "        )\n",
    "        continue\n",
    "\n",
    "    # if current snippet's font size > previous section's content but less than previous section's heading than also make a new\n",
    "    # section (e.g. title of a PDF will have the highest font size but we don't want it to subsume all sections)\n",
    "    metadata = {\"heading\": s[0], \"content_font\": 0, \"heading_font\": s[1]}\n",
    "    metadata.update(data.metadata)\n",
    "    semantic_snippets.append(Document(page_content=\"\", metadata=metadata))\n",
    "    cur_idx += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "9bf28b73-dad4-4f51-9238-4af523fa7225",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Document(page_content='Recently, various DL models and datasets have been developed for layout analysis\\ntasks. The dhSegment [22] utilizes fully convolutional networks [20] for segmen-\\ntation tasks on historical documents. Object detection-based methods like Faster\\nR-CNN [28] and Mask R-CNN [12] are used for identifying document elements [38]\\nand detecting tables [30, 26]. Most recently, Graph Neural Networks [29] have also\\nbeen used in table detection [27]. However, these models are usually implemented\\nindividually and there is no uniﬁed framework to load and use such models.\\nThere has been a surge of interest in creating open-source tools for document\\nimage processing: a search of document image analysis in Github leads to 5M\\nrelevant code pieces 6; yet most of them rely on traditional rule-based methods\\nor provide limited functionalities. The closest prior research to our work is the\\nOCR-D project7, which also tries to build a complete toolkit for DIA. However,\\nsimilar to the platform developed by Neudecker et al. [21], it is designed for\\nanalyzing historical documents, and provides no supports for recent DL models.\\nThe DocumentLayoutAnalysis project8 focuses on processing born-digital PDF\\ndocuments via analyzing the stored PDF data. Repositories like DeepLayout9\\nand Detectron2-PubLayNet10 are individual deep learning models trained on\\nlayout analysis datasets without support for the full DIA pipeline. The Document\\nAnalysis and Exploitation (DAE) platform [15] and the DeepDIVA project [2]\\naim to improve the reproducibility of DIA methods (or DL models), yet they\\nare not actively maintained. OCR engines like Tesseract [14], easyOCR11 and\\npaddleOCR12 usually do not come with comprehensive functionalities for other\\nDIA tasks like layout analysis.\\nRecent years have also seen numerous eﬀorts to create libraries for promoting\\nreproducibility and reusability in the ﬁeld of DL. Libraries like Dectectron2 [35],\\n6 The number shown is obtained by specifying the search type as ‘code’.\\n7 https://ocr-d.de/en/about\\n8 https://github.com/BobLd/DocumentLayoutAnalysis\\n9 https://github.com/leonlulu/DeepLayout\\n10 https://github.com/hpanwar08/detectron2\\n11 https://github.com/JaidedAI/EasyOCR\\n12 https://github.com/PaddlePaddle/PaddleOCR\\n4\\nZ. Shen et al.\\nFig. 1: The overall architecture of LayoutParser. For an input document image,\\nthe core LayoutParser library provides a set of oﬀ-the-shelf tools for layout\\ndetection, OCR, visualization, and storage, backed by a carefully designed layout\\ndata structure. LayoutParser also supports high level customization via eﬃcient\\nlayout annotation and model training functions. These improve model accuracy\\non the target samples. The community platform enables the easy sharing of DIA\\nmodels and whole digitization pipelines to promote reusability and reproducibility.\\nA collection of detailed documentation, tutorials and exemplar projects make\\nLayoutParser easy to learn and use.\\nAllenNLP [8] and transformers [34] have provided the community with complete\\nDL-based support for developing and deploying models for general computer\\nvision and natural language processing problems. LayoutParser, on the other\\nhand, specializes speciﬁcally in DIA tasks. LayoutParser is also equipped with a\\ncommunity platform inspired by established model hubs such as Torch Hub [23]\\nand TensorFlow Hub [1]. It enables the sharing of pretrained models as well as\\nfull document processing pipelines that are unique to DIA tasks.\\nThere have been a variety of document data collections to facilitate the\\ndevelopment of DL models. Some examples include PRImA [3](magazine layouts),\\nPubLayNet [38](academic paper layouts), Table Bank [18](tables in academic\\npapers), Newspaper Navigator Dataset [16, 17](newspaper ﬁgure layouts) and\\nHJDataset [31](historical Japanese document layouts). A spectrum of models\\ntrained on these datasets are currently available in the LayoutParser model zoo\\nto support diﬀerent use cases.\\n', metadata={'heading': '2 Related Work\\n', 'content_font': 9, 'heading_font': 11, 'source': '../../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf'})"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "semantic_snippets[4]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e87d7447-c620-4f48-b4fd-8933a614e4e1",
   "metadata": {},
   "source": [
    "## PyPDF Directory\n",
    "\n",
    "Load PDFs from directory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "78e5a485-ff53-4b0c-ba5f-9f442079b529",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders import PyPDFDirectoryLoader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "51b2fe13-3755-4031-b7ce-84d9983db71c",
   "metadata": {},
   "outputs": [],
   "source": [
    "directory_path = \"../../../docs/integrations/document_loaders/example_data/\"\n",
    "loader = PyPDFDirectoryLoader(\"example_data/\")\n",
    "\n",
    "\n",
    "docs = loader.load()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78365a16-c011-4de1-8c32-873b88e7fead",
   "metadata": {},
   "source": [
    "## Using PDFPlumber\n",
    "\n",
    "Like PyMuPDF, the output Documents contain detailed metadata about the PDF and its pages, and returns one document per page."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c8c1001b-48b1-4777-a34f-2fbdca5457df",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders import PDFPlumberLoader\n",
    "\n",
    "data = loader.load()\n",
    "data[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94795ae5-161d-4d64-963c-dbcf1e60ca15",
   "metadata": {},
   "source": [
    "## Using AmazonTextractPDFParser\n",
    "\n",
    "The AmazonTextractPDFLoader calls the [Amazon Textract Service](https://aws.amazon.com/textract/) to convert PDFs into a Document structure. The loader does pure OCR at the moment, with more features like layout support planned, depending on demand.  Single and multi-page documents are supported with up to 3000 pages and 512 MB of size.\n",
    "\n",
    "For the call to be successful an AWS account is required, similar to the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html) requirements.\n",
    "\n",
    "Besides the AWS configuration, it is very similar to the other PDF loaders, while also supporting JPEG, PNG and TIFF and non-native PDF formats."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5329e301-4bb6-4d51-aced-c9984ff6808a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders import AmazonTextractPDFLoader\n",
    "\n",
    "loader = AmazonTextractPDFLoader(\"example_data/alejandro_rosalez_sample-small.jpeg\")\n",
    "documents = loader.load()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8291366-e2ec-4460-8e97-3fae3971986e",
   "metadata": {},
   "source": [
    "## Using AzureAIDocumentIntelligenceLoader\n",
    "\n",
    "[Azure AI Document Intelligence](https://aka.ms/doc-intelligence) (formerly known as `Azure Form Recognizer`) is machine-learning \n",
    "based service that extracts texts (including handwriting), tables, document structures (e.g., titles, section headings, etc.) and key-value-pairs from\n",
    "digital or scanned PDFs, images, Office and HTML files. Document Intelligence supports `PDF`, `JPEG/JPG`, `PNG`, `BMP`, `TIFF`, `HEIF`, `DOCX`, `XLSX`, `PPTX` and `HTML`.\n",
    "\n",
    "This [current implementation](https://aka.ms/di-langchain) of a loader using `Document Intelligence` can incorporate content page-wise and turn it into LangChain documents. The default output format is markdown, which can be easily chained with `MarkdownHeaderTextSplitter` for semantic document chunking. You can also use `mode=\"single\"` or `mode=\"page\"` to return pure texts in a single page or document split by page.\n",
    "\n",
    "### Prerequisite\n",
    "\n",
    "An Azure AI Document Intelligence resource in one of the 3 preview regions: **East US**, **West US2**, **West Europe** - follow [this document](https://learn.microsoft.com/azure/ai-services/document-intelligence/create-document-intelligence-resource?view=doc-intel-4.0.0) to create one if you don't have. You will be passing `<endpoint>` and `<key>` as parameters to the loader."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12dfb5ff-ddd5-40a7-a5db-25d149d556ce",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install --upgrade --quiet  langchain langchain-community azure-ai-documentintelligence"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b06bd5d4-7093-4d12-8963-1eb41f82d21d",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders import AzureAIDocumentIntelligenceLoader\n",
    "\n",
    "file_path = \"<filepath>\"\n",
    "endpoint = \"<endpoint>\"\n",
    "key = \"<key>\"\n",
    "loader = AzureAIDocumentIntelligenceLoader(\n",
    "    api_endpoint=endpoint, api_key=key, file_path=file_path, api_model=\"prebuilt-layout\"\n",
    ")\n",
    "\n",
    "documents = loader.load()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
