FAISSfrom langchain.embeddings.openai import OpenAIEmbeddingsfaiss_index = FAISS.from_documents(pages, OpenAIEmbeddings())docs = faiss_index.similarity_search("How will the community be engaged?", k=2)for doc in docs:    print(str(doc.metadata["page"]) + ":", doc.page_content[:300])    9: 10 Z. Shen et al.    Fig. 4: Illustration of (a) the original historical Japanese document with layout    detection results and (b) a recreated version of the document image that achieves    much better character recognition recall. The reorganization algorithm rearranges    the tokens based on the their detect    3: 4 Z. Shen et al.    Efficient Data AnnotationC u s t o m i z e d  M o d e l  T r a i n i n gModel Cust omizationDI A Model HubDI A Pipeline SharingCommunity PlatformLa y out Detection ModelsDocument Images    T h e  C o r e  L a y o u t P a r s e r  L i b r a r yOCR ModuleSt or age & VisualizationLa y ouExtracting images‚ÄãUsing the rapidocr-onnxruntime package we can extract images as text as well:pip install rapidocr-onnxruntimeloader = PyPDFLoader("https://arxiv.org/pdf/2103.15348.pdf", extract_images=True)pages = loader.load()pages[4].page_content'LayoutParser : A UniÔ¨Åed Toolkit for DL-Based DIA 5\nTable 1: Current layout detection models in the LayoutParser model zoo\nDataset Base Model1Large Model Notes\nPubLayNet [38] F / M M Layouts of modern scientiÔ¨Åc documents\nPRImA [3] M - Layouts of scanned modern magazines and scientiÔ¨Åc reports\nNewspaper [17] F - Layouts of scanned US newspapers from the 20th century\nTableBank [18] F F Table region on modern scientiÔ¨Åc and business document\nHJDataset [31] F / M - Layouts of history Japanese documents\n1For each dataset, we train several models of diÔ¨Äerent sizes for diÔ¨Äerent needs (the trade-oÔ¨Ä between accuracy\nvs. computational cost). For ‚Äúbase model‚Äù and ‚Äúlarge model‚Äù, we refer to using the ResNet 50 or ResNet 101\nbackbones [ 13], respectively. One can train models of diÔ¨Äerent architectures,
Portable Document Format (PDF), standardized as ISO 32000, is a file format developed by Adobe in 1992 to present documents, including text formatting and images, in a manner independent of application software, hardware, and operating systems.
Portable Document Format (PDF), standardized as ISO 32000, is a file format developed by Adobe in 1992 to present documents, including text formatting and images, in a manner independent of application software, hardware, and operating systems. ->: FAISSfrom langchain.embeddings.openai import OpenAIEmbeddingsfaiss_index = FAISS.from_documents(pages, OpenAIEmbeddings())docs = faiss_index.similarity_search("How will the community be engaged?", k=2)for doc in docs:    print(str(doc.metadata["page"]) + ":", doc.page_content[:300])    9: 10 Z. Shen et al.    Fig. 4: Illustration of (a) the original historical Japanese document with layout    detection results and (b) a recreated version of the document image that achieves    much better character recognition recall. The reorganization algorithm rearranges    the tokens based on the their detect    3: 4 Z. Shen et al.    Efficient Data AnnotationC u s t o m i z e d  M o d e l  T r a i n i n gModel Cust omizationDI A Model HubDI A Pipeline SharingCommunity PlatformLa y out Detection ModelsDocument Images    T h e  C o r e  L a y o u t P a r s e r  L i b r a r yOCR ModuleSt or age & VisualizationLa y ouExtracting images‚ÄãUsing the rapidocr-onnxruntime package we can extract images as text as well:pip install rapidocr-onnxruntimeloader = PyPDFLoader("https://arxiv.org/pdf/2103.15348.pdf", extract_images=True)pages = loader.load()pages[4].page_content'LayoutParser : A UniÔ¨Åed Toolkit for DL-Based DIA 5\nTable 1: Current layout detection models in the LayoutParser model zoo\nDataset Base Model1Large Model Notes\nPubLayNet [38] F / M M Layouts of modern scientiÔ¨Åc documents\nPRImA [3] M - Layouts of scanned modern magazines and scientiÔ¨Åc reports\nNewspaper [17] F - Layouts of scanned US newspapers from the 20th century\nTableBank [18] F F Table region on modern scientiÔ¨Åc and business document\nHJDataset [31] F / M - Layouts of history Japanese documents\n1For each dataset, we train several models of diÔ¨Äerent sizes for diÔ¨Äerent needs (the trade-oÔ¨Ä between accuracy\nvs. computational cost). For ‚Äúbase model‚Äù and ‚Äúlarge model‚Äù, we refer to using the ResNet 50 or ResNet 101\nbackbones [ 13], respectively. One can train models of diÔ¨Äerent architectures,