AI, Rag, Vector, Deterministic Retrieval.
split, chunk, and create embeddings of the document before upserting into the vector databaseindex. We’re leveraging Langchain’s PyPDFLoader and RecursiveCharacterTextSplitter for pdf processing, metadata extraction, and chunking. For embedding, we’re using the OpenAI text-embedding-3-small model.