perform text splitting on the header grouped documents. # Define our text splitterfrom langchain.text_splitter import RecursiveCharacterTextSplitterchunk_size = 500chunk_overlap = 0text_splitter = RecursiveCharacterTextSplitter(    chunk_size=chunk_size, chunk_overlap=chunk_overlap)all_splits = text_splitter.split_documents(md_header_splits)This sets us up well do perform metadata filtering based on the document structure.Let's bring this all together by building a vectorstore first.pip install chromadb# Build vectorstore and keep the metadatafrom langchain.embeddings import OpenAIEmbeddingsfrom langchain.vectorstores import Chromavectorstore = Chroma.from_documents(documents=all_splits, embedding=OpenAIEmbeddings())Let's create a SelfQueryRetriever that can filter based upon metadata we defined.# Create retrieverfrom langchain.llms import OpenAIfrom langchain.retrievers.self_query.base import SelfQueryRetrieverfrom langchain.chains.query_constructor.base import AttributeInfo# Define our metadatametadata_field_info = [    AttributeInfo(        name="Section",        description="Part of the document that the text comes from",        type="string or list[string]",    ),]document_content_description = "Major sections of the document"# Define self query retrieverllm = OpenAI(temperature=0)retriever = SelfQueryRetriever.from_llm(    llm, vectorstore, document_content_description, metadata_field_info, verbose=True)We can see that we can query only for texts in the Introduction of the document!# Testretriever.get_relevant_documents("Summarize the Introduction section of the document")    query='Introduction' filter=Comparison(comparator=<Comparator.EQ: 'eq'>, attribute='Section', value='Introduction') limit=None    [Document(page_content='![Untitled](Auto-Evaluation%20of%20Metadata%20Filtering%2018502448c85240828f33716740f9574b/Untitled.png)', metadata={'Section': 'Introduction'}),     Document(page_content='Q+A systems often use a two-step approach: retrieve relevant
Text splitting for vector storage often uses sentences or other delimiters to keep related text together.
Text splitting for vector storage often uses sentences or other delimiters to keep related text together. ->: perform text splitting on the header grouped documents. # Define our text splitterfrom langchain.text_splitter import RecursiveCharacterTextSplitterchunk_size = 500chunk_overlap = 0text_splitter = RecursiveCharacterTextSplitter(    chunk_size=chunk_size, chunk_overlap=chunk_overlap)all_splits = text_splitter.split_documents(md_header_splits)This sets us up well do perform metadata filtering based on the document structure.Let's bring this all together by building a vectorstore first.pip install chromadb# Build vectorstore and keep the metadatafrom langchain.embeddings import OpenAIEmbeddingsfrom langchain.vectorstores import Chromavectorstore = Chroma.from_documents(documents=all_splits, embedding=OpenAIEmbeddings())Let's create a SelfQueryRetriever that can filter based upon metadata we defined.# Create retrieverfrom langchain.llms import OpenAIfrom langchain.retrievers.self_query.base import SelfQueryRetrieverfrom langchain.chains.query_constructor.base import AttributeInfo# Define our metadatametadata_field_info = [    AttributeInfo(        name="Section",        description="Part of the document that the text comes from",        type="string or list[string]",    ),]document_content_description = "Major sections of the document"# Define self query retrieverllm = OpenAI(temperature=0)retriever = SelfQueryRetriever.from_llm(    llm, vectorstore, document_content_description, metadata_field_info, verbose=True)We can see that we can query only for texts in the Introduction of the document!# Testretriever.get_relevant_documents("Summarize the Introduction section of the document")    query='Introduction' filter=Comparison(comparator=<Comparator.EQ: 'eq'>, attribute='Section', value='Introduction') limit=None    [Document(page_content='![Untitled](Auto-Evaluation%20of%20Metadata%20Filtering%2018502448c85240828f33716740f9574b/Untitled.png)', metadata={'Section': 'Introduction'}),     Document(page_content='Q+A systems often use a two-step approach: retrieve relevant