vectorstore,    cleanup="incremental",    source_id_key="source",)    {'num_added': 0, 'num_updated': 0, 'num_skipped': 2, 'num_deleted': 0}If we provide no documents with incremental indexing mode, nothing will change.index(    [], record_manager, vectorstore, cleanup="incremental", source_id_key="source")    {'num_added': 0, 'num_updated': 0, 'num_skipped': 0, 'num_deleted': 0}If we mutate a document, the new version will be written and all old versions sharing the same source will be deleted.changed_doc_2 = Document(page_content="puppy", metadata={"source": "doggy.txt"})index(    [changed_doc_2],    record_manager,    vectorstore,    cleanup="incremental",    source_id_key="source",)    {'num_added': 1, 'num_updated': 0, 'num_skipped': 0, 'num_deleted': 1}"full" deletion mode‚ÄãIn full mode the user should pass the full universe of content that should be indexed into the indexing function.Any documents that are not passed into the indexing function and are present in the vectorstore will be deleted!This behavior is useful to handle deletions of source documents._clear()all_docs = [doc1, doc2]index(all_docs, record_manager, vectorstore, cleanup="full", source_id_key="source")    {'num_added': 2, 'num_updated': 0, 'num_skipped': 0, 'num_deleted': 0}Say someone deleted the first doc:del all_docs[0]all_docs    [Document(page_content='doggy', metadata={'source': 'doggy.txt'})]Using full mode will clean up the deleted content as well.index(all_docs, record_manager, vectorstore, cleanup="full", source_id_key="source")    {'num_added': 0, 'num_updated': 0, 'num_skipped': 1, 'num_deleted': 1}Source‚ÄãThe metadata attribute contains a field called source. This source should be pointing at the ultimate provenance associated with the given document.For example, if these documents are representing chunks of some parent document, the source for both documents should be the same and reference the parent document.In general, source should always be specified. Only use a
Here, we will look at a basic indexing workflow using the LangChain indexing API.
Here, we will look at a basic indexing workflow using the LangChain indexing API. ->: vectorstore,    cleanup="incremental",    source_id_key="source",)    {'num_added': 0, 'num_updated': 0, 'num_skipped': 2, 'num_deleted': 0}If we provide no documents with incremental indexing mode, nothing will change.index(    [], record_manager, vectorstore, cleanup="incremental", source_id_key="source")    {'num_added': 0, 'num_updated': 0, 'num_skipped': 0, 'num_deleted': 0}If we mutate a document, the new version will be written and all old versions sharing the same source will be deleted.changed_doc_2 = Document(page_content="puppy", metadata={"source": "doggy.txt"})index(    [changed_doc_2],    record_manager,    vectorstore,    cleanup="incremental",    source_id_key="source",)    {'num_added': 1, 'num_updated': 0, 'num_skipped': 0, 'num_deleted': 1}"full" deletion mode‚ÄãIn full mode the user should pass the full universe of content that should be indexed into the indexing function.Any documents that are not passed into the indexing function and are present in the vectorstore will be deleted!This behavior is useful to handle deletions of source documents._clear()all_docs = [doc1, doc2]index(all_docs, record_manager, vectorstore, cleanup="full", source_id_key="source")    {'num_added': 2, 'num_updated': 0, 'num_skipped': 0, 'num_deleted': 0}Say someone deleted the first doc:del all_docs[0]all_docs    [Document(page_content='doggy', metadata={'source': 'doggy.txt'})]Using full mode will clean up the deleted content as well.index(all_docs, record_manager, vectorstore, cleanup="full", source_id_key="source")    {'num_added': 0, 'num_updated': 0, 'num_skipped': 1, 'num_deleted': 1}Source‚ÄãThe metadata attribute contains a field called source. This source should be pointing at the ultimate provenance associated with the given document.For example, if these documents are representing chunks of some parent document, the source for both documents should be the same and reference the parent document.In general, source should always be specified. Only use a