Skip to main contentü¶úÔ∏èüîó LangChainDocsUse casesIntegrationsAPICommunityChat our docsLangSmithJS/TS DocsSearchCTRLKProvidersAnthropicAWSGoogleMicrosoftOpenAIMoreComponentsLLMsChat modelsDocument loadersDocument transformersText embedding modelsVector storesRetrieversAmazon KendraArcee RetrieverArxivAzure Cognitive SearchBM25ChaindeskChatGPT PluginCohere RerankerDocArrayElasticSearch BM25Google Cloud Enterprise SearchGoogle DriveGoogle Vertex AI SearchKay.aikNNLOTR (Merger Retriever)MetalPinecone Hybrid SearchPubMedRePhraseQuerySEC filingSelf-querying retrieverSVMTavily Search APITF-IDFVespaWeaviate Hybrid SearchWikipediayou-retrieverZepToolsAgents and toolkitsMemoryCallbacksChat loadersComponentsRetrieversTF-IDFOn this pageTF-IDFTF-IDF means term-frequency times inverse document-frequency.This notebook goes over how to use a retriever that under the hood uses TF-IDF using scikit-learn package.For more information on the details of TF-IDF see this blog post.# !pip install scikit-learnfrom langchain.retrievers import TFIDFRetrieverCreate New Retriever with Texts‚Äãretriever = TFIDFRetriever.from_texts(["foo", "bar", "world", "hello", "foo bar"])Create a New Retriever with Documents‚ÄãYou can now create a new retriever with the documents you created.from langchain.schema import Documentretriever = TFIDFRetriever.from_documents(    [        Document(page_content="foo"),        Document(page_content="bar"),        Document(page_content="world"),        Document(page_content="hello"),        Document(page_content="foo bar"),    ])Use Retriever‚ÄãWe can now use the retriever!result = retriever.get_relevant_documents("foo")result    [Document(page_content='foo', metadata={}),     Document(page_content='foo bar', metadata={}),     Document(page_content='hello', metadata={}),     Document(page_content='world', metadata={})]Save and load‚ÄãYou can easily save and load this retriever, making it handy for local development!retriever.save_local("testing.pkl")retriever_copy
TF-IDF means term-frequency times inverse document-frequency.
TF-IDF means term-frequency times inverse document-frequency. ->: Skip to main contentü¶úÔ∏èüîó LangChainDocsUse casesIntegrationsAPICommunityChat our docsLangSmithJS/TS DocsSearchCTRLKProvidersAnthropicAWSGoogleMicrosoftOpenAIMoreComponentsLLMsChat modelsDocument loadersDocument transformersText embedding modelsVector storesRetrieversAmazon KendraArcee RetrieverArxivAzure Cognitive SearchBM25ChaindeskChatGPT PluginCohere RerankerDocArrayElasticSearch BM25Google Cloud Enterprise SearchGoogle DriveGoogle Vertex AI SearchKay.aikNNLOTR (Merger Retriever)MetalPinecone Hybrid SearchPubMedRePhraseQuerySEC filingSelf-querying retrieverSVMTavily Search APITF-IDFVespaWeaviate Hybrid SearchWikipediayou-retrieverZepToolsAgents and toolkitsMemoryCallbacksChat loadersComponentsRetrieversTF-IDFOn this pageTF-IDFTF-IDF means term-frequency times inverse document-frequency.This notebook goes over how to use a retriever that under the hood uses TF-IDF using scikit-learn package.For more information on the details of TF-IDF see this blog post.# !pip install scikit-learnfrom langchain.retrievers import TFIDFRetrieverCreate New Retriever with Texts‚Äãretriever = TFIDFRetriever.from_texts(["foo", "bar", "world", "hello", "foo bar"])Create a New Retriever with Documents‚ÄãYou can now create a new retriever with the documents you created.from langchain.schema import Documentretriever = TFIDFRetriever.from_documents(    [        Document(page_content="foo"),        Document(page_content="bar"),        Document(page_content="world"),        Document(page_content="hello"),        Document(page_content="foo bar"),    ])Use Retriever‚ÄãWe can now use the retriever!result = retriever.get_relevant_documents("foo")result    [Document(page_content='foo', metadata={}),     Document(page_content='foo bar', metadata={}),     Document(page_content='hello', metadata={}),     Document(page_content='world', metadata={})]Save and load‚ÄãYou can easily save and load this retriever, making it handy for local development!retriever.save_local("testing.pkl")retriever_copy